Image de couverture de livre

Scaling Laws for Data Filtering -- Data Curation cannot be Compute Agnostic

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Goyal, Sachin, Maini, Pratyush, Lipton, Zachary C., Raghunathan, Aditi, Kolter, J. Zico
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

T-MARS: Improving Visual Representations by Circumventing Text Feature Learning
par: Maini, Pratyush, et autres
Publié: (2023)

Understanding Hallucinations in Diffusion Models through Mode Interpolation
par: Aithal, Sumukh K, et autres
Publié: (2024)

Context-Parametric Inversion: Why Instruction Finetuning Can Worsen Context Reliance
par: Goyal, Sachin, et autres
Publié: (2024)

When Should We Introduce Safety Interventions During Pretraining?
par: Sam, Dylan, et autres
Publié: (2026)

TOFU: A Task of Fictitious Unlearning for LLMs
par: Maini, Pratyush, et autres
Publié: (2024)

Rethinking LLM Memorization through the Lens of Adversarial Compression
par: Schwarzschild, Avi, et autres
Publié: (2024)

Why is SAM Robust to Label Noise?
par: Baek, Christina, et autres
Publié: (2024)

Safety Pretraining: Toward the Next Generation of Safe AI
par: Maini, Pratyush, et autres
Publié: (2025)

Memorization Sinks: Isolating Memorization during LLM Training
par: Ghosal, Gaurav R., et autres
Publié: (2025)

Peeking Behind Closed Doors: Risks of LLM Evaluation by Private Data Curators
par: Bansal, Hritik, et autres
Publié: (2025)

Adaptive Data Optimization: Dynamic Sample Selection with Scaling Laws
par: Jiang, Yiding, et autres
Publié: (2024)

Mode-Conditioning Unlocks Superior Test-Time Scaling
par: Wu, Chen Henry, et autres
Publié: (2025)

Weight Ensembling Improves Reasoning in Language Models
par: Dang, Xingyu, et autres
Publié: (2025)

Test-Time Adaptation Induces Stronger Accuracy and Agreement-on-the-Line
par: Kim, Eungyeup, et autres
Publié: (2023)

Predicting the Performance of Foundation Models via Agreement-on-the-Line
par: Saxena, Rahul, et autres
Publié: (2024)

Base Models Look Human To AI Detectors
par: Xu, Yixuan Even, et autres
Publié: (2026)

An Axiomatic Approach to Model-Agnostic Concept Explanations
par: Feng, Zhili, et autres
Publié: (2024)

Inference Optimal VLMs Need Fewer Visual Tokens and More Parameters
par: Li, Kevin Y., et autres
Publié: (2024)

AcceleratedLiNGAM: Learning Causal DAGs at the speed of GPUs
par: Akinwande, Victor, et autres
Publié: (2024)

Sharpness-Aware Pretraining Mitigates Catastrophic Forgetting
par: Watts, Ishaan, et autres
Publié: (2026)

Unlocking Post-hoc Dataset Inference with Synthetic Data
par: Zhao, Bihe, et autres
Publié: (2025)

Exact Unlearning of Finetuning Data via Model Merging at Scale
par: Kuo, Kevin, et autres
Publié: (2025)

Scaling Laws for Precision
par: Kumar, Tanishq, et autres
Publié: (2024)

FUSE-ing Language Models: Zero-Shot Adapter Discovery for Prompt Optimization Across Tokenizers
par: Williams, Joshua Nathaniel, et autres
Publié: (2024)

Equilibrium Reasoners: Learning Attractors Enables Scalable Reasoning
par: Huang, Benhao, et autres
Publié: (2026)

Mimetic Initialization of MLPs
par: Trockman, Asher, et autres
Publié: (2026)

OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metrics
par: Dorna, Vineeth, et autres
Publié: (2025)

Online Data Collection for Efficient Semiparametric Inference
par: Gupta, Shantanu, et autres
Publié: (2024)

Predicting the Performance of Black-box LLMs through Follow-up Queries
par: Sam, Dylan, et autres
Publié: (2025)

Distilled Pretraining: A modern lens of Data, In-Context Learning and Test-Time Scaling
par: Goyal, Sachin, et autres
Publié: (2025)

Compute-Optimal LLMs Provably Generalize Better With Scale
par: Finzi, Marc, et autres
Publié: (2025)

From Variance to Veracity: Unbundling and Mitigating Gradient Variance in Differentiable Bundle Adjustment Layers
par: Gurumurthy, Swaminathan, et autres
Publié: (2024)

Diffusing Differentiable Representations
par: Savani, Yash, et autres
Publié: (2024)

One-Step Diffusion Distillation via Deep Equilibrium Models
par: Geng, Zhengyang, et autres
Publié: (2023)

Multimodal Data Curation Through Ranked Retrieval
par: Muthukumar, Pratyush, et autres
Publié: (2026)

Measuring Five-Nines Reliability: Sample-Efficient LLM Evaluation in Saturated Benchmarks
par: Kim, Eungyeup, et autres
Publié: (2026)

Generative Posterior Networks for Approximately Bayesian Epistemic Uncertainty Estimation
par: Roderick, Melrose, et autres
Publié: (2023)

From Entropy to Epiplexity: Rethinking Information for Computationally Bounded Intelligence
par: Finzi, Marc, et autres
Publié: (2026)

Rethinking Distance Metrics for Counterfactual Explainability
par: Williams, Joshua Nathaniel, et autres
Publié: (2024)

Massive Activations in Large Language Models
par: Sun, Mingjie, et autres
Publié: (2024)