Cover Image

A Bitter Lesson for Data Filtering

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Mohri, Christopher, Duchi, John, Hashimoto, Tatsunori
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Language Models with Conformal Factuality Guarantees
by: Mohri, Christopher, et al.
Published: (2024)

Evaluating Self-Supervised Learning via Risk Decomposition
by: Dubois, Yann, et al.
Published: (2023)

Observational Scaling Laws and the Predictability of Language Model Performance
by: Ruan, Yangjun, et al.
Published: (2024)

Linguistic Calibration of Long-Form Generations
by: Band, Neil, et al.
Published: (2024)

Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators
by: Dubois, Yann, et al.
Published: (2024)

Synthetic Data for any Differentiable Target
by: Thrush, Tristan, et al.
Published: (2026)

Trustless Audits without Revealing Data or Models
by: Waiwitlikhit, Suppakit, et al.
Published: (2024)

Reasoning to Learn from Latent Thoughts
by: Ruan, Yangjun, et al.
Published: (2025)

Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers
by: Si, Chenglei, et al.
Published: (2024)

Agentic Adversarial QA for Improving Domain-Specific LLMs
by: Grari, Vincent, et al.
Published: (2026)

Synthetic continued pretraining
by: Yang, Zitong, et al.
Published: (2024)

Graph-based Uncertainty Metrics for Long-form Language Model Outputs
by: Jiang, Mingjian, et al.
Published: (2024)

Algorithms for Learning Kernels Based on Centered Alignment
by: Cortes, Corinna, et al.
Published: (2012)

The Ideation-Execution Gap: Execution Outcomes of LLM-Generated versus Human Research Ideas
by: Si, Chenglei, et al.
Published: (2025)

Towards Execution-Grounded Automated AI Research
by: Si, Chenglei, et al.
Published: (2026)

Eliciting Language Model Behaviors with Investigator Agents
by: Li, Xiang Lisa, et al.
Published: (2025)

Neural Bayesian Filtering
by: Solinas, Christopher, et al.
Published: (2025)

AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback
by: Dubois, Yann, et al.
Published: (2023)

Improving Pretraining Data Using Perplexity Correlations
by: Thrush, Tristan, et al.
Published: (2024)

Kalman Filter for Online Classification of Non-Stationary Data
by: Titsias, Michalis K., et al.
Published: (2023)

Long-term Safe Reinforcement Learning with Binary Feedback
by: Wachi, Akifumi, et al.
Published: (2024)

One Filters All: A Generalist Filter for State Estimation
by: Liu, Shiqi, et al.
Published: (2025)

Offline vs. Online Learning in Model-based RL: Lessons for Data Collection Strategies
by: Chen, Jiaqi, et al.
Published: (2025)

Identifying the Risks of LM Agents with an LM-Emulated Sandbox
by: Ruan, Yangjun, et al.
Published: (2023)

Contrastive Representation for Data Filtering in Cross-Domain Offline Reinforcement Learning
by: Wen, Xiaoyu, et al.
Published: (2024)

Computation-Aware Kalman Filtering with Model Selection for Neural Dynamics
by: Huml, JR, et al.
Published: (2026)

Improving SMOTE via Fusing Conditional VAE for Data-adaptive Noise Filtering
by: Hong, Sungchul, et al.
Published: (2024)

Automated Filtering of Human Feedback Data for Aligning Text-to-Image Diffusion Models
by: Yang, Yongjin, et al.
Published: (2024)

Are Data Augmentation Methods in Named Entity Recognition Applicable for Uncertainty Estimation?
by: Hashimoto, Wataru, et al.
Published: (2024)

WebSTAR: Scalable Data Synthesis for Computer Use Agents with Step-Level Filtering
by: He, Yifei, et al.
Published: (2025)

Cyborg Data: Merging Human with AI Generated Training Data
by: North, Kai, et al.
Published: (2025)

s1: Simple test-time scaling
by: Muennighoff, Niklas, et al.
Published: (2025)

Deep Ignorance: Filtering Pretraining Data Builds Tamper-Resistant Safeguards into Open-Weight LLMs
by: O'Brien, Kyle, et al.
Published: (2025)

Preparing Lessons for Progressive Training on Language Models
by: Pan, Yu, et al.
Published: (2024)

Filter then Attend: Improving attention-based Time Series Forecasting with Spectral Filtering
by: Dayag, Elisha, et al.
Published: (2025)

The Routing and Filtering Structure of Attention
by: Jamil, Shafayeth, et al.
Published: (2026)

FSPO: Few-Shot Optimization of Synthetic Preferences Personalizes to Real Users
by: Singh, Anikait, et al.
Published: (2025)

Learning to (Learn at Test Time): RNNs with Expressive Hidden States
by: Sun, Yu, et al.
Published: (2024)

Convolutional Transformer Neural Collaborative Filtering
by: Li, Pang, et al.
Published: (2024)

Lessons from Learning to Spin "Pens"
by: Wang, Jun, et al.
Published: (2024)