DATE-LM: Benchmarking Data Attribution Evaluation for Large Language Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911230673813504 |
|---|---|
| author | Jiao, Cathy Pan, Yijun Xiao, Emily Sheng, Daisy Jain, Niket Zhao, Hanzhang Dasgupta, Ishita Ma, Jiaqi W. Xiong, Chenyan |
| author_facet | Jiao, Cathy Pan, Yijun Xiao, Emily Sheng, Daisy Jain, Niket Zhao, Hanzhang Dasgupta, Ishita Ma, Jiaqi W. Xiong, Chenyan |
| contents | Data attribution methods quantify the influence of training data on model outputs and are becoming increasingly relevant for a wide range of LLM research and applications, including dataset curation, model interpretability, data valuation. However, there remain critical gaps in systematic LLM-centric evaluation of data attribution methods. To this end, we introduce DATE-LM (Data Attribution Evaluation in Language Models), a unified benchmark for evaluating data attribution methods through real-world LLM applications. DATE-LM measures attribution quality through three key tasks -- training data selection, toxicity/bias filtering, and factual attribution. Our benchmark is designed for ease of use, enabling researchers to configure and run large-scale evaluations across diverse tasks and LLM architectures. Furthermore, we use DATE-LM to conduct a large-scale evaluation of existing data attribution methods. Our findings show that no single method dominates across all tasks, data attribution methods have trade-offs with simpler baselines, and method performance is sensitive to task-specific evaluation design. Finally, we release a public leaderboard for quick comparison of methods and to facilitate community engagement, with the motivation that DATE-LM can serve as a foundation for future data attribution research in LLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_09424 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | DATE-LM: Benchmarking Data Attribution Evaluation for Large Language Models Jiao, Cathy Pan, Yijun Xiao, Emily Sheng, Daisy Jain, Niket Zhao, Hanzhang Dasgupta, Ishita Ma, Jiaqi W. Xiong, Chenyan Computation and Language Data attribution methods quantify the influence of training data on model outputs and are becoming increasingly relevant for a wide range of LLM research and applications, including dataset curation, model interpretability, data valuation. However, there remain critical gaps in systematic LLM-centric evaluation of data attribution methods. To this end, we introduce DATE-LM (Data Attribution Evaluation in Language Models), a unified benchmark for evaluating data attribution methods through real-world LLM applications. DATE-LM measures attribution quality through three key tasks -- training data selection, toxicity/bias filtering, and factual attribution. Our benchmark is designed for ease of use, enabling researchers to configure and run large-scale evaluations across diverse tasks and LLM architectures. Furthermore, we use DATE-LM to conduct a large-scale evaluation of existing data attribution methods. Our findings show that no single method dominates across all tasks, data attribution methods have trade-offs with simpler baselines, and method performance is sensitive to task-specific evaluation design. Finally, we release a public leaderboard for quick comparison of methods and to facilitate community engagement, with the motivation that DATE-LM can serve as a foundation for future data attribution research in LLMs. |
| title | DATE-LM: Benchmarking Data Attribution Evaluation for Large Language Models |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2507.09424 |