Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.25817 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918178279391232 |
|---|---|
| author | Luo, Junyu Wu, Bohan Luo, Xiao Xiao, Zhiping Jin, Yiqiao Tu, Rong-Cheng Yin, Nan Wang, Yifan Yuan, Jingyang Ju, Wei Zhang, Ming |
| author_facet | Luo, Junyu Wu, Bohan Luo, Xiao Xiao, Zhiping Jin, Yiqiao Tu, Rong-Cheng Yin, Nan Wang, Yifan Yuan, Jingyang Ju, Wei Zhang, Ming |
| contents | Post-training of Large Language Models (LLMs) is crucial for unlocking their task generalization potential and domain-specific capabilities. However, the current LLM post-training paradigm faces significant data challenges, including the high costs of manual annotation and diminishing marginal returns on data scales. Therefore, achieving data-efficient post-training has become a key research question. In this paper, we present the first systematic survey of data-efficient LLM post-training from a data-centric perspective. We propose a taxonomy of data-efficient LLM post-training methods, covering data selection, data quality enhancement, synthetic data generation, data distillation and compression, and self-evolving data ecosystems. We summarize representative approaches in each category and outline future research directions. By examining the challenges in data-efficient LLM post-training, we highlight open problems and propose potential research avenues. We hope our work inspires further exploration into maximizing the potential of data utilization in large-scale model training. Paper List: https://github.com/luo-junyu/Awesome-Data-Efficient-LLM |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_25817 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A Survey on Efficient Large Language Model Training: From Data-centric Perspectives Luo, Junyu Wu, Bohan Luo, Xiao Xiao, Zhiping Jin, Yiqiao Tu, Rong-Cheng Yin, Nan Wang, Yifan Yuan, Jingyang Ju, Wei Zhang, Ming Computation and Language Post-training of Large Language Models (LLMs) is crucial for unlocking their task generalization potential and domain-specific capabilities. However, the current LLM post-training paradigm faces significant data challenges, including the high costs of manual annotation and diminishing marginal returns on data scales. Therefore, achieving data-efficient post-training has become a key research question. In this paper, we present the first systematic survey of data-efficient LLM post-training from a data-centric perspective. We propose a taxonomy of data-efficient LLM post-training methods, covering data selection, data quality enhancement, synthetic data generation, data distillation and compression, and self-evolving data ecosystems. We summarize representative approaches in each category and outline future research directions. By examining the challenges in data-efficient LLM post-training, we highlight open problems and propose potential research avenues. We hope our work inspires further exploration into maximizing the potential of data utilization in large-scale model training. Paper List: https://github.com/luo-junyu/Awesome-Data-Efficient-LLM |
| title | A Survey on Efficient Large Language Model Training: From Data-centric Perspectives |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2510.25817 |