Data-Centric AI in the Age of Large Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917699790045184 |
|---|---|
| author | Xu, Xinyi Wu, Zhaoxuan Qiao, Rui Verma, Arun Shu, Yao Wang, Jingtan Niu, Xinyuan He, Zhenfeng Chen, Jiangwei Zhou, Zijian Lau, Gregory Kang Ruey Dao, Hieu Agussurja, Lucas Sim, Rachael Hwee Ling Lin, Xiaoqiang Hu, Wenyang Dai, Zhongxiang Koh, Pang Wei Low, Bryan Kian Hsiang |
| author_facet | Xu, Xinyi Wu, Zhaoxuan Qiao, Rui Verma, Arun Shu, Yao Wang, Jingtan Niu, Xinyuan He, Zhenfeng Chen, Jiangwei Zhou, Zijian Lau, Gregory Kang Ruey Dao, Hieu Agussurja, Lucas Sim, Rachael Hwee Ling Lin, Xiaoqiang Hu, Wenyang Dai, Zhongxiang Koh, Pang Wei Low, Bryan Kian Hsiang |
| contents | This position paper proposes a data-centric viewpoint of AI research, focusing on large language models (LLMs). We start by making the key observation that data is instrumental in the developmental (e.g., pretraining and fine-tuning) and inferential stages (e.g., in-context learning) of LLMs, and yet it receives disproportionally low attention from the research community. We identify four specific scenarios centered around data, covering data-centric benchmarks and data curation, data attribution, knowledge transfer, and inference contextualization. In each scenario, we underscore the importance of data, highlight promising research directions, and articulate the potential impacts on the research community and, where applicable, the society as a whole. For instance, we advocate for a suite of data-centric benchmarks tailored to the scale and complexity of data for LLMs. These benchmarks can be used to develop new data curation methods and document research efforts and results, which can help promote openness and transparency in AI and LLM research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_14473 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Data-Centric AI in the Age of Large Language Models Xu, Xinyi Wu, Zhaoxuan Qiao, Rui Verma, Arun Shu, Yao Wang, Jingtan Niu, Xinyuan He, Zhenfeng Chen, Jiangwei Zhou, Zijian Lau, Gregory Kang Ruey Dao, Hieu Agussurja, Lucas Sim, Rachael Hwee Ling Lin, Xiaoqiang Hu, Wenyang Dai, Zhongxiang Koh, Pang Wei Low, Bryan Kian Hsiang Machine Learning Computation and Language This position paper proposes a data-centric viewpoint of AI research, focusing on large language models (LLMs). We start by making the key observation that data is instrumental in the developmental (e.g., pretraining and fine-tuning) and inferential stages (e.g., in-context learning) of LLMs, and yet it receives disproportionally low attention from the research community. We identify four specific scenarios centered around data, covering data-centric benchmarks and data curation, data attribution, knowledge transfer, and inference contextualization. In each scenario, we underscore the importance of data, highlight promising research directions, and articulate the potential impacts on the research community and, where applicable, the society as a whole. For instance, we advocate for a suite of data-centric benchmarks tailored to the scale and complexity of data for LLMs. These benchmarks can be used to develop new data curation methods and document research efforts and results, which can help promote openness and transparency in AI and LLM research. |
| title | Data-Centric AI in the Age of Large Language Models |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2406.14473 |