Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2505.19147 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915547752431616 |
|---|---|
| author | Liu, Xuyang Wen, Zichen Wang, Shaobo Chen, Junjie Tao, Zhishan Wang, Yubo Chen, Tailai Jin, Xiangqi Zou, Chang Wang, Yiyu Liao, Chenfei Zheng, Xu Chen, Honggang Li, Weijia Hu, Xuming He, Conghui Zhang, Linfeng |
| author_facet | Liu, Xuyang Wen, Zichen Wang, Shaobo Chen, Junjie Tao, Zhishan Wang, Yubo Chen, Tailai Jin, Xiangqi Zou, Chang Wang, Yiyu Liao, Chenfei Zheng, Xu Chen, Honggang Li, Weijia Hu, Xuming He, Conghui Zhang, Linfeng |
| contents | The advancement of large language models (LLMs) and multi-modal LLMs (MLLMs) has historically relied on scaling model parameters. However, as hardware limits constrain further model growth, the primary computational bottleneck has shifted to the quadratic cost of self-attention over increasingly long sequences by ultra-long text contexts, high-resolution images, and extended videos. In this position paper, \textbf{we argue that the focus of research for efficient artificial intelligence (AI) is shifting from model-centric compression to data-centric compression}. We position data-centric compression as the emerging paradigm, which improves AI efficiency by directly compressing the volume of data processed during model training or inference. To formalize this shift, we establish a unified framework for existing efficiency strategies and demonstrate why it constitutes a crucial paradigm change for long-context AI. We then systematically review the landscape of data-centric compression methods, analyzing their benefits across diverse scenarios. Finally, we outline key challenges and promising future research directions. Our work aims to provide a novel perspective on AI efficiency, synthesize existing efforts, and catalyze innovation to address the challenges posed by ever-increasing context lengths. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_19147 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Shifting AI Efficiency From Model-Centric to Data-Centric Compression Liu, Xuyang Wen, Zichen Wang, Shaobo Chen, Junjie Tao, Zhishan Wang, Yubo Chen, Tailai Jin, Xiangqi Zou, Chang Wang, Yiyu Liao, Chenfei Zheng, Xu Chen, Honggang Li, Weijia Hu, Xuming He, Conghui Zhang, Linfeng Computation and Language Artificial Intelligence Computer Vision and Pattern Recognition The advancement of large language models (LLMs) and multi-modal LLMs (MLLMs) has historically relied on scaling model parameters. However, as hardware limits constrain further model growth, the primary computational bottleneck has shifted to the quadratic cost of self-attention over increasingly long sequences by ultra-long text contexts, high-resolution images, and extended videos. In this position paper, \textbf{we argue that the focus of research for efficient artificial intelligence (AI) is shifting from model-centric compression to data-centric compression}. We position data-centric compression as the emerging paradigm, which improves AI efficiency by directly compressing the volume of data processed during model training or inference. To formalize this shift, we establish a unified framework for existing efficiency strategies and demonstrate why it constitutes a crucial paradigm change for long-context AI. We then systematically review the landscape of data-centric compression methods, analyzing their benefits across diverse scenarios. Finally, we outline key challenges and promising future research directions. Our work aims to provide a novel perspective on AI efficiency, synthesize existing efforts, and catalyze innovation to address the challenges posed by ever-increasing context lengths. |
| title | Shifting AI Efficiency From Model-Centric to Data-Centric Compression |
| topic | Computation and Language Artificial Intelligence Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2505.19147 |