Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2606.02437 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910282275618816 |
|---|---|
| author | Lab, Mind : Cao, Song Cao, Vic Chen, Kaijie Fan, Bunny Feng, Hera Feng, Huan Fu, Arthur Gao, Jun Gu, Hongquan Guan, Aaron Hong, Mutian Hou, Hailee Hua, Peixuan Huang, Charles Jiang, Miles Jiang, Nora Jiang, Yuyi Jin, Autumn Kong, Fancy Lei, Kyrie Li, Alexy Li, Dawn Li, Ray Li, Theo Li, Wenhao Lin, Jiayi Liu, Domini Liu, Heshan Liu, Kairus Liu, Logan Luo, Maeve Lv, Runism Ma, Pony Niu, Verity Qiu, Anson Wang, Vincent Yao, Maxwell Ye, Regis Ye, Wenlin Ye, Yanying Ying, Josh Zeng, Danney Zhan, Salmon Zhang, Anya Zhang, Ruijia Zhang, Shiyang Zhang, Sueky Zhang, Ya Zhao, Wei Zhou, Ada Zhou, Sizer Zhu, Xinyue Zhuang, Murphy |
| author_facet | Lab, Mind : Cao, Song Cao, Vic Chen, Kaijie Fan, Bunny Feng, Hera Feng, Huan Fu, Arthur Gao, Jun Gu, Hongquan Guan, Aaron Hong, Mutian Hou, Hailee Hua, Peixuan Huang, Charles Jiang, Miles Jiang, Nora Jiang, Yuyi Jin, Autumn Kong, Fancy Lei, Kyrie Li, Alexy Li, Dawn Li, Ray Li, Theo Li, Wenhao Lin, Jiayi Liu, Domini Liu, Heshan Liu, Kairus Liu, Logan Luo, Maeve Lv, Runism Ma, Pony Niu, Verity Qiu, Anson Wang, Vincent Yao, Maxwell Ye, Regis Ye, Wenlin Ye, Yanying Ying, Josh Zeng, Danney Zhan, Salmon Zhang, Anya Zhang, Ruijia Zhang, Shiyang Zhang, Sueky Zhang, Ya Zhao, Wei Zhou, Ada Zhou, Sizer Zhu, Xinyue Zhuang, Murphy |
| contents | Parameter-efficient fine-tuning (PEFT) is usually treated as a cheaper alternative to full fine-tuning. We study a broader role: small trainable adapters as persistent local state on top of strong shared foundation models. In this framing, the base model provides shared competence while adapters carry instance-specific behavior such as preferences, skills, tool habits, and memory-like updates. We organize the problem around three scaling axes: Scale Up, where stronger shared priors make small local updates more useful; Scale Down, where we study how small adapters can be while remaining reliable; and Scale Out, where many persistent adapted instances coexist. MinT provides one infrastructure example for managing adapter identity, revision, provenance, evaluation, and serving residency. Together, the results suggest that PEFT can be a compact substrate for persistent personal models rather than only a budget substitute for full fine-tuning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2606_02437 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | On the Scaling of PEFT: Towards Million Personal Models of Trillion Parameters Lab, Mind : Cao, Song Cao, Vic Chen, Kaijie Fan, Bunny Feng, Hera Feng, Huan Fu, Arthur Gao, Jun Gu, Hongquan Guan, Aaron Hong, Mutian Hou, Hailee Hua, Peixuan Huang, Charles Jiang, Miles Jiang, Nora Jiang, Yuyi Jin, Autumn Kong, Fancy Lei, Kyrie Li, Alexy Li, Dawn Li, Ray Li, Theo Li, Wenhao Lin, Jiayi Liu, Domini Liu, Heshan Liu, Kairus Liu, Logan Luo, Maeve Lv, Runism Ma, Pony Niu, Verity Qiu, Anson Wang, Vincent Yao, Maxwell Ye, Regis Ye, Wenlin Ye, Yanying Ying, Josh Zeng, Danney Zhan, Salmon Zhang, Anya Zhang, Ruijia Zhang, Shiyang Zhang, Sueky Zhang, Ya Zhao, Wei Zhou, Ada Zhou, Sizer Zhu, Xinyue Zhuang, Murphy Machine Learning Computation and Language Parameter-efficient fine-tuning (PEFT) is usually treated as a cheaper alternative to full fine-tuning. We study a broader role: small trainable adapters as persistent local state on top of strong shared foundation models. In this framing, the base model provides shared competence while adapters carry instance-specific behavior such as preferences, skills, tool habits, and memory-like updates. We organize the problem around three scaling axes: Scale Up, where stronger shared priors make small local updates more useful; Scale Down, where we study how small adapters can be while remaining reliable; and Scale Out, where many persistent adapted instances coexist. MinT provides one infrastructure example for managing adapter identity, revision, provenance, evaluation, and serving residency. Together, the results suggest that PEFT can be a compact substrate for persistent personal models rather than only a budget substitute for full fine-tuning. |
| title | On the Scaling of PEFT: Towards Million Personal Models of Trillion Parameters |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2606.02437 |