Scaling Agents via Continual Pre-training
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908542965907456 |
|---|---|
| author | Su, Liangcai Zhang, Zhen Li, Guangyu Chen, Zhuo Wang, Chenxi Song, Maojia Wang, Xinyu Li, Kuan Wu, Jialong Chen, Xuanzhong Qiao, Zile Zhang, Zhongwang Yin, Huifeng Cai, Shihao Fang, Runnan Tao, Zhengwei Yin, Wenbiao Qian, Chenxiong Jiang, Yong Xie, Pengjun Huang, Fei Zhou, Jingren |
| author_facet | Su, Liangcai Zhang, Zhen Li, Guangyu Chen, Zhuo Wang, Chenxi Song, Maojia Wang, Xinyu Li, Kuan Wu, Jialong Chen, Xuanzhong Qiao, Zile Zhang, Zhongwang Yin, Huifeng Cai, Shihao Fang, Runnan Tao, Zhengwei Yin, Wenbiao Qian, Chenxiong Jiang, Yong Xie, Pengjun Huang, Fei Zhou, Jingren |
| contents | Large language models (LLMs) have evolved into agentic systems capable of autonomous tool use and multi-step reasoning for complex problem-solving. However, post-training approaches building upon general-purpose foundation models consistently underperform in agentic tasks, particularly in open-source implementations. We identify the root cause: the absence of robust agentic foundation models forces models during post-training to simultaneously learn diverse agentic behaviors while aligning them to expert demonstrations, thereby creating fundamental optimization tensions. To this end, we are the first to propose incorporating Agentic Continual Pre-training (Agentic CPT) into the deep research agents training pipeline to build powerful agentic foundational models. Based on this approach, we develop a deep research agent model named AgentFounder. We evaluate our AgentFounder-30B on 10 benchmarks and achieve state-of-the-art performance while retains strong tool-use ability, notably 39.9% on BrowseComp-en, 43.3% on BrowseComp-zh, and 31.5% Pass@1 on HLE. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_13310 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Scaling Agents via Continual Pre-training Su, Liangcai Zhang, Zhen Li, Guangyu Chen, Zhuo Wang, Chenxi Song, Maojia Wang, Xinyu Li, Kuan Wu, Jialong Chen, Xuanzhong Qiao, Zile Zhang, Zhongwang Yin, Huifeng Cai, Shihao Fang, Runnan Tao, Zhengwei Yin, Wenbiao Qian, Chenxiong Jiang, Yong Xie, Pengjun Huang, Fei Zhou, Jingren Computation and Language Large language models (LLMs) have evolved into agentic systems capable of autonomous tool use and multi-step reasoning for complex problem-solving. However, post-training approaches building upon general-purpose foundation models consistently underperform in agentic tasks, particularly in open-source implementations. We identify the root cause: the absence of robust agentic foundation models forces models during post-training to simultaneously learn diverse agentic behaviors while aligning them to expert demonstrations, thereby creating fundamental optimization tensions. To this end, we are the first to propose incorporating Agentic Continual Pre-training (Agentic CPT) into the deep research agents training pipeline to build powerful agentic foundational models. Based on this approach, we develop a deep research agent model named AgentFounder. We evaluate our AgentFounder-30B on 10 benchmarks and achieve state-of-the-art performance while retains strong tool-use ability, notably 39.9% on BrowseComp-en, 43.3% on BrowseComp-zh, and 31.5% Pass@1 on HLE. |
| title | Scaling Agents via Continual Pre-training |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2509.13310 |