Youtu-LLM: Unlocking the Native Agentic Potential for Lightweight Large Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911352291852288 |
|---|---|
| author | Lu, Junru Qin, Jiarui Qiao, Lingfeng Li, Yinghui Dai, Xinyi Ke, Bo He, Jianfeng Qiao, Ruizhi Yin, Di Sun, Xing Wu, Yunsheng Liu, Yinsong Liu, Shuangyin Tang, Mingkong Lin, Haodong Kuang, Jiayi Meng, Fanxu Tang, Xiaojuan Xi, Yunjia Huang, Junjie Yang, Haotong Shen, Zhenyi Li, Yangning Zhang, Qianwen Yu, Yifei An, Siyu Dong, Junnan Wang, Qiufeng Wang, Jie Chen, Keyu Wen, Wei Guo, Taian Shen, Zhifeng Yu, Daohai Li, Jiahao Li, Ke Li, Zongyi Tan, Xiaoyu |
| author_facet | Lu, Junru Qin, Jiarui Qiao, Lingfeng Li, Yinghui Dai, Xinyi Ke, Bo He, Jianfeng Qiao, Ruizhi Yin, Di Sun, Xing Wu, Yunsheng Liu, Yinsong Liu, Shuangyin Tang, Mingkong Lin, Haodong Kuang, Jiayi Meng, Fanxu Tang, Xiaojuan Xi, Yunjia Huang, Junjie Yang, Haotong Shen, Zhenyi Li, Yangning Zhang, Qianwen Yu, Yifei An, Siyu Dong, Junnan Wang, Qiufeng Wang, Jie Chen, Keyu Wen, Wei Guo, Taian Shen, Zhifeng Yu, Daohai Li, Jiahao Li, Ke Li, Zongyi Tan, Xiaoyu |
| contents | We introduce Youtu-LLM, a lightweight yet powerful language model that harmonizes high computational efficiency with native agentic intelligence. Unlike typical small models that rely on distillation, Youtu-LLM (1.96B) is pre-trained from scratch to systematically cultivate reasoning and planning capabilities. The key technical advancements are as follows: (1) Compact Architecture with Long-Context Support: Built on a dense Multi-Latent Attention (MLA) architecture with a novel STEM-oriented vocabulary, Youtu-LLM supports a 128k context window. This design enables robust long-context reasoning and state tracking within a minimal memory footprint, making it ideal for long-horizon agent and reasoning tasks. (2) Principled "Commonsense-STEM-Agent" Curriculum: We curated a massive corpus of approximately 11T tokens and implemented a multi-stage training strategy. By progressively shifting the pre-training data distribution from general commonsense to complex STEM and agentic tasks, we ensure the model acquires deep cognitive abilities rather than superficial alignment. (3) Scalable Agentic Mid-training: Specifically for the agentic mid-training, we employ diverse data construction schemes to synthesize rich and varied trajectories across math, coding, and tool-use domains. This high-quality data enables the model to internalize planning and reflection behaviors effectively. Extensive evaluations show that Youtu-LLM sets a new state-of-the-art for sub-2B LLMs. On general benchmarks, it achieves competitive performance against larger models, while on agent-specific tasks, it significantly surpasses existing SOTA baselines, demonstrating that lightweight models can possess strong intrinsic agentic capabilities. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_24618 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Youtu-LLM: Unlocking the Native Agentic Potential for Lightweight Large Language Models Lu, Junru Qin, Jiarui Qiao, Lingfeng Li, Yinghui Dai, Xinyi Ke, Bo He, Jianfeng Qiao, Ruizhi Yin, Di Sun, Xing Wu, Yunsheng Liu, Yinsong Liu, Shuangyin Tang, Mingkong Lin, Haodong Kuang, Jiayi Meng, Fanxu Tang, Xiaojuan Xi, Yunjia Huang, Junjie Yang, Haotong Shen, Zhenyi Li, Yangning Zhang, Qianwen Yu, Yifei An, Siyu Dong, Junnan Wang, Qiufeng Wang, Jie Chen, Keyu Wen, Wei Guo, Taian Shen, Zhifeng Yu, Daohai Li, Jiahao Li, Ke Li, Zongyi Tan, Xiaoyu Computation and Language We introduce Youtu-LLM, a lightweight yet powerful language model that harmonizes high computational efficiency with native agentic intelligence. Unlike typical small models that rely on distillation, Youtu-LLM (1.96B) is pre-trained from scratch to systematically cultivate reasoning and planning capabilities. The key technical advancements are as follows: (1) Compact Architecture with Long-Context Support: Built on a dense Multi-Latent Attention (MLA) architecture with a novel STEM-oriented vocabulary, Youtu-LLM supports a 128k context window. This design enables robust long-context reasoning and state tracking within a minimal memory footprint, making it ideal for long-horizon agent and reasoning tasks. (2) Principled "Commonsense-STEM-Agent" Curriculum: We curated a massive corpus of approximately 11T tokens and implemented a multi-stage training strategy. By progressively shifting the pre-training data distribution from general commonsense to complex STEM and agentic tasks, we ensure the model acquires deep cognitive abilities rather than superficial alignment. (3) Scalable Agentic Mid-training: Specifically for the agentic mid-training, we employ diverse data construction schemes to synthesize rich and varied trajectories across math, coding, and tool-use domains. This high-quality data enables the model to internalize planning and reflection behaviors effectively. Extensive evaluations show that Youtu-LLM sets a new state-of-the-art for sub-2B LLMs. On general benchmarks, it achieves competitive performance against larger models, while on agent-specific tasks, it significantly surpasses existing SOTA baselines, demonstrating that lightweight models can possess strong intrinsic agentic capabilities. |
| title | Youtu-LLM: Unlocking the Native Agentic Potential for Lightweight Large Language Models |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2512.24618 |