Pangu Embedded: An Efficient Dual-system LLM Reasoner with Metacognition
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915311285960704 |
|---|---|
| author | Chen, Hanting Wang, Yasheng Han, Kai Li, Dong Li, Lin Bi, Zhenni Li, Jinpeng Wang, Haoyu Mi, Fei Zhu, Mingjian Wang, Bin Song, Kaikai Fu, Yifei He, Xu Luo, Yu Zhu, Chong He, Quan Wu, Xueyu He, Wei Hu, Hailin Tang, Yehui Tao, Dacheng Chen, Xinghao Wang, Yunhe |
| author_facet | Chen, Hanting Wang, Yasheng Han, Kai Li, Dong Li, Lin Bi, Zhenni Li, Jinpeng Wang, Haoyu Mi, Fei Zhu, Mingjian Wang, Bin Song, Kaikai Fu, Yifei He, Xu Luo, Yu Zhu, Chong He, Quan Wu, Xueyu He, Wei Hu, Hailin Tang, Yehui Tao, Dacheng Chen, Xinghao Wang, Yunhe |
| contents | This work presents Pangu Embedded, an efficient Large Language Model (LLM) reasoner developed on Ascend Neural Processing Units (NPUs), featuring flexible fast and slow thinking capabilities. Pangu Embedded addresses the significant computational costs and inference latency challenges prevalent in existing reasoning-optimized LLMs. We propose a two-stage training framework for its construction. In Stage 1, the model is finetuned via an iterative distillation process, incorporating inter-iteration model merging to effectively aggregate complementary knowledge. This is followed by reinforcement learning on Ascend clusters, optimized by a latency-tolerant scheduler that combines stale synchronous parallelism with prioritized data queues. The RL process is guided by a Multi-source Adaptive Reward System (MARS), which generates dynamic, task-specific reward signals using deterministic metrics and lightweight LLM evaluators for mathematics, coding, and general problem-solving tasks. Stage 2 introduces a dual-system framework, endowing Pangu Embedded with a "fast" mode for routine queries and a deeper "slow" mode for complex inference. This framework offers both manual mode switching for user control and an automatic, complexity-aware mode selection mechanism that dynamically allocates computational resources to balance latency and reasoning depth. Experimental results on benchmarks including AIME 2024, GPQA, and LiveCodeBench demonstrate that Pangu Embedded with 7B parameters, outperforms similar-size models like Qwen3-8B and GLM4-9B. It delivers rapid responses and state-of-the-art reasoning quality within a single, unified model architecture, highlighting a promising direction for developing powerful yet practically deployable LLM reasoners. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_22375 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Pangu Embedded: An Efficient Dual-system LLM Reasoner with Metacognition Chen, Hanting Wang, Yasheng Han, Kai Li, Dong Li, Lin Bi, Zhenni Li, Jinpeng Wang, Haoyu Mi, Fei Zhu, Mingjian Wang, Bin Song, Kaikai Fu, Yifei He, Xu Luo, Yu Zhu, Chong He, Quan Wu, Xueyu He, Wei Hu, Hailin Tang, Yehui Tao, Dacheng Chen, Xinghao Wang, Yunhe Computation and Language This work presents Pangu Embedded, an efficient Large Language Model (LLM) reasoner developed on Ascend Neural Processing Units (NPUs), featuring flexible fast and slow thinking capabilities. Pangu Embedded addresses the significant computational costs and inference latency challenges prevalent in existing reasoning-optimized LLMs. We propose a two-stage training framework for its construction. In Stage 1, the model is finetuned via an iterative distillation process, incorporating inter-iteration model merging to effectively aggregate complementary knowledge. This is followed by reinforcement learning on Ascend clusters, optimized by a latency-tolerant scheduler that combines stale synchronous parallelism with prioritized data queues. The RL process is guided by a Multi-source Adaptive Reward System (MARS), which generates dynamic, task-specific reward signals using deterministic metrics and lightweight LLM evaluators for mathematics, coding, and general problem-solving tasks. Stage 2 introduces a dual-system framework, endowing Pangu Embedded with a "fast" mode for routine queries and a deeper "slow" mode for complex inference. This framework offers both manual mode switching for user control and an automatic, complexity-aware mode selection mechanism that dynamically allocates computational resources to balance latency and reasoning depth. Experimental results on benchmarks including AIME 2024, GPQA, and LiveCodeBench demonstrate that Pangu Embedded with 7B parameters, outperforms similar-size models like Qwen3-8B and GLM4-9B. It delivers rapid responses and state-of-the-art reasoning quality within a single, unified model architecture, highlighting a promising direction for developing powerful yet practically deployable LLM reasoners. |
| title | Pangu Embedded: An Efficient Dual-system LLM Reasoner with Metacognition |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2505.22375 |