ERNIE 5.0 Technical Report
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908813330743296 |
|---|---|
| author | Wang, Haifeng Wu, Hua Wu, Tian Sun, Yu Liu, Jing Yu, Dianhai Ma, Yanjun He, Jingzhou He, Zhongjun Hong, Dou Liu, Qiwen Wang, Shuohuan Shang, Junyuan Zhang, Zhenyu Ding, Yuchen Zeng, Jinle Yang, Jiabin Shen, Liang Chen, Ruibiao Yin, Weichong Ding, Siyu Dai, Dai Feng, Shikun Bao, Siqi He, Bolei Chen, Yan Jiao, Zhenyu Zhang, Ruiqing Chen, Zeyu Dang, Qingqing Deng, Kaipeng Jiang, Jiajun Gong, Enlei Wang, Guoxia Sha, Yanlin Liu, Yi Zheng, Yehan Xu, Weijian Liu, Jiaxiang Zeng, Zengfeng Qu, Yingqi Li, Zhongli Zhang, Zhengkun Wang, Xiyang Xu, Zixiang Xu, Xinchao Huang, Zhengjie Wang, Dong Chen, Bingjin Chang, Yue Yuan, Xing Huang, Shiwei Zhao, Qiao Ding, Xinzhe Qiao, Shuangshuang Yang, Baoshan Tang, Bihong Li, Bin Wang, Bingquan Tang, Binhan Zheng, Binxiong Cui, Bo Ke, Bo Zhang, Bo Zhang, Bowen Zhang, Boyan Liu, Boyang Zhang, Caiji Li, Can Xu, Chang Pang, Chao Zhang, Chao Yuan, Chaoyi Chen, Chen Cui, Cheng Yin, Chenlin Gan, Chun Chai, Chunguang Fang, Chuyu Han, Cuiyun Zhang, Dan Feng, Danlei Zhu, Danxiang Sun, Dong Li, Dongbo Li, Dongdong Liu, Dongdong Liu, Dongxue Ding, Fan Hu, Fan Li, Fan Mo, Fan Wu, Feisheng Liu, Fengwei Hu, Gangqiang Lu, Gaofeng Yong, Gaopeng Tian, Gexiao Wang, Guan Ni, Guangchen Wu, Guangshuo Wang, Guanzhong Liu, Guihua Li, Guishun Li, Haibin Liang, Haijian Ming, Haipeng Wang, Haisu Lu, Haiyang Lin, Haiye Zhou, Han Lou, Hangting Du, Hanwen Zhang, Hanzhi Chen, Hao Du, Hao Liu, Hao Zhou, Hao Jiang, Haochen Tian, Haodong Wang, Haoshuang Geng, Haozhe Yin, Heju Chen, Hong Xue, Hongchen Liu, Hongen Zhang, Honggeng Xu, Hongji Chen, Hongwei Zhang, Hongyang Zhang, Hongyuan Lu, Hua Chen, Huan Wang, Huan He, Huang Liu, Hui Zhong, Hui Ruan, Huibin Lu, Jiafeng Liang, Jiage Hu, Jiahao Hu, Jiahao Yang, Jiajie Li, Jialin Chen, Jian Wu, Jian Yang, Jianfeng Jiang, Jianguang Wang, Jianhua Chen, Jianye Liu, Jiaodi Zhou, Jiarui Lv, Jiawei Zhou, Jiaxin Liu, Jiaxuan Han, Jie Sun, Jie Fang, Jiefan Liu, Jihan Liu, Jihua Hu, Jing Qian, Jing Yan, Jing Du, Jingdong Wang, Jingdong Wu, Jingjing Li, Jingyong Wang, Jinheng Li, Jinjin Lu, Jinliang Yu, Jinlin Liu, Jinnan Feng, Jixiang Huang, Jiyi Zhang, Jiyuan Liang, Jun Xia, Jun Yu, Jun Chen, Junda Feng, Junhao Xiang, Junhong Li, Junliang Liu, Kai Chen, Kailun Su, Kairan Hu, Kang Zhou, Kangkang Chen, Ke Wei, Ke Huang, Kui Wu, Kun Chen, Kunbin Han, Lei Sun, Lei Wen, Lei Meng, Linghui Yu, Linhao Ouyang, Liping Zhang, Liwen Ji, Longbin Wang, Longzhi Sun, Meng Tian, Meng Li, Mengfei Zeng, Mengqi Zhang, Mengyu Hong, Ming Zhou, Mingcheng Huang, Mingming Chen, Mingxin Cai, Mingzhu Gu, Naibin Qiu, Nemin Wang, Nian Qiu, Peng Zhao, Peng Zou, Pengyu Wang, Qi Xin, Qi Wang, Qian Zhu, Qiang Luo, Qianhui Yang, Qianwei He, Qianyue Wu, Qifei Li, Qinrui Bao, Qiwen Zhang, Quan Liu, Quanxiang Xie, Qunyi Zhan, Rongrui Dai, Rufeng Peng, Rui Liu, Ruian Xu, Ruihao Wang, Ruijie Zhang, Ruixi Liu, Ruixuan Shi, Runsheng Wang, Ruting Kang, Senbo Lu, Shan Yu, Shaofei Gong, Shaotian Hu, Shenwei Zheng, Shifeng Guo, Shihao Fan, Shilong Liu, Shiqin Gu, Shiwei Zhang, Shixi Yao, Shuai Zhang, Shuang Liu, Shuangqiao Liang, Shuhao He, Shuwei Yang, Shuwen He, Sijun Dai, Siming Wu, Siming Long, Siyi Deng, Songhe Dong, Suhui Liang, Suyin Hu, Teng Xu, Tianchan Lv, Tianliang Yang, Tianmeng Wei, Tianyi Gao, Tiezhu Sun, Ting Zhang, Ting Luo, Tingdan He, Wei Luan, Wei Yin, Wei Zhang, Wei Zhou, Wei Gong, Weibao Li, Weibin Huang, Weicheng Dang, Weichong Zhu, Weiguo Zhang, Weilong Tan, Weiqi Huang, Wen Chang, Wenbin Du, Wenjing Miao, Wenlong Luo, Wenpei Wu, Wenquan Shi, Xi Zhao, Xi Gao, Xiang Zhang, Xiangguo Yu, Xiangrui Wang, Xiangsen Wang, Xiangzhe Luo, Xianlong Ma, Xianying Tan, Xiao Lin, Xiaocong Wang, Xiaofei Peng, Xiaofeng Wu, Xiaofeng Xu, Xiaojian Yuan, Xiaolan Cui, Xiaopeng Han, Xiaotian Liu, Xiaoxiong Fei, Xiaoxu Wu, Xiaoxuan Wang, Xiaoyu Zhang, Xiaoyu Sun, Xin Wang, Xin Huang, Xinhui Zhu, Xinming Yu, Xintong Xu, Xinyi Wang, Xinyu Li, Xiuxian Zhu, XuanShi Xu, Xue Lv, Xueying Li, Xuhong Wei, Xulong Chen, Xuyi Shi, Yabing Wang, Yafeng Li, Yamei Liu, Yan Cheng, Yanfu Gao, Yang Liang, Yang Wang, Yang Wang, Yang Yang, Yang Liu, Yanlong Fu, Yannian Wang, Yanpeng Lin, Yanzheng Chen, Yao Shen, Yaozong Han, Yaqian Yang, Yehua Chai, Yekun Wang, Yesong Song, Yi Zhang, Yichen Wang, Yifei Guo, Yifeng Kou, Yifeng Chen, Yilong Guo, Yilong Wang, Yiming Chen, Ying Wang, Ying Wu, Yingsheng Lin, Yingzhan Yang, Yinqi Xing, Yiran Lei, Yishu Tu, Yixiang Chen, Yiyan Zhang, Yong Li, Yonghua Ma, Yongqiang Dai, Yongxing Zhang, Yongyue Ran, Yu Sun, Yu Zhang, Yu-Wen Michael Liu, Yuang Liu, Yuanle Zhou, Yuanyuan Zhang, Yubo Han, Yuchen Wang, Yucheng Gao, Yude Luo, Yuedong Dong, Yuehu Hu, Yufeng Cao, Yuhui Yun, Yuhui Chen, Yukun Gao, Yukun Li, Yukun Zhang, Yumeng Fan, Yun Ma, Yun Zhang, Yunfei Xie, Yunshen Xu, Yuping Zhang, Yuqin Liu, Yuqing Li, Yurui Wang, Yuwen Lu, Yuxiang Cai, Zefeng Zhao, Zelin Zhang, Zelun Lin, Zenan Dong, Zezhao Pan, Zhaowu Liu, Zhaoyu Dong, Zhe Zhang, Zhe Zhang, Zhen Wu, Zhengfan Wei, Zhengrui Ning, Zhengsheng Li, Zhenxing Li, Zhenyu Qian, Zhenyu Li, Zhenyun Li, Zhi Chen, Zhichao Dong, Zhicheng Feng, Zhida Feng, Zhifan Deng, Zhihao Yu, Zhijin Chen, Zhiyang Zheng, Zhonghui Guo, Zhuangzhuang Zhang, Zhujun Sun, Zhuo Liu, Zichang Lin, Zihan Huang, Zihao Zhu, Zihe Zhao, Ziheng Chen, Ziping Zhu, Zixuan Xu, Ziyang Liang, Ziyi Gao, Ziyuan |
| author_facet | Wang, Haifeng Wu, Hua Wu, Tian Sun, Yu Liu, Jing Yu, Dianhai Ma, Yanjun He, Jingzhou He, Zhongjun Hong, Dou Liu, Qiwen Wang, Shuohuan Shang, Junyuan Zhang, Zhenyu Ding, Yuchen Zeng, Jinle Yang, Jiabin Shen, Liang Chen, Ruibiao Yin, Weichong Ding, Siyu Dai, Dai Feng, Shikun Bao, Siqi He, Bolei Chen, Yan Jiao, Zhenyu Zhang, Ruiqing Chen, Zeyu Dang, Qingqing Deng, Kaipeng Jiang, Jiajun Gong, Enlei Wang, Guoxia Sha, Yanlin Liu, Yi Zheng, Yehan Xu, Weijian Liu, Jiaxiang Zeng, Zengfeng Qu, Yingqi Li, Zhongli Zhang, Zhengkun Wang, Xiyang Xu, Zixiang Xu, Xinchao Huang, Zhengjie Wang, Dong Chen, Bingjin Chang, Yue Yuan, Xing Huang, Shiwei Zhao, Qiao Ding, Xinzhe Qiao, Shuangshuang Yang, Baoshan Tang, Bihong Li, Bin Wang, Bingquan Tang, Binhan Zheng, Binxiong Cui, Bo Ke, Bo Zhang, Bo Zhang, Bowen Zhang, Boyan Liu, Boyang Zhang, Caiji Li, Can Xu, Chang Pang, Chao Zhang, Chao Yuan, Chaoyi Chen, Chen Cui, Cheng Yin, Chenlin Gan, Chun Chai, Chunguang Fang, Chuyu Han, Cuiyun Zhang, Dan Feng, Danlei Zhu, Danxiang Sun, Dong Li, Dongbo Li, Dongdong Liu, Dongdong Liu, Dongxue Ding, Fan Hu, Fan Li, Fan Mo, Fan Wu, Feisheng Liu, Fengwei Hu, Gangqiang Lu, Gaofeng Yong, Gaopeng Tian, Gexiao Wang, Guan Ni, Guangchen Wu, Guangshuo Wang, Guanzhong Liu, Guihua Li, Guishun Li, Haibin Liang, Haijian Ming, Haipeng Wang, Haisu Lu, Haiyang Lin, Haiye Zhou, Han Lou, Hangting Du, Hanwen Zhang, Hanzhi Chen, Hao Du, Hao Liu, Hao Zhou, Hao Jiang, Haochen Tian, Haodong Wang, Haoshuang Geng, Haozhe Yin, Heju Chen, Hong Xue, Hongchen Liu, Hongen Zhang, Honggeng Xu, Hongji Chen, Hongwei Zhang, Hongyang Zhang, Hongyuan Lu, Hua Chen, Huan Wang, Huan He, Huang Liu, Hui Zhong, Hui Ruan, Huibin Lu, Jiafeng Liang, Jiage Hu, Jiahao Hu, Jiahao Yang, Jiajie Li, Jialin Chen, Jian Wu, Jian Yang, Jianfeng Jiang, Jianguang Wang, Jianhua Chen, Jianye Liu, Jiaodi Zhou, Jiarui Lv, Jiawei Zhou, Jiaxin Liu, Jiaxuan Han, Jie Sun, Jie Fang, Jiefan Liu, Jihan Liu, Jihua Hu, Jing Qian, Jing Yan, Jing Du, Jingdong Wang, Jingdong Wu, Jingjing Li, Jingyong Wang, Jinheng Li, Jinjin Lu, Jinliang Yu, Jinlin Liu, Jinnan Feng, Jixiang Huang, Jiyi Zhang, Jiyuan Liang, Jun Xia, Jun Yu, Jun Chen, Junda Feng, Junhao Xiang, Junhong Li, Junliang Liu, Kai Chen, Kailun Su, Kairan Hu, Kang Zhou, Kangkang Chen, Ke Wei, Ke Huang, Kui Wu, Kun Chen, Kunbin Han, Lei Sun, Lei Wen, Lei Meng, Linghui Yu, Linhao Ouyang, Liping Zhang, Liwen Ji, Longbin Wang, Longzhi Sun, Meng Tian, Meng Li, Mengfei Zeng, Mengqi Zhang, Mengyu Hong, Ming Zhou, Mingcheng Huang, Mingming Chen, Mingxin Cai, Mingzhu Gu, Naibin Qiu, Nemin Wang, Nian Qiu, Peng Zhao, Peng Zou, Pengyu Wang, Qi Xin, Qi Wang, Qian Zhu, Qiang Luo, Qianhui Yang, Qianwei He, Qianyue Wu, Qifei Li, Qinrui Bao, Qiwen Zhang, Quan Liu, Quanxiang Xie, Qunyi Zhan, Rongrui Dai, Rufeng Peng, Rui Liu, Ruian Xu, Ruihao Wang, Ruijie Zhang, Ruixi Liu, Ruixuan Shi, Runsheng Wang, Ruting Kang, Senbo Lu, Shan Yu, Shaofei Gong, Shaotian Hu, Shenwei Zheng, Shifeng Guo, Shihao Fan, Shilong Liu, Shiqin Gu, Shiwei Zhang, Shixi Yao, Shuai Zhang, Shuang Liu, Shuangqiao Liang, Shuhao He, Shuwei Yang, Shuwen He, Sijun Dai, Siming Wu, Siming Long, Siyi Deng, Songhe Dong, Suhui Liang, Suyin Hu, Teng Xu, Tianchan Lv, Tianliang Yang, Tianmeng Wei, Tianyi Gao, Tiezhu Sun, Ting Zhang, Ting Luo, Tingdan He, Wei Luan, Wei Yin, Wei Zhang, Wei Zhou, Wei Gong, Weibao Li, Weibin Huang, Weicheng Dang, Weichong Zhu, Weiguo Zhang, Weilong Tan, Weiqi Huang, Wen Chang, Wenbin Du, Wenjing Miao, Wenlong Luo, Wenpei Wu, Wenquan Shi, Xi Zhao, Xi Gao, Xiang Zhang, Xiangguo Yu, Xiangrui Wang, Xiangsen Wang, Xiangzhe Luo, Xianlong Ma, Xianying Tan, Xiao Lin, Xiaocong Wang, Xiaofei Peng, Xiaofeng Wu, Xiaofeng Xu, Xiaojian Yuan, Xiaolan Cui, Xiaopeng Han, Xiaotian Liu, Xiaoxiong Fei, Xiaoxu Wu, Xiaoxuan Wang, Xiaoyu Zhang, Xiaoyu Sun, Xin Wang, Xin Huang, Xinhui Zhu, Xinming Yu, Xintong Xu, Xinyi Wang, Xinyu Li, Xiuxian Zhu, XuanShi Xu, Xue Lv, Xueying Li, Xuhong Wei, Xulong Chen, Xuyi Shi, Yabing Wang, Yafeng Li, Yamei Liu, Yan Cheng, Yanfu Gao, Yang Liang, Yang Wang, Yang Wang, Yang Yang, Yang Liu, Yanlong Fu, Yannian Wang, Yanpeng Lin, Yanzheng Chen, Yao Shen, Yaozong Han, Yaqian Yang, Yehua Chai, Yekun Wang, Yesong Song, Yi Zhang, Yichen Wang, Yifei Guo, Yifeng Kou, Yifeng Chen, Yilong Guo, Yilong Wang, Yiming Chen, Ying Wang, Ying Wu, Yingsheng Lin, Yingzhan Yang, Yinqi Xing, Yiran Lei, Yishu Tu, Yixiang Chen, Yiyan Zhang, Yong Li, Yonghua Ma, Yongqiang Dai, Yongxing Zhang, Yongyue Ran, Yu Sun, Yu Zhang, Yu-Wen Michael Liu, Yuang Liu, Yuanle Zhou, Yuanyuan Zhang, Yubo Han, Yuchen Wang, Yucheng Gao, Yude Luo, Yuedong Dong, Yuehu Hu, Yufeng Cao, Yuhui Yun, Yuhui Chen, Yukun Gao, Yukun Li, Yukun Zhang, Yumeng Fan, Yun Ma, Yun Zhang, Yunfei Xie, Yunshen Xu, Yuping Zhang, Yuqin Liu, Yuqing Li, Yurui Wang, Yuwen Lu, Yuxiang Cai, Zefeng Zhao, Zelin Zhang, Zelun Lin, Zenan Dong, Zezhao Pan, Zhaowu Liu, Zhaoyu Dong, Zhe Zhang, Zhe Zhang, Zhen Wu, Zhengfan Wei, Zhengrui Ning, Zhengsheng Li, Zhenxing Li, Zhenyu Qian, Zhenyu Li, Zhenyun Li, Zhi Chen, Zhichao Dong, Zhicheng Feng, Zhida Feng, Zhifan Deng, Zhihao Yu, Zhijin Chen, Zhiyang Zheng, Zhonghui Guo, Zhuangzhuang Zhang, Zhujun Sun, Zhuo Liu, Zichang Lin, Zihan Huang, Zihao Zhu, Zihe Zhao, Ziheng Chen, Ziping Zhu, Zixuan Xu, Ziyang Liang, Ziyi Gao, Ziyuan |
| contents | In this report, we introduce ERNIE 5.0, a natively autoregressive foundation model desinged for unified multimodal understanding and generation across text, image, video, and audio. All modalities are trained from scratch under a unified next-group-of-tokens prediction objective, based on an ultra-sparse mixture-of-experts (MoE) architecture with modality-agnostic expert routing. To address practical challenges in large-scale deployment under diverse resource constraints, ERNIE 5.0 adopts a novel elastic training paradigm. Within a single pre-training run, the model learns a family of sub-models with varying depths, expert capacities, and routing sparsity, enabling flexible trade-offs among performance, model size, and inference latency in memory- or time-constrained scenarios. Moreover, we systematically address the challenges of scaling reinforcement learning to unified foundation models, thereby guaranteeing efficient and stable post-training under ultra-sparse MoE architectures and diverse multimodal settings. Extensive experiments demonstrate that ERNIE 5.0 achieves strong and balanced performance across multiple modalities. To the best of our knowledge, among publicly disclosed models, ERNIE 5.0 represents the first production-scale realization of a trillion-parameter unified autoregressive model that supports both multimodal understanding and generation. To facilitate further research, we present detailed visualizations of modality-agnostic expert routing in the unified model, alongside comprehensive empirical analysis of elastic training, aiming to offer profound insights to the community. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_04705 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | ERNIE 5.0 Technical Report Wang, Haifeng Wu, Hua Wu, Tian Sun, Yu Liu, Jing Yu, Dianhai Ma, Yanjun He, Jingzhou He, Zhongjun Hong, Dou Liu, Qiwen Wang, Shuohuan Shang, Junyuan Zhang, Zhenyu Ding, Yuchen Zeng, Jinle Yang, Jiabin Shen, Liang Chen, Ruibiao Yin, Weichong Ding, Siyu Dai, Dai Feng, Shikun Bao, Siqi He, Bolei Chen, Yan Jiao, Zhenyu Zhang, Ruiqing Chen, Zeyu Dang, Qingqing Deng, Kaipeng Jiang, Jiajun Gong, Enlei Wang, Guoxia Sha, Yanlin Liu, Yi Zheng, Yehan Xu, Weijian Liu, Jiaxiang Zeng, Zengfeng Qu, Yingqi Li, Zhongli Zhang, Zhengkun Wang, Xiyang Xu, Zixiang Xu, Xinchao Huang, Zhengjie Wang, Dong Chen, Bingjin Chang, Yue Yuan, Xing Huang, Shiwei Zhao, Qiao Ding, Xinzhe Qiao, Shuangshuang Yang, Baoshan Tang, Bihong Li, Bin Wang, Bingquan Tang, Binhan Zheng, Binxiong Cui, Bo Ke, Bo Zhang, Bo Zhang, Bowen Zhang, Boyan Liu, Boyang Zhang, Caiji Li, Can Xu, Chang Pang, Chao Zhang, Chao Yuan, Chaoyi Chen, Chen Cui, Cheng Yin, Chenlin Gan, Chun Chai, Chunguang Fang, Chuyu Han, Cuiyun Zhang, Dan Feng, Danlei Zhu, Danxiang Sun, Dong Li, Dongbo Li, Dongdong Liu, Dongdong Liu, Dongxue Ding, Fan Hu, Fan Li, Fan Mo, Fan Wu, Feisheng Liu, Fengwei Hu, Gangqiang Lu, Gaofeng Yong, Gaopeng Tian, Gexiao Wang, Guan Ni, Guangchen Wu, Guangshuo Wang, Guanzhong Liu, Guihua Li, Guishun Li, Haibin Liang, Haijian Ming, Haipeng Wang, Haisu Lu, Haiyang Lin, Haiye Zhou, Han Lou, Hangting Du, Hanwen Zhang, Hanzhi Chen, Hao Du, Hao Liu, Hao Zhou, Hao Jiang, Haochen Tian, Haodong Wang, Haoshuang Geng, Haozhe Yin, Heju Chen, Hong Xue, Hongchen Liu, Hongen Zhang, Honggeng Xu, Hongji Chen, Hongwei Zhang, Hongyang Zhang, Hongyuan Lu, Hua Chen, Huan Wang, Huan He, Huang Liu, Hui Zhong, Hui Ruan, Huibin Lu, Jiafeng Liang, Jiage Hu, Jiahao Hu, Jiahao Yang, Jiajie Li, Jialin Chen, Jian Wu, Jian Yang, Jianfeng Jiang, Jianguang Wang, Jianhua Chen, Jianye Liu, Jiaodi Zhou, Jiarui Lv, Jiawei Zhou, Jiaxin Liu, Jiaxuan Han, Jie Sun, Jie Fang, Jiefan Liu, Jihan Liu, Jihua Hu, Jing Qian, Jing Yan, Jing Du, Jingdong Wang, Jingdong Wu, Jingjing Li, Jingyong Wang, Jinheng Li, Jinjin Lu, Jinliang Yu, Jinlin Liu, Jinnan Feng, Jixiang Huang, Jiyi Zhang, Jiyuan Liang, Jun Xia, Jun Yu, Jun Chen, Junda Feng, Junhao Xiang, Junhong Li, Junliang Liu, Kai Chen, Kailun Su, Kairan Hu, Kang Zhou, Kangkang Chen, Ke Wei, Ke Huang, Kui Wu, Kun Chen, Kunbin Han, Lei Sun, Lei Wen, Lei Meng, Linghui Yu, Linhao Ouyang, Liping Zhang, Liwen Ji, Longbin Wang, Longzhi Sun, Meng Tian, Meng Li, Mengfei Zeng, Mengqi Zhang, Mengyu Hong, Ming Zhou, Mingcheng Huang, Mingming Chen, Mingxin Cai, Mingzhu Gu, Naibin Qiu, Nemin Wang, Nian Qiu, Peng Zhao, Peng Zou, Pengyu Wang, Qi Xin, Qi Wang, Qian Zhu, Qiang Luo, Qianhui Yang, Qianwei He, Qianyue Wu, Qifei Li, Qinrui Bao, Qiwen Zhang, Quan Liu, Quanxiang Xie, Qunyi Zhan, Rongrui Dai, Rufeng Peng, Rui Liu, Ruian Xu, Ruihao Wang, Ruijie Zhang, Ruixi Liu, Ruixuan Shi, Runsheng Wang, Ruting Kang, Senbo Lu, Shan Yu, Shaofei Gong, Shaotian Hu, Shenwei Zheng, Shifeng Guo, Shihao Fan, Shilong Liu, Shiqin Gu, Shiwei Zhang, Shixi Yao, Shuai Zhang, Shuang Liu, Shuangqiao Liang, Shuhao He, Shuwei Yang, Shuwen He, Sijun Dai, Siming Wu, Siming Long, Siyi Deng, Songhe Dong, Suhui Liang, Suyin Hu, Teng Xu, Tianchan Lv, Tianliang Yang, Tianmeng Wei, Tianyi Gao, Tiezhu Sun, Ting Zhang, Ting Luo, Tingdan He, Wei Luan, Wei Yin, Wei Zhang, Wei Zhou, Wei Gong, Weibao Li, Weibin Huang, Weicheng Dang, Weichong Zhu, Weiguo Zhang, Weilong Tan, Weiqi Huang, Wen Chang, Wenbin Du, Wenjing Miao, Wenlong Luo, Wenpei Wu, Wenquan Shi, Xi Zhao, Xi Gao, Xiang Zhang, Xiangguo Yu, Xiangrui Wang, Xiangsen Wang, Xiangzhe Luo, Xianlong Ma, Xianying Tan, Xiao Lin, Xiaocong Wang, Xiaofei Peng, Xiaofeng Wu, Xiaofeng Xu, Xiaojian Yuan, Xiaolan Cui, Xiaopeng Han, Xiaotian Liu, Xiaoxiong Fei, Xiaoxu Wu, Xiaoxuan Wang, Xiaoyu Zhang, Xiaoyu Sun, Xin Wang, Xin Huang, Xinhui Zhu, Xinming Yu, Xintong Xu, Xinyi Wang, Xinyu Li, Xiuxian Zhu, XuanShi Xu, Xue Lv, Xueying Li, Xuhong Wei, Xulong Chen, Xuyi Shi, Yabing Wang, Yafeng Li, Yamei Liu, Yan Cheng, Yanfu Gao, Yang Liang, Yang Wang, Yang Wang, Yang Yang, Yang Liu, Yanlong Fu, Yannian Wang, Yanpeng Lin, Yanzheng Chen, Yao Shen, Yaozong Han, Yaqian Yang, Yehua Chai, Yekun Wang, Yesong Song, Yi Zhang, Yichen Wang, Yifei Guo, Yifeng Kou, Yifeng Chen, Yilong Guo, Yilong Wang, Yiming Chen, Ying Wang, Ying Wu, Yingsheng Lin, Yingzhan Yang, Yinqi Xing, Yiran Lei, Yishu Tu, Yixiang Chen, Yiyan Zhang, Yong Li, Yonghua Ma, Yongqiang Dai, Yongxing Zhang, Yongyue Ran, Yu Sun, Yu Zhang, Yu-Wen Michael Liu, Yuang Liu, Yuanle Zhou, Yuanyuan Zhang, Yubo Han, Yuchen Wang, Yucheng Gao, Yude Luo, Yuedong Dong, Yuehu Hu, Yufeng Cao, Yuhui Yun, Yuhui Chen, Yukun Gao, Yukun Li, Yukun Zhang, Yumeng Fan, Yun Ma, Yun Zhang, Yunfei Xie, Yunshen Xu, Yuping Zhang, Yuqin Liu, Yuqing Li, Yurui Wang, Yuwen Lu, Yuxiang Cai, Zefeng Zhao, Zelin Zhang, Zelun Lin, Zenan Dong, Zezhao Pan, Zhaowu Liu, Zhaoyu Dong, Zhe Zhang, Zhe Zhang, Zhen Wu, Zhengfan Wei, Zhengrui Ning, Zhengsheng Li, Zhenxing Li, Zhenyu Qian, Zhenyu Li, Zhenyun Li, Zhi Chen, Zhichao Dong, Zhicheng Feng, Zhida Feng, Zhifan Deng, Zhihao Yu, Zhijin Chen, Zhiyang Zheng, Zhonghui Guo, Zhuangzhuang Zhang, Zhujun Sun, Zhuo Liu, Zichang Lin, Zihan Huang, Zihao Zhu, Zihe Zhao, Ziheng Chen, Ziping Zhu, Zixuan Xu, Ziyang Liang, Ziyi Gao, Ziyuan Computation and Language In this report, we introduce ERNIE 5.0, a natively autoregressive foundation model desinged for unified multimodal understanding and generation across text, image, video, and audio. All modalities are trained from scratch under a unified next-group-of-tokens prediction objective, based on an ultra-sparse mixture-of-experts (MoE) architecture with modality-agnostic expert routing. To address practical challenges in large-scale deployment under diverse resource constraints, ERNIE 5.0 adopts a novel elastic training paradigm. Within a single pre-training run, the model learns a family of sub-models with varying depths, expert capacities, and routing sparsity, enabling flexible trade-offs among performance, model size, and inference latency in memory- or time-constrained scenarios. Moreover, we systematically address the challenges of scaling reinforcement learning to unified foundation models, thereby guaranteeing efficient and stable post-training under ultra-sparse MoE architectures and diverse multimodal settings. Extensive experiments demonstrate that ERNIE 5.0 achieves strong and balanced performance across multiple modalities. To the best of our knowledge, among publicly disclosed models, ERNIE 5.0 represents the first production-scale realization of a trillion-parameter unified autoregressive model that supports both multimodal understanding and generation. To facilitate further research, we present detailed visualizations of modality-agnostic expert routing in the unified model, alongside comprehensive empirical analysis of elastic training, aiming to offer profound insights to the community. |
| title | ERNIE 5.0 Technical Report |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2602.04705 |