Image de couverture de livre

MiMo-V2-Flash Technical Report

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Core Team, Xiao, Bangjun, Xia, Bingquan, Yang, Bo, Gao, Bofei, Shen, Bowen, Zhang, Chen, He, Chenhong, Lou, Chiheng, Luo, Fuli, Wang, Gang, Xie, Gang, Zhang, Hailin, Lv, Hanglong, Li, Hanyu, Chen, Heyu, Xu, Hongshen, Zhang, Houbin, Liu, Huaqiu, Duo, Jiangshan, Wei, Jianyu, Xiao, Jiebao, Dong, Jinhao, Shi, Jun, Hu, Junhao, Bao, Kainan, Zhou, Kang, Li, Lei, Zhao, Liang, Zhang, Linghao, Li, Peidian, Chen, Qianli, Liu, Shaohui, Yu, Shihua, Cao, Shijie, Chen, Shimao, Yu, Shouqiu, Liu, Shuo, Zhou, Tianling, Su, Weijiang, Wang, Weikun, Ma, Wenhan, Deng, Xiangwei, Mao, Bohan, Ye, Bowen, Cai, Can, Wang, Chenghua, Zhu, Chengxuan, Ma, Chong, Chen, Chun, Li, Chunan, Zhu, Dawei, Xiao, Deshan, Zhang, Dong, Zhang, Duo, Liu, Fangyue, Yang, Feiyu, Shi, Fengyuan, Wang, Guoan, Tian, Hao, Wu, Hao, Qu, Heng, Yi, Hongfei, An, Hongxu, Guan, Hongyi, Zhang, Xing, Song, Yifan, Yan, Yihan, Zhao, Yihao, Lai, Yingchun, Gao, Yizhao, Cheng, Yu, Tian, Yuanyuan, Wang, Yudong, Tang, Zhen, Tang, Zhengju, Wen, Zhengtao, Song, Zhichao, Zheng, Zhixian, Jiang, Zihan, Wen, Jian, Sun, Jiarui, Li, Jiawei, Xue, Jinlong, Xia, Jun, Fang, Kai, Zhu, Menghang, Chen, Nuo, Tu, Qian, Zhang, Qihao, Wang, Qiying, Li, Rang, Ma, Rui, Zhang, Shaolei, Wang, Shengfan, Li, Shicheng, Gu, Shuhao, Ren, Shuhuai, Deng, Sirui, Guo, Tao, Lu, Tianyang, Zhuang, Weiji, Zhang, Weikang, Xiong, Weimin, Huang, Wenshan, Yang, Wenyu, Zhang, Xin, Yong, Xing, Wang, Xu, Xie, Xueyang, Jiang, Yilin, Yang, Yixin, He, Yongzhe, Tu, Yu, Dong, Yuanliang, Liu, Yuchen, Ma, Yue, Yu, Yue, Xiang, Yuxing, Huang, Zhaojun, Lin, Zhenru, Xu, Zhipeng, Chen, Zhiyang, Deng, Zhonghua, Zhang, Zihan, Yue, Zihao
Format:	Preprint
Publié:	2026
Sujets:	Computation and Language Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

_version_	1866917188888166400
author	Core Team Xiao, Bangjun Xia, Bingquan Yang, Bo Gao, Bofei Shen, Bowen Zhang, Chen He, Chenhong Lou, Chiheng Luo, Fuli Wang, Gang Xie, Gang Zhang, Hailin Lv, Hanglong Li, Hanyu Chen, Heyu Xu, Hongshen Zhang, Houbin Liu, Huaqiu Duo, Jiangshan Wei, Jianyu Xiao, Jiebao Dong, Jinhao Shi, Jun Hu, Junhao Bao, Kainan Zhou, Kang Li, Lei Zhao, Liang Zhang, Linghao Li, Peidian Chen, Qianli Liu, Shaohui Yu, Shihua Cao, Shijie Chen, Shimao Yu, Shouqiu Liu, Shuo Zhou, Tianling Su, Weijiang Wang, Weikun Ma, Wenhan Deng, Xiangwei Mao, Bohan Ye, Bowen Cai, Can Wang, Chenghua Zhu, Chengxuan Ma, Chong Chen, Chun Li, Chunan Zhu, Dawei Xiao, Deshan Zhang, Dong Zhang, Duo Liu, Fangyue Yang, Feiyu Shi, Fengyuan Wang, Guoan Tian, Hao Wu, Hao Qu, Heng Yi, Hongfei An, Hongxu Guan, Hongyi Zhang, Xing Song, Yifan Yan, Yihan Zhao, Yihao Lai, Yingchun Gao, Yizhao Cheng, Yu Tian, Yuanyuan Wang, Yudong Tang, Zhen Tang, Zhengju Wen, Zhengtao Song, Zhichao Zheng, Zhixian Jiang, Zihan Wen, Jian Sun, Jiarui Li, Jiawei Xue, Jinlong Xia, Jun Fang, Kai Zhu, Menghang Chen, Nuo Tu, Qian Zhang, Qihao Wang, Qiying Li, Rang Ma, Rui Zhang, Shaolei Wang, Shengfan Li, Shicheng Gu, Shuhao Ren, Shuhuai Deng, Sirui Guo, Tao Lu, Tianyang Zhuang, Weiji Zhang, Weikang Xiong, Weimin Huang, Wenshan Yang, Wenyu Zhang, Xin Yong, Xing Wang, Xu Xie, Xueyang Jiang, Yilin Yang, Yixin He, Yongzhe Tu, Yu Dong, Yuanliang Liu, Yuchen Ma, Yue Yu, Yue Xiang, Yuxing Huang, Zhaojun Lin, Zhenru Xu, Zhipeng Chen, Zhiyang Deng, Zhonghua Zhang, Zihan Yue, Zihao
author_facet	Core Team Xiao, Bangjun Xia, Bingquan Yang, Bo Gao, Bofei Shen, Bowen Zhang, Chen He, Chenhong Lou, Chiheng Luo, Fuli Wang, Gang Xie, Gang Zhang, Hailin Lv, Hanglong Li, Hanyu Chen, Heyu Xu, Hongshen Zhang, Houbin Liu, Huaqiu Duo, Jiangshan Wei, Jianyu Xiao, Jiebao Dong, Jinhao Shi, Jun Hu, Junhao Bao, Kainan Zhou, Kang Li, Lei Zhao, Liang Zhang, Linghao Li, Peidian Chen, Qianli Liu, Shaohui Yu, Shihua Cao, Shijie Chen, Shimao Yu, Shouqiu Liu, Shuo Zhou, Tianling Su, Weijiang Wang, Weikun Ma, Wenhan Deng, Xiangwei Mao, Bohan Ye, Bowen Cai, Can Wang, Chenghua Zhu, Chengxuan Ma, Chong Chen, Chun Li, Chunan Zhu, Dawei Xiao, Deshan Zhang, Dong Zhang, Duo Liu, Fangyue Yang, Feiyu Shi, Fengyuan Wang, Guoan Tian, Hao Wu, Hao Qu, Heng Yi, Hongfei An, Hongxu Guan, Hongyi Zhang, Xing Song, Yifan Yan, Yihan Zhao, Yihao Lai, Yingchun Gao, Yizhao Cheng, Yu Tian, Yuanyuan Wang, Yudong Tang, Zhen Tang, Zhengju Wen, Zhengtao Song, Zhichao Zheng, Zhixian Jiang, Zihan Wen, Jian Sun, Jiarui Li, Jiawei Xue, Jinlong Xia, Jun Fang, Kai Zhu, Menghang Chen, Nuo Tu, Qian Zhang, Qihao Wang, Qiying Li, Rang Ma, Rui Zhang, Shaolei Wang, Shengfan Li, Shicheng Gu, Shuhao Ren, Shuhuai Deng, Sirui Guo, Tao Lu, Tianyang Zhuang, Weiji Zhang, Weikang Xiong, Weimin Huang, Wenshan Yang, Wenyu Zhang, Xin Yong, Xing Wang, Xu Xie, Xueyang Jiang, Yilin Yang, Yixin He, Yongzhe Tu, Yu Dong, Yuanliang Liu, Yuchen Ma, Yue Yu, Yue Xiang, Yuxing Huang, Zhaojun Lin, Zhenru Xu, Zhipeng Chen, Zhiyang Deng, Zhonghua Zhang, Zihan Yue, Zihao
contents	We present MiMo-V2-Flash, a Mixture-of-Experts (MoE) model with 309B total parameters and 15B active parameters, designed for fast, strong reasoning and agentic capabilities. MiMo-V2-Flash adopts a hybrid attention architecture that interleaves Sliding Window Attention (SWA) with global attention, with a 128-token sliding window under a 5:1 hybrid ratio. The model is pre-trained on 27 trillion tokens with Multi-Token Prediction (MTP), employing a native 32k context length and subsequently extended to 256k. To efficiently scale post-training compute, MiMo-V2-Flash introduces a novel Multi-Teacher On-Policy Distillation (MOPD) paradigm. In this framework, domain-specialized teachers (e.g., trained via large-scale reinforcement learning) provide dense and token-level reward, enabling the student model to perfectly master teacher expertise. MiMo-V2-Flash rivals top-tier open-weight models such as DeepSeek-V3.2 and Kimi-K2, despite using only 1/2 and 1/3 of their total parameters, respectively. During inference, by repurposing MTP as a draft model for speculative decoding, MiMo-V2-Flash achieves up to 3.6 acceptance length and 2.6x decoding speedup with three MTP layers. We open-source both the model weights and the three-layer MTP weights to foster open research and community collaboration.
format	Preprint
id	arxiv_https___arxiv_org_abs_2601_02780
institution	arXiv
publishDate	2026
record_format	arxiv
spellingShingle	MiMo-V2-Flash Technical Report Core Team Xiao, Bangjun Xia, Bingquan Yang, Bo Gao, Bofei Shen, Bowen Zhang, Chen He, Chenhong Lou, Chiheng Luo, Fuli Wang, Gang Xie, Gang Zhang, Hailin Lv, Hanglong Li, Hanyu Chen, Heyu Xu, Hongshen Zhang, Houbin Liu, Huaqiu Duo, Jiangshan Wei, Jianyu Xiao, Jiebao Dong, Jinhao Shi, Jun Hu, Junhao Bao, Kainan Zhou, Kang Li, Lei Zhao, Liang Zhang, Linghao Li, Peidian Chen, Qianli Liu, Shaohui Yu, Shihua Cao, Shijie Chen, Shimao Yu, Shouqiu Liu, Shuo Zhou, Tianling Su, Weijiang Wang, Weikun Ma, Wenhan Deng, Xiangwei Mao, Bohan Ye, Bowen Cai, Can Wang, Chenghua Zhu, Chengxuan Ma, Chong Chen, Chun Li, Chunan Zhu, Dawei Xiao, Deshan Zhang, Dong Zhang, Duo Liu, Fangyue Yang, Feiyu Shi, Fengyuan Wang, Guoan Tian, Hao Wu, Hao Qu, Heng Yi, Hongfei An, Hongxu Guan, Hongyi Zhang, Xing Song, Yifan Yan, Yihan Zhao, Yihao Lai, Yingchun Gao, Yizhao Cheng, Yu Tian, Yuanyuan Wang, Yudong Tang, Zhen Tang, Zhengju Wen, Zhengtao Song, Zhichao Zheng, Zhixian Jiang, Zihan Wen, Jian Sun, Jiarui Li, Jiawei Xue, Jinlong Xia, Jun Fang, Kai Zhu, Menghang Chen, Nuo Tu, Qian Zhang, Qihao Wang, Qiying Li, Rang Ma, Rui Zhang, Shaolei Wang, Shengfan Li, Shicheng Gu, Shuhao Ren, Shuhuai Deng, Sirui Guo, Tao Lu, Tianyang Zhuang, Weiji Zhang, Weikang Xiong, Weimin Huang, Wenshan Yang, Wenyu Zhang, Xin Yong, Xing Wang, Xu Xie, Xueyang Jiang, Yilin Yang, Yixin He, Yongzhe Tu, Yu Dong, Yuanliang Liu, Yuchen Ma, Yue Yu, Yue Xiang, Yuxing Huang, Zhaojun Lin, Zhenru Xu, Zhipeng Chen, Zhiyang Deng, Zhonghua Zhang, Zihan Yue, Zihao Computation and Language Artificial Intelligence We present MiMo-V2-Flash, a Mixture-of-Experts (MoE) model with 309B total parameters and 15B active parameters, designed for fast, strong reasoning and agentic capabilities. MiMo-V2-Flash adopts a hybrid attention architecture that interleaves Sliding Window Attention (SWA) with global attention, with a 128-token sliding window under a 5:1 hybrid ratio. The model is pre-trained on 27 trillion tokens with Multi-Token Prediction (MTP), employing a native 32k context length and subsequently extended to 256k. To efficiently scale post-training compute, MiMo-V2-Flash introduces a novel Multi-Teacher On-Policy Distillation (MOPD) paradigm. In this framework, domain-specialized teachers (e.g., trained via large-scale reinforcement learning) provide dense and token-level reward, enabling the student model to perfectly master teacher expertise. MiMo-V2-Flash rivals top-tier open-weight models such as DeepSeek-V3.2 and Kimi-K2, despite using only 1/2 and 1/3 of their total parameters, respectively. During inference, by repurposing MTP as a draft model for speculative decoding, MiMo-V2-Flash achieves up to 3.6 acceptance length and 2.6x decoding speedup with three MTP layers. We open-source both the model weights and the three-layer MTP weights to foster open research and community collaboration.
title	MiMo-V2-Flash Technical Report
topic	Computation and Language Artificial Intelligence
url	https://arxiv.org/abs/2601.02780