Staff View: :: Library Catalog

Saved in:

Bibliographic Details
Main Authors:	MiniMax, :, Chen, Aili, Li, Aonian, Gong, Bangwei, Jiang, Binyang, Fei, Bo, Yang, Bo, Shan, Boji, Yu, Changqing, Wang, Chao, Zhu, Cheng, Xiao, Chengjun, Du, Chengyu, Zhang, Chi, Qiao, Chu, Zhang, Chunhao, Du, Chunhui, Guo, Congchao, Chen, Da, Ding, Deming, Sun, Dianjun, Li, Dong, Jiao, Enwei, Zhou, Haigang, Zhang, Haimo, Ding, Han, Sun, Haohai, Feng, Haoyu, Cai, Huaiguang, Zhu, Haichao, Sun, Jian, Zhuang, Jiaqi, Cai, Jiaren, Song, Jiayuan, Zhu, Jin, Li, Jingyang, Tian, Jinhao, Liu, Jinli, Xu, Junhao, Yan, Junjie, Liu, Junteng, He, Junxian, Feng, Kaiyi, Yang, Ke, Xiao, Kecheng, Han, Le, Wang, Leyang, Yu, Lianfei, Feng, Liheng, Li, Lin, Zheng, Lin, Du, Linge, Yang, Lingyu, Zeng, Lunbin, Yu, Minghui, Tao, Mingliang, Chi, Mingyuan, Zhang, Mozhi, Lin, Mujie, Hu, Nan, Di, Nongyu, Gao, Peng, Li, Pengfei, Zhao, Pengyu, Ren, Qibing, Xu, Qidi, Li, Qile, Wang, Qin, Tian, Rong, Leng, Ruitao, Chen, Shaoxiang, Chen, Shaoyu, Shi, Shengmin, Weng, Shitong, Guan, Shuchang, Yu, Shuqi, Li, Sichen, Zhu, Songquan, Li, Tengfei, Cai, Tianchi, Liang, Tianrun, Cheng, Weiyu, Kong, Weize, Li, Wenkai, Chen, Xiancai, Song, Xiangjun, Luo, Xiao, Su, Xiao, Li, Xiaobo, Han, Xiaodong, Hou, Xinzhu, Lu, Xuan, Zou, Xun, Shen, Xuyang, Gong, Yan, Ma, Yan, Wang, Yang, Shi, Yiqi, Zhong, Yiran, Duan, Yonghong, Fu, Yongxiang, Hu, Yongyi, Gao, Yu, Fan, Yuanxiang, Yang, Yufeng, Li, Yuhao, Hu, Yulin, Huang, Yunan, Li, Yunji, Xu, Yunzhi, Mao, Yuxin, Shi, Yuxuan, Wenren, Yuze, Li, Zehan, Li, Zelin, Tian, Zhanxu, Zhu, Zhengmao, Fan, Zhenhua, Wu, Zhenzhen, Xu, Zhichao, Yu, Zhihang, Lyu, Zhiheng, Jiang, Zhuo, Gao, Zibo, Wu, Zijia, Song, Zijian, Sun, Zijun
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Machine Learning
Online Access:	https://arxiv.org/abs/2506.13585
Tags:	Add Tag No Tags, Be the first to tag this record!

_version_	1866911007523209216
author	MiniMax : Chen, Aili Li, Aonian Gong, Bangwei Jiang, Binyang Fei, Bo Yang, Bo Shan, Boji Yu, Changqing Wang, Chao Zhu, Cheng Xiao, Chengjun Du, Chengyu Zhang, Chi Qiao, Chu Zhang, Chunhao Du, Chunhui Guo, Congchao Chen, Da Ding, Deming Sun, Dianjun Li, Dong Jiao, Enwei Zhou, Haigang Zhang, Haimo Ding, Han Sun, Haohai Feng, Haoyu Cai, Huaiguang Zhu, Haichao Sun, Jian Zhuang, Jiaqi Cai, Jiaren Song, Jiayuan Zhu, Jin Li, Jingyang Tian, Jinhao Liu, Jinli Xu, Junhao Yan, Junjie Liu, Junteng He, Junxian Feng, Kaiyi Yang, Ke Xiao, Kecheng Han, Le Wang, Leyang Yu, Lianfei Feng, Liheng Li, Lin Zheng, Lin Du, Linge Yang, Lingyu Zeng, Lunbin Yu, Minghui Tao, Mingliang Chi, Mingyuan Zhang, Mozhi Lin, Mujie Hu, Nan Di, Nongyu Gao, Peng Li, Pengfei Zhao, Pengyu Ren, Qibing Xu, Qidi Li, Qile Wang, Qin Tian, Rong Leng, Ruitao Chen, Shaoxiang Chen, Shaoyu Shi, Shengmin Weng, Shitong Guan, Shuchang Yu, Shuqi Li, Sichen Zhu, Songquan Li, Tengfei Cai, Tianchi Liang, Tianrun Cheng, Weiyu Kong, Weize Li, Wenkai Chen, Xiancai Song, Xiangjun Luo, Xiao Su, Xiao Li, Xiaobo Han, Xiaodong Hou, Xinzhu Lu, Xuan Zou, Xun Shen, Xuyang Gong, Yan Ma, Yan Wang, Yang Shi, Yiqi Zhong, Yiran Duan, Yonghong Fu, Yongxiang Hu, Yongyi Gao, Yu Fan, Yuanxiang Yang, Yufeng Li, Yuhao Hu, Yulin Huang, Yunan Li, Yunji Xu, Yunzhi Mao, Yuxin Shi, Yuxuan Wenren, Yuze Li, Zehan Li, Zelin Tian, Zhanxu Zhu, Zhengmao Fan, Zhenhua Wu, Zhenzhen Xu, Zhichao Yu, Zhihang Lyu, Zhiheng Jiang, Zhuo Gao, Zibo Wu, Zijia Song, Zijian Sun, Zijun
author_facet	MiniMax : Chen, Aili Li, Aonian Gong, Bangwei Jiang, Binyang Fei, Bo Yang, Bo Shan, Boji Yu, Changqing Wang, Chao Zhu, Cheng Xiao, Chengjun Du, Chengyu Zhang, Chi Qiao, Chu Zhang, Chunhao Du, Chunhui Guo, Congchao Chen, Da Ding, Deming Sun, Dianjun Li, Dong Jiao, Enwei Zhou, Haigang Zhang, Haimo Ding, Han Sun, Haohai Feng, Haoyu Cai, Huaiguang Zhu, Haichao Sun, Jian Zhuang, Jiaqi Cai, Jiaren Song, Jiayuan Zhu, Jin Li, Jingyang Tian, Jinhao Liu, Jinli Xu, Junhao Yan, Junjie Liu, Junteng He, Junxian Feng, Kaiyi Yang, Ke Xiao, Kecheng Han, Le Wang, Leyang Yu, Lianfei Feng, Liheng Li, Lin Zheng, Lin Du, Linge Yang, Lingyu Zeng, Lunbin Yu, Minghui Tao, Mingliang Chi, Mingyuan Zhang, Mozhi Lin, Mujie Hu, Nan Di, Nongyu Gao, Peng Li, Pengfei Zhao, Pengyu Ren, Qibing Xu, Qidi Li, Qile Wang, Qin Tian, Rong Leng, Ruitao Chen, Shaoxiang Chen, Shaoyu Shi, Shengmin Weng, Shitong Guan, Shuchang Yu, Shuqi Li, Sichen Zhu, Songquan Li, Tengfei Cai, Tianchi Liang, Tianrun Cheng, Weiyu Kong, Weize Li, Wenkai Chen, Xiancai Song, Xiangjun Luo, Xiao Su, Xiao Li, Xiaobo Han, Xiaodong Hou, Xinzhu Lu, Xuan Zou, Xun Shen, Xuyang Gong, Yan Ma, Yan Wang, Yang Shi, Yiqi Zhong, Yiran Duan, Yonghong Fu, Yongxiang Hu, Yongyi Gao, Yu Fan, Yuanxiang Yang, Yufeng Li, Yuhao Hu, Yulin Huang, Yunan Li, Yunji Xu, Yunzhi Mao, Yuxin Shi, Yuxuan Wenren, Yuze Li, Zehan Li, Zelin Tian, Zhanxu Zhu, Zhengmao Fan, Zhenhua Wu, Zhenzhen Xu, Zhichao Yu, Zhihang Lyu, Zhiheng Jiang, Zhuo Gao, Zibo Wu, Zijia Song, Zijian Sun, Zijun
contents	We introduce MiniMax-M1, the world's first open-weight, large-scale hybrid-attention reasoning model. MiniMax-M1 is powered by a hybrid Mixture-of-Experts (MoE) architecture combined with a lightning attention mechanism. The model is developed based on our previous MiniMax-Text-01 model, which contains a total of 456 billion parameters with 45.9 billion parameters activated per token. The M1 model natively supports a context length of 1 million tokens, 8x the context size of DeepSeek R1. Furthermore, the lightning attention mechanism in MiniMax-M1 enables efficient scaling of test-time compute. These properties make M1 particularly suitable for complex tasks that require processing long inputs and thinking extensively. MiniMax-M1 is trained using large-scale reinforcement learning (RL) on diverse problems including sandbox-based, real-world software engineering environments. In addition to M1's inherent efficiency advantage for RL training, we propose CISPO, a novel RL algorithm to further enhance RL efficiency. CISPO clips importance sampling weights rather than token updates, outperforming other competitive RL variants. Combining hybrid-attention and CISPO enables MiniMax-M1's full RL training on 512 H800 GPUs to complete in only three weeks, with a rental cost of just $534,700. We release two versions of MiniMax-M1 models with 40K and 80K thinking budgets respectively, where the 40K model represents an intermediate phase of the 80K training. Experiments on standard benchmarks show that our models are comparable or superior to strong open-weight models such as the original DeepSeek-R1 and Qwen3-235B, with particular strengths in complex software engineering, tool utilization, and long-context tasks. We publicly release MiniMax-M1 at https://github.com/MiniMax-AI/MiniMax-M1.
format	Preprint
id	arxiv_https___arxiv_org_abs_2506_13585
institution	arXiv
publishDate	2025
record_format	arxiv
spellingShingle	MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention MiniMax : Chen, Aili Li, Aonian Gong, Bangwei Jiang, Binyang Fei, Bo Yang, Bo Shan, Boji Yu, Changqing Wang, Chao Zhu, Cheng Xiao, Chengjun Du, Chengyu Zhang, Chi Qiao, Chu Zhang, Chunhao Du, Chunhui Guo, Congchao Chen, Da Ding, Deming Sun, Dianjun Li, Dong Jiao, Enwei Zhou, Haigang Zhang, Haimo Ding, Han Sun, Haohai Feng, Haoyu Cai, Huaiguang Zhu, Haichao Sun, Jian Zhuang, Jiaqi Cai, Jiaren Song, Jiayuan Zhu, Jin Li, Jingyang Tian, Jinhao Liu, Jinli Xu, Junhao Yan, Junjie Liu, Junteng He, Junxian Feng, Kaiyi Yang, Ke Xiao, Kecheng Han, Le Wang, Leyang Yu, Lianfei Feng, Liheng Li, Lin Zheng, Lin Du, Linge Yang, Lingyu Zeng, Lunbin Yu, Minghui Tao, Mingliang Chi, Mingyuan Zhang, Mozhi Lin, Mujie Hu, Nan Di, Nongyu Gao, Peng Li, Pengfei Zhao, Pengyu Ren, Qibing Xu, Qidi Li, Qile Wang, Qin Tian, Rong Leng, Ruitao Chen, Shaoxiang Chen, Shaoyu Shi, Shengmin Weng, Shitong Guan, Shuchang Yu, Shuqi Li, Sichen Zhu, Songquan Li, Tengfei Cai, Tianchi Liang, Tianrun Cheng, Weiyu Kong, Weize Li, Wenkai Chen, Xiancai Song, Xiangjun Luo, Xiao Su, Xiao Li, Xiaobo Han, Xiaodong Hou, Xinzhu Lu, Xuan Zou, Xun Shen, Xuyang Gong, Yan Ma, Yan Wang, Yang Shi, Yiqi Zhong, Yiran Duan, Yonghong Fu, Yongxiang Hu, Yongyi Gao, Yu Fan, Yuanxiang Yang, Yufeng Li, Yuhao Hu, Yulin Huang, Yunan Li, Yunji Xu, Yunzhi Mao, Yuxin Shi, Yuxuan Wenren, Yuze Li, Zehan Li, Zelin Tian, Zhanxu Zhu, Zhengmao Fan, Zhenhua Wu, Zhenzhen Xu, Zhichao Yu, Zhihang Lyu, Zhiheng Jiang, Zhuo Gao, Zibo Wu, Zijia Song, Zijian Sun, Zijun Computation and Language Machine Learning We introduce MiniMax-M1, the world's first open-weight, large-scale hybrid-attention reasoning model. MiniMax-M1 is powered by a hybrid Mixture-of-Experts (MoE) architecture combined with a lightning attention mechanism. The model is developed based on our previous MiniMax-Text-01 model, which contains a total of 456 billion parameters with 45.9 billion parameters activated per token. The M1 model natively supports a context length of 1 million tokens, 8x the context size of DeepSeek R1. Furthermore, the lightning attention mechanism in MiniMax-M1 enables efficient scaling of test-time compute. These properties make M1 particularly suitable for complex tasks that require processing long inputs and thinking extensively. MiniMax-M1 is trained using large-scale reinforcement learning (RL) on diverse problems including sandbox-based, real-world software engineering environments. In addition to M1's inherent efficiency advantage for RL training, we propose CISPO, a novel RL algorithm to further enhance RL efficiency. CISPO clips importance sampling weights rather than token updates, outperforming other competitive RL variants. Combining hybrid-attention and CISPO enables MiniMax-M1's full RL training on 512 H800 GPUs to complete in only three weeks, with a rental cost of just $534,700. We release two versions of MiniMax-M1 models with 40K and 80K thinking budgets respectively, where the 40K model represents an intermediate phase of the 80K training. Experiments on standard benchmarks show that our models are comparable or superior to strong open-weight models such as the original DeepSeek-R1 and Qwen3-235B, with particular strengths in complex software engineering, tool utilization, and long-context tasks. We publicly release MiniMax-M1 at https://github.com/MiniMax-AI/MiniMax-M1.
title	MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention
topic	Computation and Language Machine Learning
url	https://arxiv.org/abs/2506.13585

Similar Items