Staff View: :: Library Catalog

Saved in:

Bibliographic Details
Main Authors:	Seed, Li, Baisheng, Wu, Banggu, Ma, Bole, Xiao, Bowen, Zhang, Chaoyi, Li, Cheng, Wang, Chengyi, Xu, Chengyin, Zhang, Chi, Hu, Chong, Zan, Daoguang, Zhu, Defa, Xu, Dongyu, Li, Du, Wu, Faming, Xia, Fan, Zhang, Ge, Shi, Guang, Chen, Haobin, Zhu, Hongyu, Huang, Hongzhi, Zhou, Huan, Dou, Huanzhang, Duan, Jianhui, Lu, Jianqiao, Jiang, Jianyu, Xu, Jiayi, Chen, Jiecao, Chen, Jin, Ma, Jin, Su, Jing, Chen, Jingji, Wang, Jun, Yuan, Jun, Liu, Juncai, Zhou, Jundong, Hua, Kai, Shen, Kai, Xiang, Kai, Chen, Kaiyuan, Liu, Kang, Shen, Ke, Xiang, Liang, Yan, Lin, Luo, Lishu, Zhang, Mengyao, Ding, Ming, Zhang, Mofan, Liang, Nianning, Li, Peng, Huang, Penghao, Mu, Pengpeng, Huang, Qi, Ma, Qianli, Min, Qiyang, Yu, Qiying, Pang, Renming, Zhang, Ru, Yan, Shen, Zhao, Shixiong, Cao, Shuaishuai, Wu, Shuang, Chen, Siyan, Li, Siyu, Qiao, Siyuan, Sun, Tao, Xin, Tian, Fan, Tiantian, Huang, Ting, Fan, Ting-Han, Jia, Wei, Zhang, Wenqiang, Liu, Wenxuan, Wu, Xiangzhong, Zuo, Xiaochen, Jia, Xiaoying, Yang, Ximing, Liu, Xin, Yu, Xin, Bin, Xingyan, Hao, Xintong, Luo, Xiongcai, Li, Xujing, Zhou, Xun, Peng, Yanghua, Chen, Yangrui, Lin, Yi, Leng, Yichong, Li, Yinghao, Song, Yingshuan, Ma, Yiyuan, Shan, Yong, Xiang, Yongan, Wu, Yonghui, Zhang, Yongtao, Yao, Yongzhen, Bao, Yu, Yang, Yuehang, Yuan, Yufeng, Li, Yunshui, Xian, Yuqiao, Zeng, Yutao, Wang, Yuxuan, Hong, Zehua, Wang, Zehua, Wang, Zengzhi, Yang, Zeyu, Yin, Zhengqiang, Lu, Zhenyi, Zhang, Zhexi, Chen, Zhi, Zhang, Zhi, Lin, Zhiqi, Huang, Zihao, Xu, Zilin, Wei, Ziyun, Wang, Zuo
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	https://arxiv.org/abs/2511.11238
Tags:	Add Tag No Tags, Be the first to tag this record!

_version_	1866909907806060544
author	Seed Li, Baisheng Wu, Banggu Ma, Bole Xiao, Bowen Zhang, Chaoyi Li, Cheng Wang, Chengyi Xu, Chengyin Zhang, Chi Hu, Chong Zan, Daoguang Zhu, Defa Xu, Dongyu Li, Du Wu, Faming Xia, Fan Zhang, Ge Shi, Guang Chen, Haobin Zhu, Hongyu Huang, Hongzhi Zhou, Huan Dou, Huanzhang Duan, Jianhui Lu, Jianqiao Jiang, Jianyu Xu, Jiayi Chen, Jiecao Chen, Jin Ma, Jin Su, Jing Chen, Jingji Wang, Jun Yuan, Jun Liu, Juncai Zhou, Jundong Hua, Kai Shen, Kai Xiang, Kai Chen, Kaiyuan Liu, Kang Shen, Ke Xiang, Liang Yan, Lin Luo, Lishu Zhang, Mengyao Ding, Ming Zhang, Mofan Liang, Nianning Li, Peng Huang, Penghao Mu, Pengpeng Huang, Qi Ma, Qianli Min, Qiyang Yu, Qiying Pang, Renming Zhang, Ru Yan, Shen Yan, Shen Zhao, Shixiong Cao, Shuaishuai Wu, Shuang Chen, Siyan Li, Siyu Qiao, Siyuan Sun, Tao Xin, Tian Fan, Tiantian Huang, Ting Fan, Ting-Han Jia, Wei Zhang, Wenqiang Liu, Wenxuan Wu, Xiangzhong Zuo, Xiaochen Jia, Xiaoying Yang, Ximing Liu, Xin Yu, Xin Bin, Xingyan Hao, Xintong Luo, Xiongcai Li, Xujing Zhou, Xun Peng, Yanghua Chen, Yangrui Lin, Yi Leng, Yichong Li, Yinghao Song, Yingshuan Ma, Yiyuan Shan, Yong Xiang, Yongan Wu, Yonghui Zhang, Yongtao Yao, Yongzhen Bao, Yu Yang, Yuehang Yuan, Yufeng Li, Yunshui Xian, Yuqiao Zeng, Yutao Wang, Yuxuan Hong, Zehua Wang, Zehua Wang, Zengzhi Yang, Zeyu Yin, Zhengqiang Lu, Zhenyi Zhang, Zhexi Chen, Zhi Zhang, Zhi Lin, Zhiqi Huang, Zihao Xu, Zilin Wei, Ziyun Wang, Zuo
author_facet	Seed Li, Baisheng Wu, Banggu Ma, Bole Xiao, Bowen Zhang, Chaoyi Li, Cheng Wang, Chengyi Xu, Chengyin Zhang, Chi Hu, Chong Zan, Daoguang Zhu, Defa Xu, Dongyu Li, Du Wu, Faming Xia, Fan Zhang, Ge Shi, Guang Chen, Haobin Zhu, Hongyu Huang, Hongzhi Zhou, Huan Dou, Huanzhang Duan, Jianhui Lu, Jianqiao Jiang, Jianyu Xu, Jiayi Chen, Jiecao Chen, Jin Ma, Jin Su, Jing Chen, Jingji Wang, Jun Yuan, Jun Liu, Juncai Zhou, Jundong Hua, Kai Shen, Kai Xiang, Kai Chen, Kaiyuan Liu, Kang Shen, Ke Xiang, Liang Yan, Lin Luo, Lishu Zhang, Mengyao Ding, Ming Zhang, Mofan Liang, Nianning Li, Peng Huang, Penghao Mu, Pengpeng Huang, Qi Ma, Qianli Min, Qiyang Yu, Qiying Pang, Renming Zhang, Ru Yan, Shen Yan, Shen Zhao, Shixiong Cao, Shuaishuai Wu, Shuang Chen, Siyan Li, Siyu Qiao, Siyuan Sun, Tao Xin, Tian Fan, Tiantian Huang, Ting Fan, Ting-Han Jia, Wei Zhang, Wenqiang Liu, Wenxuan Wu, Xiangzhong Zuo, Xiaochen Jia, Xiaoying Yang, Ximing Liu, Xin Yu, Xin Bin, Xingyan Hao, Xintong Luo, Xiongcai Li, Xujing Zhou, Xun Peng, Yanghua Chen, Yangrui Lin, Yi Leng, Yichong Li, Yinghao Song, Yingshuan Ma, Yiyuan Shan, Yong Xiang, Yongan Wu, Yonghui Zhang, Yongtao Yao, Yongzhen Bao, Yu Yang, Yuehang Yuan, Yufeng Li, Yunshui Xian, Yuqiao Zeng, Yutao Wang, Yuxuan Hong, Zehua Wang, Zehua Wang, Zengzhi Yang, Zeyu Yin, Zhengqiang Lu, Zhenyi Zhang, Zhexi Chen, Zhi Zhang, Zhi Lin, Zhiqi Huang, Zihao Xu, Zilin Wei, Ziyun Wang, Zuo
contents	We introduce Virtual Width Networks (VWN), a framework that delivers the benefits of wider representations without incurring the quadratic cost of increasing the hidden size. VWN decouples representational width from backbone width, expanding the embedding space while keeping backbone compute nearly constant. In our large-scale experiment, an 8-times expansion accelerates optimization by over 2 times for next-token and 3 times for next-2-token prediction. The advantage amplifies over training as both the loss gap grows and the convergence-speedup ratio increases, showing that VWN is not only token-efficient but also increasingly effective with scale. Moreover, we identify an approximately log-linear scaling relation between virtual width and loss reduction, offering an initial empirical basis and motivation for exploring virtual-width scaling as a new dimension of large-model efficiency.
format	Preprint
id	arxiv_https___arxiv_org_abs_2511_11238
institution	arXiv
publishDate	2025
record_format	arxiv
spellingShingle	Virtual Width Networks Seed Li, Baisheng Wu, Banggu Ma, Bole Xiao, Bowen Zhang, Chaoyi Li, Cheng Wang, Chengyi Xu, Chengyin Zhang, Chi Hu, Chong Zan, Daoguang Zhu, Defa Xu, Dongyu Li, Du Wu, Faming Xia, Fan Zhang, Ge Shi, Guang Chen, Haobin Zhu, Hongyu Huang, Hongzhi Zhou, Huan Dou, Huanzhang Duan, Jianhui Lu, Jianqiao Jiang, Jianyu Xu, Jiayi Chen, Jiecao Chen, Jin Ma, Jin Su, Jing Chen, Jingji Wang, Jun Yuan, Jun Liu, Juncai Zhou, Jundong Hua, Kai Shen, Kai Xiang, Kai Chen, Kaiyuan Liu, Kang Shen, Ke Xiang, Liang Yan, Lin Luo, Lishu Zhang, Mengyao Ding, Ming Zhang, Mofan Liang, Nianning Li, Peng Huang, Penghao Mu, Pengpeng Huang, Qi Ma, Qianli Min, Qiyang Yu, Qiying Pang, Renming Zhang, Ru Yan, Shen Yan, Shen Zhao, Shixiong Cao, Shuaishuai Wu, Shuang Chen, Siyan Li, Siyu Qiao, Siyuan Sun, Tao Xin, Tian Fan, Tiantian Huang, Ting Fan, Ting-Han Jia, Wei Zhang, Wenqiang Liu, Wenxuan Wu, Xiangzhong Zuo, Xiaochen Jia, Xiaoying Yang, Ximing Liu, Xin Yu, Xin Bin, Xingyan Hao, Xintong Luo, Xiongcai Li, Xujing Zhou, Xun Peng, Yanghua Chen, Yangrui Lin, Yi Leng, Yichong Li, Yinghao Song, Yingshuan Ma, Yiyuan Shan, Yong Xiang, Yongan Wu, Yonghui Zhang, Yongtao Yao, Yongzhen Bao, Yu Yang, Yuehang Yuan, Yufeng Li, Yunshui Xian, Yuqiao Zeng, Yutao Wang, Yuxuan Hong, Zehua Wang, Zehua Wang, Zengzhi Yang, Zeyu Yin, Zhengqiang Lu, Zhenyi Zhang, Zhexi Chen, Zhi Zhang, Zhi Lin, Zhiqi Huang, Zihao Xu, Zilin Wei, Ziyun Wang, Zuo Machine Learning Artificial Intelligence We introduce Virtual Width Networks (VWN), a framework that delivers the benefits of wider representations without incurring the quadratic cost of increasing the hidden size. VWN decouples representational width from backbone width, expanding the embedding space while keeping backbone compute nearly constant. In our large-scale experiment, an 8-times expansion accelerates optimization by over 2 times for next-token and 3 times for next-2-token prediction. The advantage amplifies over training as both the loss gap grows and the convergence-speedup ratio increases, showing that VWN is not only token-efficient but also increasingly effective with scale. Moreover, we identify an approximately log-linear scaling relation between virtual width and loss reduction, offering an initial empirical basis and motivation for exploring virtual-width scaling as a new dimension of large-model efficiency.
title	Virtual Width Networks
topic	Machine Learning Artificial Intelligence
url	https://arxiv.org/abs/2511.11238

Similar Items