Cover Image

Scaling Laws for Linear Complexity Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Shen, Xuyang, Li, Dong, Leng, Ruitao, Qin, Zhen, Sun, Weigao, Zhong, Yiran
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Unlocking the Secrets of Linear Complexity Sequence Model from A Unified Perspective
by: Qin, Zhen, et al.
Published: (2024)

Various Lengths, Constant Speed: Efficient Language Modeling with Lightning Attention
by: Qin, Zhen, et al.
Published: (2024)

Linear Attention Sequence Parallelism
by: Sun, Weigao, et al.
Published: (2024)

Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models
by: Qin, Zhen, et al.
Published: (2024)

HGRN2: Gated Linear RNNs with State Expansion
by: Qin, Zhen, et al.
Published: (2024)

Elucidating the Design Space of Decay in Linear Attention
by: Qin, Zhen, et al.
Published: (2025)

CO2: Efficient Distributed Training with Full Communication-Computation Overlap
by: Sun, Weigao, et al.
Published: (2024)

TransNormerLLM: A Faster and Better Large Language Model with Improved TransNormer
by: Qin, Zhen, et al.
Published: (2023)

You Only Scan Once: Efficient Multi-dimension Sequential Modeling with LightNet
by: Qin, Zhen, et al.
Published: (2024)

LASP-2: Rethinking Sequence Parallelism for Linear Attention and Its Hybrid
by: Sun, Weigao, et al.
Published: (2025)

Rethinking RL Scaling for Vision Language Models: A Transparent, From-Scratch Framework and Comprehensive Evaluation Scheme
by: Ma, Yan, et al.
Published: (2025)

FlashSampling: Fast and Memory-Efficient Exact Sampling
by: Ruiz, Tomas, et al.
Published: (2026)

LLaMA-MoE v2: Exploring Sparsity of LLaMA from Perspective of Mixture-of-Experts with Post-Training
by: Qu, Xiaoye, et al.
Published: (2024)

Liger: Linearizing Large Language Models to Gated Recurrent Structures
by: Lan, Disen, et al.
Published: (2025)

Scaling Laws of Synthetic Data for Language Models
by: Qin, Zeyu, et al.
Published: (2025)

Linear-MoE: Linear Sequence Modeling Meets Mixture-of-Experts
by: Sun, Weigao, et al.
Published: (2025)

MoM: Linear Sequence Modeling with Mixture-of-Memories
by: Du, Jusen, et al.
Published: (2025)

Parallel Scaling Law for Language Models
by: Chen, Mouxiang, et al.
Published: (2025)

Scaling Laws for Fact Memorization of Large Language Models
by: Lu, Xingyu, et al.
Published: (2024)

Finding the Minimal Parameter Budget for Implicit Reasoning: A Data Complexity Driven Scaling Law for Language Models
by: Wang, Xinyi, et al.
Published: (2025)

Temporal Scaling Law for Large Language Models
by: Xiong, Yizhe, et al.
Published: (2024)

Autoregressive Image Generation with Linear Complexity: A Spatial-Aware Decay Perspective
by: Mao, Yuxin, et al.
Published: (2025)

Scaling Laws for Multilingual Language Models
by: He, Yifei, et al.
Published: (2024)

Scaling Laws for Discriminative Classification in Large Language Models
by: Wyatte, Dean, et al.
Published: (2024)

CogniBench: A Legal-inspired Framework and Dataset for Assessing Cognitive Faithfulness of Large Language Models
by: Tang, Xiaqiang, et al.
Published: (2025)

Can Language Models Discover Scaling Laws?
by: Lin, Haowei, et al.
Published: (2025)

On the Scaling Laws of Geographical Representation in Language Models
by: Godey, Nathan, et al.
Published: (2024)

When Reasoning Meets Its Laws
by: Zhang, Junyu, et al.
Published: (2025)

D-CPT Law: Domain-specific Continual Pre-Training Scaling Law for Large Language Models
by: Que, Haoran, et al.
Published: (2024)

InfoLaw: Information Scaling Laws for Large Language Models with Quality-Weighted Mixture Data and Repetition
by: Liu, Fengze, et al.
Published: (2026)

RWKV-X: A Linear Complexity Hybrid Language Model
by: Hou, Haowen, et al.
Published: (2025)

Scaling Laws for Code: Every Programming Language Matters
by: Yang, Jian, et al.
Published: (2025)

Scaling Laws for Upcycling Mixture-of-Experts Language Models
by: Liew, Seng Pei, et al.
Published: (2025)

MiniMax-01: Scaling Foundation Models with Lightning Attention
by: MiniMax, et al.
Published: (2025)

Native Hybrid Attention for Efficient Sequence Modeling
by: Du, Jusen, et al.
Published: (2025)

Exploring Scaling Laws for EHR Foundation Models
by: Zhang, Sheng, et al.
Published: (2025)

Exploring Training and Inference Scaling Laws in Generative Retrieval
by: Cai, Hongru, et al.
Published: (2025)

Scaling Laws for Post Training Quantized Large Language Models
by: Xu, Zifei, et al.
Published: (2024)

Scaling Law for Language Models Training Considering Batch Size
by: Shuai, Xian, et al.
Published: (2024)

Scaling Laws for Downstream Task Performance of Large Language Models
by: Isik, Berivan, et al.
Published: (2024)