Cover Image

Unlocking the Secrets of Linear Complexity Sequence Model from A Unified Perspective

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Qin, Zhen, Shen, Xuyang, Li, Dong, Sun, Weigao, Birchfield, Stan, Hartley, Richard, Zhong, Yiran
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Scaling Laws for Linear Complexity Language Models
by: Shen, Xuyang, et al.
Published: (2024)

Linear Attention Sequence Parallelism
by: Sun, Weigao, et al.
Published: (2024)

Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models
by: Qin, Zhen, et al.
Published: (2024)

HGRN2: Gated Linear RNNs with State Expansion
by: Qin, Zhen, et al.
Published: (2024)

Various Lengths, Constant Speed: Efficient Language Modeling with Lightning Attention
by: Qin, Zhen, et al.
Published: (2024)

Elucidating the Design Space of Decay in Linear Attention
by: Qin, Zhen, et al.
Published: (2025)

CO2: Efficient Distributed Training with Full Communication-Computation Overlap
by: Sun, Weigao, et al.
Published: (2024)

LASP-2: Rethinking Sequence Parallelism for Linear Attention and Its Hybrid
by: Sun, Weigao, et al.
Published: (2025)

TransNormerLLM: A Faster and Better Large Language Model with Improved TransNormer
by: Qin, Zhen, et al.
Published: (2023)

You Only Scan Once: Efficient Multi-dimension Sequential Modeling with LightNet
by: Qin, Zhen, et al.
Published: (2024)

LLaMA-MoE v2: Exploring Sparsity of LLaMA from Perspective of Mixture-of-Experts with Post-Training
by: Qu, Xiaoye, et al.
Published: (2024)

Linear-MoE: Linear Sequence Modeling Meets Mixture-of-Experts
by: Sun, Weigao, et al.
Published: (2025)

MoM: Linear Sequence Modeling with Mixture-of-Memories
by: Du, Jusen, et al.
Published: (2025)

FlashSampling: Fast and Memory-Efficient Exact Sampling
by: Ruiz, Tomas, et al.
Published: (2026)

Native Hybrid Attention for Efficient Sequence Modeling
by: Du, Jusen, et al.
Published: (2025)

Autoregressive Image Generation with Linear Complexity: A Spatial-Aware Decay Perspective
by: Mao, Yuxin, et al.
Published: (2025)

Rethinking RL Scaling for Vision Language Models: A Transparent, From-Scratch Framework and Comprehensive Evaluation Scheme
by: Ma, Yan, et al.
Published: (2025)

Liger: Linearizing Large Language Models to Gated Recurrent Structures
by: Lan, Disen, et al.
Published: (2025)

RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Robotics
by: Song, Chan Hee, et al.
Published: (2024)

FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects
by: Wen, Bowen, et al.
Published: (2023)

Unlocking Efficient Long-to-Short LLM Reasoning with Model Merging
by: Wu, Han, et al.
Published: (2025)

Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data
by: Ou, Jingyang, et al.
Published: (2024)

Your Transformer is Secretly Linear
by: Razzhigaev, Anton, et al.
Published: (2024)

RWKV-X: A Linear Complexity Hybrid Language Model
by: Hou, Haowen, et al.
Published: (2025)

Why Attention Patterns Exist: A Unifying Temporal Perspective Analysis
by: Yang, Qingyue, et al.
Published: (2026)

Fast-FoundationStereo: Real-Time Zero-Shot Stereo Matching
by: Wen, Bowen, et al.
Published: (2025)

AnyGPT: Unified Multimodal LLM with Discrete Sequence Modeling
by: Zhan, Jun, et al.
Published: (2024)

Youtu-LLM: Unlocking the Native Agentic Potential for Lightweight Large Language Models
by: Lu, Junru, et al.
Published: (2025)

Unlocking the Power of Large Language Models for Entity Alignment
by: Jiang, Xuhui, et al.
Published: (2024)

Gated Slot Attention for Efficient Linear-Time Sequence Modeling
by: Zhang, Yu, et al.
Published: (2024)

Pointer: Linear-Complexity Long-Range Modeling without Pre-training
by: Li, Zixi
Published: (2025)

Deep Non-rigid Structure-from-Motion Revisited: Canonicalization and Sequence Modeling
by: Deng, Hui, et al.
Published: (2024)

CogniBench: A Legal-inspired Framework and Dataset for Assessing Cognitive Faithfulness of Large Language Models
by: Tang, Xiaqiang, et al.
Published: (2025)

Parallelizing Linear Transformers with the Delta Rule over Sequence Length
by: Yang, Songlin, et al.
Published: (2024)

Sequence to Sequence Reward Modeling: Improving RLHF by Language Feedback
by: Zhou, Jiayi, et al.
Published: (2024)

Extending Context Window of Large Language Models from a Distributional Perspective
by: Wu, Yingsheng, et al.
Published: (2024)

Is Compression Really Linear with Code Intelligence?
by: Xuyang, Shijie, et al.
Published: (2025)

A Global Context Mechanism for Sequence Labeling
by: Xu, Conglei, et al.
Published: (2023)

Long-range Modeling and Processing of Multimodal Event Sequences
by: Li, Jichu, et al.
Published: (2026)

Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Space
by: Zhang, Zhen, et al.
Published: (2025)