Image de couverture de livre

Parallax: Parameterized Local Linear Attention for Language Modeling

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Zuo, Yifei, Pai, Dhruv, Zeng, Zhichen, Dewulf, Alec, Hu, Shuming, Wang, Zhaoran
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning Artificial Intelligence Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Local Linear Attention: An Optimal Interpolation of Linear and Softmax Attention For Test-Time Regression
par: Zuo, Yifei, et autres
Publié: (2025)

BRiTE: Bootstrapping Reinforced Thinking Process to Enhance Language Model Reasoning
par: Zhong, Han, et autres
Publié: (2025)

ALAS: Autonomous Learning Agent for Self-Updating Language Models
par: Atreja, Dhruv
Publié: (2025)

Are Transformers Able to Reason by Connecting Separated Knowledge in Training Data?
par: Yin, Yutong, et autres
Publié: (2025)

Linear Chain Transformation: Expanding Optimization Dynamics for Fine-Tuning Large Language Models
par: Wang, Yulong, et autres
Publié: (2024)

When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models
par: You, Haoran, et autres
Publié: (2024)

Attention Speaks Volumes: Localizing and Mitigating Bias in Language Models
par: Adiga, Rishabh, et autres
Publié: (2024)

Weak-to-Strong Search: Align Large Language Models via Searching over Small Language Models
par: Zhou, Zhanhui, et autres
Publié: (2024)

Higher-order Linear Attention
par: Zhang, Yifan, et autres
Publié: (2025)

Mixture of Heterogeneous Grouped Experts for Language Modeling
par: Ma, Zhicheng, et autres
Publié: (2026)

Tactic: Adaptive Sparse Attention with Clustering and Distribution Fitting for Long-Context LLMs
par: Zhu, Kan, et autres
Publié: (2025)

Emulated Disalignment: Safety Alignment for Large Language Models May Backfire!
par: Zhou, Zhanhui, et autres
Publié: (2024)

Liger: Linearizing Large Language Models to Gated Recurrent Structures
par: Lan, Disen, et autres
Publié: (2025)

CORM: Cache Optimization with Recent Message for Large Language Model Inference
par: Dai, Jincheng, et autres
Publié: (2024)

MiniCPM-SALA: Hybridizing Sparse and Linear Attention for Efficient Long-Context Modeling
par: MiniCPM Team, et autres
Publié: (2026)

Untangling Component Imbalance in Hybrid Linear Attention Conversion Methods
par: Benfeghoul, Martin, et autres
Publié: (2025)

SAIL: Self-Improving Efficient Online Alignment of Large Language Models
par: Ding, Mucong, et autres
Publié: (2024)

Auto-ICL: In-Context Learning without Human Supervision
par: Yang, Jinghan, et autres
Publié: (2023)

When Attention Sink Emerges in Language Models: An Empirical View
par: Gu, Xiangming, et autres
Publié: (2024)

Equivalent Linear Mappings of Large Language Models
par: Golden, James R.
Publié: (2025)

Sliding Window Attention Training for Efficient Large Language Models
par: Fu, Zichuan, et autres
Publié: (2025)

ALPINE: Unveiling the Planning Capability of Autoregressive Learning in Language Models
par: Wang, Siwei, et autres
Publié: (2024)

SLOT: Structuring the Output of Large Language Models
par: Wang, Darren Yow-Bang, et autres
Publié: (2025)

Self-Improvement in Language Models: The Sharpening Mechanism
par: Huang, Audrey, et autres
Publié: (2024)

Token-Efficient Leverage Learning in Large Language Models
par: Zeng, Yuanhao, et autres
Publié: (2024)

Linear Representation Transferability Hypothesis: Leveraging Small Models to Steer Large Models
par: Bello, Femi, et autres
Publié: (2025)

The Linear Representation Hypothesis and the Geometry of Large Language Models
par: Park, Kiho, et autres
Publié: (2023)

Understanding and Improving Length Generalization in Hierarchical Sparse Attention Models
par: Leng, Jiaqi, et autres
Publié: (2025)

Scaling Attention to Very Long Sequences in Linear Time with Wavelet-Enhanced Random Spectral Attention (WERSA)
par: Dentamaro, Vincenzo
Publié: (2025)

Explicit Multi-head Attention for Inter-head Interaction in Large Language Models
par: Peng, Runyu, et autres
Publié: (2026)

LMDX: Language Model-based Document Information Extraction and Localization
par: Perot, Vincent, et autres
Publié: (2023)

Native Hybrid Attention for Efficient Sequence Modeling
par: Du, Jusen, et autres
Publié: (2025)

Shaping capabilities with token-level data filtering
par: Rathi, Neil, et autres
Publié: (2026)

HiCI: Hierarchical Construction-Integration for Long-Context Attention
par: Zeng, Xiangyu, et autres
Publié: (2026)

Learning to Reason as Action Abstractions with Scalable Mid-Training RL
par: Zhang, Shenao, et autres
Publié: (2025)

Instruction Following by Principled Boosting Attention of Large Language Models
par: Guardieiro, Vitoria, et autres
Publié: (2025)

Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Models
par: Duan, Jinhao, et autres
Publié: (2023)

LoLCATs: On Low-Rank Linearizing of Large Language Models
par: Zhang, Michael, et autres
Publié: (2024)

The Structure of Relation Decoding Linear Operators in Large Language Models
par: Christ, Miranda Anna, et autres
Publié: (2025)

Let Models Speak Ciphers: Multiagent Debate through Embeddings
par: Pham, Chau, et autres
Publié: (2023)