:: Library Catalog

Image de couverture de livre

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	He, Zhengfu, Wang, Junxuan, Lin, Rui, Ge, Xuyang, Shu, Wentao, Tang, Qiong, Zhang, Junping, Qiu, Xipeng
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Computation and Language
Accès en ligne:	https://arxiv.org/abs/2504.20938
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Dimensional Collapse in Transformer Attention Outputs: A Challenge for Sparse Dictionary Learning
par: Wang, Junxuan, et autres
Publié: (2025)

Automatically Identifying Local and Global Circuits with Linear Computation Graphs
par: Ge, Xuyang, et autres
Publié: (2024)

Towards Universality: Studying Mechanistic Similarity Across Language Model Architectures
par: Wang, Junxuan, et autres
Publié: (2024)

Tracing the Thought of a Grandmaster-level Chess-Playing Transformer
par: Lin, Rui, et autres
Publié: (2026)

Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoders
par: He, Zhengfu, et autres
Publié: (2024)

Evolution of Concepts in Language Model Pre-Training
par: Ge, Xuyang, et autres
Publié: (2025)

Dictionary Learning Improves Patch-Free Circuit Discovery in Mechanistic Interpretability: A Case Study on Othello-GPT
par: He, Zhengfu, et autres
Publié: (2024)

A Distributional View for Visual Mechanistic Interpretability: KL-Minimal Soft-Constraint Principle
par: Zhou, Guancheng, et autres
Publié: (2026)

LoLA: Low-Rank Linear Attention With Sparse Caching
par: McDermott, Luke, et autres
Publié: (2025)

Sparse Attention Decomposition Applied to Circuit Tracing
par: Franco, Gabriel, et autres
Publié: (2024)

MAP: Revisiting Weight Decomposition for Low-Rank Adaptation
par: Si, Chongjie, et autres
Publié: (2025)

AdaLomo: Low-memory Optimization with Adaptive Learning Rate
par: Lv, Kai, et autres
Publié: (2023)

Assigning Distinct Roles to Quantized and Low-Rank Matrices Toward Optimal Weight Decomposition
par: Cho, Yoonjun, et autres
Publié: (2025)

How Attention Sinks Emerge in Large Language Models: An Interpretability Perspective
par: Peng, Runyu, et autres
Publié: (2026)

How Sparse Attention Approximates Exact Attention? Your Attention is Naturally $n^C$-Sparse
par: Deng, Yichuan, et autres
Publié: (2024)

DropLoRA: Sparse Low-Rank Adaptation for Parameter-Efficient Fine-Tuning
par: Zhang, Haojie
Publié: (2025)

Balanced Aggregation: Understanding and Fixing Aggregation Bias in GRPO
par: Zeng, Zhiyuan, et autres
Publié: (2026)

E-Sparse: Boosting the Large Language Model Inference through Entropy-based N:M Sparsity
par: Li, Yun, et autres
Publié: (2023)

Efficient Low Rank Attention for Long-Context Inference in Large Language Models
par: Li, Tenghui, et autres
Publié: (2025)

Eigen Attention: Attention in Low-Rank Space for KV Cache Compression
par: Saxena, Utkarsh, et autres
Publié: (2024)

From Sparse Dependence to Sparse Attention: Unveiling How Chain-of-Thought Enhances Transformer Sample Efficiency
par: Wen, Kaiyue, et autres
Publié: (2024)

MoR: Mixture of Ranks for Low-Rank Adaptation Tuning
par: Tang, Chuanyu, et autres
Publié: (2024)

A3 : an Analytical Low-Rank Approximation Framework for Attention
par: Wong, Jeffrey T. H., et autres
Publié: (2025)

Beyond Attention Magnitude: Leveraging Inter-layer Rank Consistency for Efficient Vision-Language-Action Models
par: Liu, Peiju, et autres
Publié: (2026)

FLoE: Fisher-Based Layer Selection for Efficient Sparse Adaptation of Low-Rank Experts
par: Wang, Xinyi, et autres
Publié: (2025)

Explicit Multi-head Attention for Inter-head Interaction in Large Language Models
par: Peng, Runyu, et autres
Publié: (2026)

Domain2Vec: Vectorizing Datasets to Find the Optimal Data Mixture without Training
par: Zhang, Mozhi, et autres
Publié: (2025)

Beyond Higher Rank: Token-wise Input-Output Projections for Efficient Low-Rank Adaptation
par: Li, Shiwei, et autres
Publié: (2025)

Alleviating Forgetfulness of Linear Attention by Hybrid Sparse Attention and Contextualized Learnable Token Eviction
par: He, Mutian, et autres
Publié: (2025)

Understanding and Improving Length Generalization in Hierarchical Sparse Attention Models
par: Leng, Jiaqi, et autres
Publié: (2025)

Towards Understanding the Robustness of Sparse Autoencoders
par: Saiyed, Ahson, et autres
Publié: (2026)

Dynamic Rank Reinforcement Learning for Adaptive Low-Rank Multi-Head Self Attention in Large Language Models
par: Erden, Caner
Publié: (2025)

FLuRKA: Fast and accurate unified Low-Rank & Kernel Attention
par: Gupta, Ahan, et autres
Publié: (2023)

Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention
par: Yuan, Jingyang, et autres
Publié: (2025)

Prism: Spectral-Aware Block-Sparse Attention
par: Wang, Xinghao, et autres
Publié: (2026)

Basis Selection: Low-Rank Decomposition of Pretrained Large Language Models for Target Applications
par: Li, Yang, et autres
Publié: (2024)

Linear Attention Sequence Parallelism
par: Sun, Weigao, et autres
Publié: (2024)

Revisiting Graph-Tokenizing Large Language Models: A Systematic Evaluation of Graph Token Understanding
par: Zhang, Zhongjian, et autres
Publié: (2026)

pQuant: Towards Effective Low-Bit Language Models via Decoupled Linear Quantization-Aware Training
par: Zhang, Wenzheng, et autres
Publié: (2026)

EDoRA: Efficient Weight-Decomposed Low-Rank Adaptation via Singular Value Decomposition
par: Nasiri, Hamid, et autres
Publié: (2025)