Cover Image

Attention Editing: A Versatile Framework for Cross-Architecture Attention Conversion

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Cheng, Zhen, Yang, Hao-Bo, Huang, Wan-Yi, Li, Jin-Long
Format:	Preprint
Published:	2026
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

CATP: Cross-Attention Token Pruning for Accuracy Preserved Multimodal Model Inference
by: Liao, Ruqi, et al.
Published: (2024)

Revealing and Mitigating Over-Attention in Knowledge Editing
by: Wang, Pinzheng, et al.
Published: (2025)

Mixture-of-Depths Attention
by: Zhu, Lianghui, et al.
Published: (2026)

Cross-Attention Speculative Decoding
by: Zhong, Wei, et al.
Published: (2025)

HySparse: A Hybrid Sparse Attention Architecture with Oracle Token Selection and KV Cache Sharing
by: Gao, Yizhao, et al.
Published: (2026)

Decoupling Knowledge and Reasoning in Transformers: A Modular Architecture with Generalized Cross-Attention
by: Guo, Zhenyu, et al.
Published: (2025)

Mass-Editing Memory with Attention in Transformers: A cross-lingual exploration of knowledge
by: Tamayo, Daniel, et al.
Published: (2025)

Cross-Attention Watermarking of Large Language Models
by: Baldassini, Folco Bertini, et al.
Published: (2024)

Multi-granularity Interactive Attention Framework for Residual Hierarchical Pronunciation Assessment
by: Han, Hong, et al.
Published: (2026)

Round Attention: A Novel Round-Level Attention Mechanism to Accelerate LLM Inference
by: Tang, Yaohua, et al.
Published: (2025)

A Static and Dynamic Attention Framework for Multi Turn Dialogue Generation
by: Zhang, Wei-Nan, et al.
Published: (2024)

Every Attention Matters: An Efficient Hybrid Architecture for Long-Context Reasoning
by: Ling Team, et al.
Published: (2025)

MSWA: Refining Local Attention with Multi-ScaleWindow Attention
by: Xu, Yixing, et al.
Published: (2025)

Less Is More: Fast and Accurate Reasoning with Cross-Head Unified Sparse Attention
by: Yang, Lijie, et al.
Published: (2025)

Cross-Layer Attention Probing for Fine-Grained Hallucination Detection
by: Suresh, Malavika, et al.
Published: (2025)

Enhancing Essay Scoring with Adversarial Weights Perturbation and Metric-specific AttentionPooling
by: Huang, Jiaxin, et al.
Published: (2024)

Spectral Attention Steering for Prompt Highlighting
by: Li, Weixian Waylon, et al.
Published: (2026)

ReAttention: Training-Free Infinite Context with Finite Attention Scope
by: Liu, Xiaoran, et al.
Published: (2024)

C2LLM Technical Report: A New Frontier in Code Retrieval via Adaptive Cross-Attention Pooling
by: Qin, Jin, et al.
Published: (2025)

Sentinel: Decoding Context Utilization via Attention Probing for Efficient LLM Context Compression
by: Zhang, Yong, et al.
Published: (2025)

RACCooN: A Versatile Instructional Video Editing Framework with Auto-Generated Narratives
by: Yoon, Jaehong, et al.
Published: (2024)

Multi-View Attention Syntactic Enhanced Graph Convolutional Network for Aspect-based Sentiment Analysis
by: Huang, Xiang, et al.
Published: (2025)

Mitigating Attention Localization in Small Scale: Self-Attention Refinement via One-step Belief Propagation
by: Lee, Nakyung, et al.
Published: (2025)

Higher-order Linear Attention
by: Zhang, Yifan, et al.
Published: (2025)

Cross-Attention is Half Explanation in Speech-to-Text Models
by: Papi, Sara, et al.
Published: (2025)

Attention-MoA: Enhancing Mixture-of-Agents via Inter-Agent Semantic Attention and Deep Residual Synthesis
by: Wen, Jianyu, et al.
Published: (2026)

A3 : an Analytical Low-Rank Approximation Framework for Attention
by: Wong, Jeffrey T. H., et al.
Published: (2025)

CrashSage: A Large Language Model-Centered Framework for Contextual and Interpretable Traffic Crash Analysis
by: Zhen, Hao, et al.
Published: (2025)

Align, Don't Divide: Revisiting the LoRA Architecture in Multi-Task Learning
by: Liu, Jinda, et al.
Published: (2025)

DashAttention: Differentiable and Adaptive Sparse Hierarchical Attention
by: Huang, Yuxiang, et al.
Published: (2026)

Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models
by: Qin, Zhen, et al.
Published: (2024)

Elastic Attention: Test-time Adaptive Sparsity Ratios for Efficient Transformers
by: Tang, Zecheng, et al.
Published: (2026)

Untangling Component Imbalance in Hybrid Linear Attention Conversion Methods
by: Benfeghoul, Martin, et al.
Published: (2025)

GTA: Grouped-head latenT Attention
by: Sun, Luoyang, et al.
Published: (2025)

Full Attention Strikes Back: Transferring Full Attention into Sparse within Hundred Training Steps
by: Zhou, Yanke, et al.
Published: (2026)

Affine-Scaled Attention: Towards Flexible and Stable Transformer Attention
by: Bae, Jeongin, et al.
Published: (2026)

Hybrid Linear Attention Done Right: Efficient Distillation and Effective Architectures for Extremely Long Contexts
by: Chen, Yingfa, et al.
Published: (2026)

BMIKE-53: Investigating Cross-Lingual Knowledge Editing with In-Context Learning
by: Nie, Ercong, et al.
Published: (2024)

How Sparse Attention Approximates Exact Attention? Your Attention is Naturally $n^C$-Sparse
by: Deng, Yichuan, et al.
Published: (2024)

Attention-guided Evidence Grounding for Spoken Question Answering
by: Yang, Ke, et al.
Published: (2026)