Copertina

Decomposing Attention To Find Context-Sensitive Neurons

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autore principale:	Gibson, Alex
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Computation and Language Artificial Intelligence Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Finding Culture-Sensitive Neurons in Vision-Language Models
di: Zhao, Xiutian, et al.
Pubblicazione: (2025)

DATA: Decomposed Attention-based Task Adaptation for Rehearsal-Free Continual Learning
di: Liao, Huanxuan, et al.
Pubblicazione: (2025)

Too Long, Didn't Model: Decomposing LLM Long-Context Understanding With Novels
di: Hamilton, Sil, et al.
Pubblicazione: (2025)

Decomposing and Measuring Evaluation Awareness
di: Li, Changling, et al.
Pubblicazione: (2026)

Learning and Enforcing Context-Sensitive Control for LLMs
di: Albinhassan, Mohammad, et al.
Pubblicazione: (2026)

Which Attention Heads Matter for In-Context Learning?
di: Yin, Kayo, et al.
Pubblicazione: (2025)

SEAL: Scaling to Emphasize Attention for Long-Context Retrieval
di: Lee, Changhun, et al.
Pubblicazione: (2025)

Decomposing Representation Space into Interpretable Subspaces with Unsupervised Learning
di: Huang, Xinting, et al.
Pubblicazione: (2025)

Cost-Optimal Grouped-Query Attention for Long-Context Modeling
di: Chen, Yingfa, et al.
Pubblicazione: (2025)

MoBA: Mixture of Block Attention for Long-Context LLMs
di: Lu, Enzhe, et al.
Pubblicazione: (2025)

HiCI: Hierarchical Construction-Integration for Long-Context Attention
di: Zeng, Xiangyu, et al.
Pubblicazione: (2026)

In-Context Learning with Transformers: Softmax Attention Adapts to Function Lipschitzness
di: Collins, Liam, et al.
Pubblicazione: (2024)

SampleAttention: Near-Lossless Acceleration of Long Context LLM Inference with Adaptive Structured Sparse Attention
di: Zhu, Qianchao, et al.
Pubblicazione: (2024)

Neurons Speak in Ranges: Breaking Free from Discrete Neuronal Attribution
di: Haider, Muhammad Umair, et al.
Pubblicazione: (2025)

Large Language Models Are Latent Variable Models: Explaining and Finding Good Demonstrations for In-Context Learning
di: Wang, Xinyi, et al.
Pubblicazione: (2023)

LLM Assertiveness can be Mechanistically Decomposed into Emotional and Logical Components
di: Tsujimura, Hikaru, et al.
Pubblicazione: (2025)

SpreadsheetArena: Decomposing Preference in LLM Generation of Spreadsheet Workbooks
di: Kundurthy, Srivatsa, et al.
Pubblicazione: (2026)

Every Attention Matters: An Efficient Hybrid Architecture for Long-Context Reasoning
di: Ling Team, et al.
Pubblicazione: (2025)

Interpreting Context Look-ups in Transformers: Investigating Attention-MLP Interactions
di: Neo, Clement, et al.
Pubblicazione: (2024)

NeuronTune: Fine-Grained Neuron Modulation for Balanced Safety-Utility Alignment in LLMs
di: Pan, Birong, et al.
Pubblicazione: (2025)

Decomposing Elements of Problem Solving: What "Math" Does RL Teach?
di: Qin, Tian, et al.
Pubblicazione: (2025)

Tactic: Adaptive Sparse Attention with Clustering and Distribution Fitting for Long-Context LLMs
di: Zhu, Kan, et al.
Pubblicazione: (2025)

Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization
di: Hsieh, Cheng-Yu, et al.
Pubblicazione: (2024)

Analyzing the Attention Heads for Pronoun Disambiguation in Context-aware Machine Translation Models
di: Mąka, Paweł, et al.
Pubblicazione: (2024)

PRISM: A Geometric Risk Bound that Decomposes Drift into Scale, Shape, and Head
di: Lin, Chieh-Yen, et al.
Pubblicazione: (2026)

Distributional Associations vs In-Context Reasoning: A Study of Feed-forward and Attention Layers
di: Chen, Lei, et al.
Pubblicazione: (2024)

MiniCPM-SALA: Hybridizing Sparse and Linear Attention for Efficient Long-Context Modeling
di: MiniCPM Team, et al.
Pubblicazione: (2026)

Confidence Regulation Neurons in Language Models
di: Stolfo, Alessandro, et al.
Pubblicazione: (2024)

Training Acceleration of Low-Rank Decomposed Networks using Sequential Freezing and Rank Quantization
di: Hajimolahoseini, Habib, et al.
Pubblicazione: (2023)

Hybrid Linear Attention Done Right: Efficient Distillation and Effective Architectures for Extremely Long Contexts
di: Chen, Yingfa, et al.
Pubblicazione: (2026)

EDoRA: Efficient Weight-Decomposed Low-Rank Adaptation via Singular Value Decomposition
di: Nasiri, Hamid, et al.
Pubblicazione: (2025)

NEAT: Concept driven Neuron Attribution in LLMs
di: Kavuri, Vivek Hruday, et al.
Pubblicazione: (2025)

LaMDA: Large Model Fine-Tuning via Spectrally Decomposed Low-Dimensional Adaptation
di: Azizi, Seyedarmin, et al.
Pubblicazione: (2024)

3DS: Medical Domain Adaptation of LLMs via Decomposed Difficulty-based Data Selection
di: Ding, Hongxin, et al.
Pubblicazione: (2024)

CoSy: Evaluating Textual Explanations of Neurons
di: Kopf, Laura, et al.
Pubblicazione: (2024)

Universal Neurons in GPT2 Language Models
di: Gurnee, Wes, et al.
Pubblicazione: (2024)

CoCA: Fusing Position Embedding with Collinear Constrained Attention in Transformers for Long Context Window Extending
di: Zhu, Shiyi, et al.
Pubblicazione: (2023)

Fortify the Shortest Stave in Attention: Enhancing Context Awareness of Large Language Models for Effective Tool Use
di: Chen, Yuhan, et al.
Pubblicazione: (2023)

Context Is Not Comprehension
di: Pan, Alex, et al.
Pubblicazione: (2025)

DecomposeRL: Learning to Ask Useful, Informative, and Diverse Questions for Semi-Supervised, Traceable Claim Verification
di: Dipta, Shubhashis Roy, et al.
Pubblicazione: (2026)