Image de couverture de livre

Massive Values in Self-Attention Modules are the Key to Contextual Knowledge Understanding

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Jin, Mingyu, Mei, Kai, Xu, Wujiang, Sun, Mingjie, Tang, Ruixiang, Du, Mengnan, Liu, Zirui, Zhang, Yongfeng
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

SAGE: An Agentic Explainer Framework for Interpreting SAE Features in Language Models
par: Han, Jiaojiao, et autres
Publié: (2025)

SAE-SSV: Supervised Steering in Sparse Representation Spaces for Reliable Control of Language Models
par: He, Zirui, et autres
Publié: (2025)

Time Series Forecasting with LLMs: Understanding and Enhancing Model Capabilities
par: Tang, Hua, et autres
Publié: (2024)

Knowledge Graph Large Language Model (KG-LLM) for Link Prediction
par: Shu, Dong, et autres
Publié: (2024)

OmniRouter: Budget and Performance Controllable Multi-LLM Routing
par: Mei, Kai, et autres
Publié: (2025)

MoralBench: Moral Evaluation of LLMs
par: Ji, Jianchao, et autres
Publié: (2024)

A-MEM: Agentic Memory for LLM Agents
par: Xu, Wujiang, et autres
Publié: (2025)

DBR: Divergence-Based Regularization for Debiasing Natural Language Understanding Models
par: Li, Zihao, et autres
Publié: (2025)

AIOS: LLM Agent Operating System
par: Mei, Kai, et autres
Publié: (2024)

What if LLMs Have Different World Views: Simulating Alien Civilizations with LLM-based Agents
par: Xue, Zhaoqian, et autres
Publié: (2024)

From Commands to Prompts: LLM-based Semantic File System for AIOS
par: Shi, Zeru, et autres
Publié: (2024)

Exploring Concept Depth: How Large Language Models Acquire Knowledge and Concept at Different Layers?
par: Jin, Mingyu, et autres
Publié: (2024)

Farther the Shift, Sparser the Representation: Analyzing OOD Mechanisms in LLMs
par: Jin, Mingyu, et autres
Publié: (2026)

Pooling and Semantic Shift: The Fundamental Challenges in Long Text Embedding and Retrieval
par: Gao, Hang, et autres
Publié: (2026)

A Single Layer to Explain Them All:Understanding Massive Activations in Large Language Models
par: Shi, Zeru, et autres
Publié: (2026)

The Impact of Reasoning Step Length on Large Language Models
par: Jin, Mingyu, et autres
Publié: (2024)

TrustAgent: Towards Safe and Trustworthy LLM-based Agents
par: Hua, Wenyue, et autres
Publié: (2024)

iAgent: LLM Agent as a Shield between User and Recommender Systems
par: Xu, Wujiang, et autres
Publié: (2025)

Disentangling Memory and Reasoning Ability in Large Language Models
par: Jin, Mingyu, et autres
Publié: (2024)

EPO: Entropy-regularized Policy Optimization for LLM Agents Reinforcement Learning
par: Xu, Wujiang, et autres
Publié: (2025)

FinAnchor: Aligned Multi-Model Representations for Financial Prediction
par: He, Zirui, et autres
Publié: (2026)

Decoding Knowledge in Large Language Models: A Framework for Categorization and Comprehension
par: Fang, Yanbo, et autres
Publié: (2025)

When Reward Hacking Rebounds: Understanding and Mitigating It with Representation-Level Signals
par: Wu, Rui, et autres
Publié: (2026)

Understanding the Physics of Key-Value Cache Compression for LLMs through Attention Dynamics
par: Ananthanarayanan, Samhruth, et autres
Publié: (2026)

Trust or Abstain? A Self-Aware RAG Approach
par: Zhu, Xi, et autres
Publié: (2026)

AIOS Compiler: LLM as Interpreter for Natural Language Programming and Flow Programming of AI Agents
par: Xu, Shuyuan, et autres
Publié: (2024)

SAE-FiRE: Enhancing Earnings Surprise Predictions Through Sparse Autoencoder Feature Selection
par: Zhang, Huopu, et autres
Publié: (2025)

MixSD: Mixed Contextual Self-Distillation for Knowledge Injection
par: Liu, Jiarui, et autres
Publié: (2026)

Context Copying Modulation: The Role of Entropy Neurons in Managing Parametric and Contextual Knowledge Conflicts
par: Tighidet, Zineddine, et autres
Publié: (2025)

Massive Activations in Large Language Models
par: Sun, Mingjie, et autres
Publié: (2024)

Health-LLM: Personalized Retrieval-Augmented Disease Prediction System
par: Yu, Qinkai, et autres
Publié: (2024)

Rep2Text: Decoding Full Text from a Single LLM Token Representation
par: Zhao, Haiyan, et autres
Publié: (2025)

LawLLM: Law Large Language Model for the US Legal System
par: Shu, Dong, et autres
Publié: (2024)

Data-centric NLP Backdoor Defense from the Lens of Memorization
par: Wang, Zhenting, et autres
Publié: (2024)

Towards Generalizable Implicit In-Context Learning with Attention Routing
par: Li, Jiaqian, et autres
Publié: (2025)

Structured Knowledge Representation through Contextual Pages for Retrieval-Augmented Generation
par: Li, Xinze, et autres
Publié: (2026)

Universal Activation Verbalizer: A Unified Framework for Cross-Model Activation Explanation
par: Zhao, Haiyan, et autres
Publié: (2026)

SpeCache: Speculative Key-Value Caching for Efficient Generation of LLMs
par: Jie, Shibo, et autres
Publié: (2025)

Cerebrum (AIOS SDK): A Platform for Agent Development, Deployment, Distribution, and Discovery
par: Rama, Balaji, et autres
Publié: (2025)

Language Ranker: A Metric for Quantifying LLM Performance Across High and Low-Resource Languages
par: Li, Zihao, et autres
Publié: (2024)