Cover Image

Task Vector Geometry Underlies Dual Modes of Task Inference in Transformers

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yan, Hao, Yang, Haolin, Zhong, Yiqiao
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Unifying Attention Heads and Task Vectors via Hidden State Geometry in In-Context Learning
by: Yang, Haolin, et al.
Published: (2025)

How does Multi-Task Training Affect Transformer In-Context Capabilities? Investigations with Function Classes
by: Bhasin, Harmon, et al.
Published: (2024)

Exploring the Impact of a Transformer's Latent Space Geometry on Downstream Task Performance
by: Marbut, Anna C., et al.
Published: (2024)

Mechanism of Task-oriented Information Removal in In-context Learning
by: Cho, Hakaze, et al.
Published: (2025)

Adaptive Task Vectors for Large Language Models
by: Kang, Joonseong, et al.
Published: (2025)

Out-of-distribution generalization via composition: a lens through induction heads in Transformers
by: Song, Jiajun, et al.
Published: (2024)

The Geometries of Truth Are Orthogonal Across Tasks
by: Azizian, Waiss, et al.
Published: (2025)

The NLP Task Effectiveness of Long-Range Transformers
by: Qin, Guanghui, et al.
Published: (2022)

Circuit Component Reuse Across Tasks in Transformer Language Models
by: Merullo, Jack, et al.
Published: (2023)

A Framework for Evaluating LLMs Under Task Indeterminacy
by: Guerdan, Luke, et al.
Published: (2024)

Frustratingly Easy Task-aware Pruning for Large Language Models
by: Tian, Yuanhe, et al.
Published: (2025)

Identifying Factual Inconsistencies in Summaries: Grounding LLM Inference via Task Taxonomy
by: Xu, Liyan, et al.
Published: (2024)

Layer-Aware Task Arithmetic: Disentangling Task-Specific and Instruction-Following Knowledge
by: Chen, Yan-Lun, et al.
Published: (2025)

An Analysis of Embedding Layers and Similarity Scores using Siamese Neural Networks
by: Bingi, Yash, et al.
Published: (2023)

Emergence and Effectiveness of Task Vectors in In-Context Learning: An Encoder Decoder Perspective
by: Han, Seungwook, et al.
Published: (2024)

TRIM: Hybrid Inference via Targeted Stepwise Routing in Multi-Step Reasoning Tasks
by: Kapoor, Vansh, et al.
Published: (2026)

FLARE: Task-agnostic embedding model evaluation through a normalization process
by: Jiang, Jingzhou, et al.
Published: (2026)

Bridging the Knowledge Void: Inference-time Acquisition of Unfamiliar Programming Languages for Coding Tasks
by: Shen, Chen, et al.
Published: (2026)

IDT: Dual-Task Adversarial Attacks for Privacy Protection
by: Faustini, Pedro, et al.
Published: (2024)

ComplexFormer: Disruptively Advancing Transformer Inference Ability via Head-Specific Complex Vector Attention
by: Shao, Jintian, et al.
Published: (2025)

TELL-TALE: Task Efficient LLMs with Task Aware Layer Elimination
by: Naim, Omar, et al.
Published: (2025)

Do Transformers Use their Depth Adaptively? Evidence from a Relational Reasoning Task
by: Curth, Alicia, et al.
Published: (2026)

LeaPformer: Enabling Linear Transformers for Autoregressive and Simultaneous Tasks via Learned Proportions
by: Agostinelli, Victor, et al.
Published: (2024)

Disentangling Task Conflicts in Multi-Task LoRA via Orthogonal Gradient Projection
by: Yang, Ziyu, et al.
Published: (2026)

Benchmarking the Thinking Mode of Multimodal Large Language Models in Clinical Tasks
by: Hong, Jindong, et al.
Published: (2025)

Task-Aware LoRA Adapter Composition via Similarity Retrieval in Vector Databases
by: Adsul, Riya, et al.
Published: (2026)

Learning Task Representations from In-Context Learning
by: Saglam, Baturay, et al.
Published: (2025)

ComplexityNet: Increasing LLM Inference Efficiency by Learning Task Complexity
by: Bae, Henry, et al.
Published: (2023)

Diversifying the Expert Knowledge for Task-Agnostic Pruning in Sparse Mixture-of-Experts
by: Zhang, Zeliang, et al.
Published: (2024)

On the Geometry of Positional Encodings in Transformers
by: Cirrincione, Giansalvo
Published: (2026)

Investigating the Pre-Training Dynamics of In-Context Learning: Task Recognition vs. Task Learning
by: Wang, Xiaolei, et al.
Published: (2024)

RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrieval
by: Liu, Di, et al.
Published: (2024)

Task Vectors, Learned Not Extracted: Performance Gains and Mechanistic Insight
by: Yang, Haolin, et al.
Published: (2025)

Depth-Width tradeoffs in Algorithmic Reasoning of Graph Tasks with Transformers
by: Yehudai, Gilad, et al.
Published: (2025)

Weight-Inherited Distillation for Task-Agnostic BERT Compression
by: Wu, Taiqiang, et al.
Published: (2023)

EduRABSA: An Education Review Dataset for Aspect-based Sentiment Analysis Tasks
by: Hua, Yan Cathy, et al.
Published: (2025)

Lessons from the Use of Natural Language Inference (NLI) in Requirements Engineering Tasks
by: Fazelnia, Mohamad, et al.
Published: (2024)

Inference-Time Rethinking with Latent Thought Vectors for Math Reasoning
by: Kong, Deqian, et al.
Published: (2026)

Observable Propagation: Uncovering Feature Vectors in Transformers
by: Dunefsky, Jacob, et al.
Published: (2023)

Towards Understanding Multi-Task Learning (Generalization) of LLMs via Detecting and Exploring Task-Specific Neurons
by: Leng, Yongqi, et al.
Published: (2024)