Cover Image

Temporal Dependencies in In-Context Learning: The Role of Induction Heads

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Bajaj, Anooshka, Mistry, Deven Mahesh, Maini, Sahaj Singh, Aggarwal, Yash, Dickson, Billy, Tiganj, Zoran
Format:	Preprint
Published:	2026
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Beyond Semantics: How Temporal Biases Shape Retrieval in Transformer and State-Space Models
by: Bajaj, Anooshka, et al.
Published: (2025)

Emergence of Episodic Memory in Transformers: Characterizing Changes in Temporal Structure of Attention Scores During Training
by: Mistry, Deven Mahesh, et al.
Published: (2025)

Gradual Forgetting: Logarithmic Compression for Extending Transformer Context Windows
by: Dickson, Billy, et al.
Published: (2025)

Who Do LLMs Trust? Human Experts Matter More Than Other LLMs
by: Bajaj, Anooshka, et al.
Published: (2026)

High Volatility and Action Bias Distinguish LLMs from Humans in Group Coordination
by: Maini, Sahaj Singh, et al.
Published: (2026)

Different Paths to Harmful Compliance: Behavioral Side Effects and Mechanistic Divergence Across LLM Jailbreaks
by: Kabir, Md Rysul, et al.
Published: (2026)

Vision-language models learn the geometry of human perceptual space
by: Sanders, Craig, et al.
Published: (2025)

On the Emergence of Induction Heads for In-Context Learning
by: Musat, Tiberiu, et al.
Published: (2025)

Identifying Semantic Induction Heads to Understand In-Context Learning
by: Ren, Jie, et al.
Published: (2024)

In-Context Learning in Speech Language Models: Analyzing the Role of Acoustic Features, Linguistic Structure, and Induction Heads
by: Pouw, Charlotte, et al.
Published: (2026)

Beyond Induction Heads: In-Context Meta Learning Induces Multi-Phase Circuit Emergence
by: Minegishi, Gouki, et al.
Published: (2025)

RexBERT: Context Specialized Bidirectional Encoders for E-commerce
by: Bajaj, Rahul, et al.
Published: (2026)

Interpretable Next-token Prediction via the Generalized Induction Head
by: Kim, Eunji, et al.
Published: (2024)

REAMS: Reasoning Enhanced Algorithm for Maths Solving
by: Singh, Eishkaran, et al.
Published: (2025)

Fourier Head: Helping Large Language Models Learn Complex Probability Distributions
by: Gillman, Nate, et al.
Published: (2024)

Induction Head Toxicity Mechanistically Explains Repetition Curse in Large Language Models
by: Wang, Shuxun, et al.
Published: (2025)

LongHeads: Multi-Head Attention is Secretly a Long Context Processor
by: Lu, Yi, et al.
Published: (2024)

Which Attention Heads Matter for In-Context Learning?
by: Yin, Kayo, et al.
Published: (2025)

Unveiling Induction Heads: Provable Training Dynamics and Feature Learning in Transformers
by: Chen, Siyu, et al.
Published: (2024)

Timeline-based Sentence Decomposition with In-Context Learning for Temporal Fact Extraction
by: Chen, Jianhao, et al.
Published: (2024)

Language Models' Factuality Depends on the Language of Inquiry
by: Aggarwal, Tushar, et al.
Published: (2025)

Induction Signatures Are Not Enough: A Matched-Compute Study of Load-Bearing Structure in In-Context Learning
by: Sabry, Mohammed, et al.
Published: (2025)

Peeking Behind Closed Doors: Risks of LLM Evaluation by Private Data Curators
by: Bansal, Hritik, et al.
Published: (2025)

Rethinking the Chain-of-Thought: The Roles of In-Context Learning and Pre-trained Priors
by: Yang, Hao, et al.
Published: (2025)

Leveraging In-Context Learning for Language Model Agents
by: Gupta, Shivanshu, et al.
Published: (2025)

Fine-Grained Uncertainty Quantification for Long-Form Language Model Outputs: A Comparative Study
by: Bouchard, Dylan, et al.
Published: (2026)

Automated Extraction and Creation of FBS Design Reasoning Knowledge Graphs from Structured Data in Product Catalogues Lacking Contextual Information
by: Sahadevan, Vijayalaxmi, et al.
Published: (2024)

Temporal Knowledge Question Answering via Abstract Reasoning Induction
by: Chen, Ziyang, et al.
Published: (2023)

Discourse-Aware In-Context Learning for Temporal Expression Normalization
by: Gautam, Akash Kumar, et al.
Published: (2024)

Condition-Gated Reasoning for Context-Dependent Biomedical Question Answering
by: Parekh, Jash Rajesh, et al.
Published: (2026)

LycheeDecode: Accelerating Long-Context LLM Inference via Hybrid-Head Sparse Decoding
by: Lin, Gang, et al.
Published: (2026)

Exploring the Role of Transliteration in In-Context Learning for Low-resource Languages Written in Non-Latin Scripts
by: Ma, Chunlan, et al.
Published: (2024)

A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applications
by: Lin, Minhua, et al.
Published: (2025)

Learning Probabilistic Temporal Logic Specifications for Stochastic Systems
by: Roy, Rajarshi, et al.
Published: (2025)

Few-shot Transfer Learning for Knowledge Base Question Answering: Fusing Supervised Models with In-Context Learning
by: Patidar, Mayur, et al.
Published: (2023)

The Role of Diversity in In-Context Learning for Large Language Models
by: Xiao, Wenyang, et al.
Published: (2025)

OWL: Overcoming Window Length-Dependence in Speculative Decoding for Long-Context Inputs
by: Lee, Jaeseong, et al.
Published: (2025)

Linguistic Structure Induction from Language Models
by: Momen, Omar
Published: (2024)

Grammar Induction from Visual, Speech and Text
by: Zhao, Yu, et al.
Published: (2024)

Musical Phrase Segmentation via Grammatical Induction
by: Perkins, Reed, et al.
Published: (2024)