Imagen de Portada

Temporal Dependencies in In-Context Learning: The Role of Induction Heads

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Bajaj, Anooshka, Mistry, Deven Mahesh, Maini, Sahaj Singh, Aggarwal, Yash, Dickson, Billy, Tiganj, Zoran
Formato:	Preprint
Publicado:	2026
Materias:	Computation and Language Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Beyond Semantics: How Temporal Biases Shape Retrieval in Transformer and State-Space Models
por: Bajaj, Anooshka, et al.
Publicado: (2025)

Emergence of Episodic Memory in Transformers: Characterizing Changes in Temporal Structure of Attention Scores During Training
por: Mistry, Deven Mahesh, et al.
Publicado: (2025)

Gradual Forgetting: Logarithmic Compression for Extending Transformer Context Windows
por: Dickson, Billy, et al.
Publicado: (2025)

Who Do LLMs Trust? Human Experts Matter More Than Other LLMs
por: Bajaj, Anooshka, et al.
Publicado: (2026)

High Volatility and Action Bias Distinguish LLMs from Humans in Group Coordination
por: Maini, Sahaj Singh, et al.
Publicado: (2026)

Different Paths to Harmful Compliance: Behavioral Side Effects and Mechanistic Divergence Across LLM Jailbreaks
por: Kabir, Md Rysul, et al.
Publicado: (2026)

Vision-language models learn the geometry of human perceptual space
por: Sanders, Craig, et al.
Publicado: (2025)

On the Emergence of Induction Heads for In-Context Learning
por: Musat, Tiberiu, et al.
Publicado: (2025)

Identifying Semantic Induction Heads to Understand In-Context Learning
por: Ren, Jie, et al.
Publicado: (2024)

In-Context Learning in Speech Language Models: Analyzing the Role of Acoustic Features, Linguistic Structure, and Induction Heads
por: Pouw, Charlotte, et al.
Publicado: (2026)

Beyond Induction Heads: In-Context Meta Learning Induces Multi-Phase Circuit Emergence
por: Minegishi, Gouki, et al.
Publicado: (2025)

RexBERT: Context Specialized Bidirectional Encoders for E-commerce
por: Bajaj, Rahul, et al.
Publicado: (2026)

Interpretable Next-token Prediction via the Generalized Induction Head
por: Kim, Eunji, et al.
Publicado: (2024)

REAMS: Reasoning Enhanced Algorithm for Maths Solving
por: Singh, Eishkaran, et al.
Publicado: (2025)

Fourier Head: Helping Large Language Models Learn Complex Probability Distributions
por: Gillman, Nate, et al.
Publicado: (2024)

Induction Head Toxicity Mechanistically Explains Repetition Curse in Large Language Models
por: Wang, Shuxun, et al.
Publicado: (2025)

LongHeads: Multi-Head Attention is Secretly a Long Context Processor
por: Lu, Yi, et al.
Publicado: (2024)

Which Attention Heads Matter for In-Context Learning?
por: Yin, Kayo, et al.
Publicado: (2025)

Unveiling Induction Heads: Provable Training Dynamics and Feature Learning in Transformers
por: Chen, Siyu, et al.
Publicado: (2024)

Timeline-based Sentence Decomposition with In-Context Learning for Temporal Fact Extraction
por: Chen, Jianhao, et al.
Publicado: (2024)

Language Models' Factuality Depends on the Language of Inquiry
por: Aggarwal, Tushar, et al.
Publicado: (2025)

Induction Signatures Are Not Enough: A Matched-Compute Study of Load-Bearing Structure in In-Context Learning
por: Sabry, Mohammed, et al.
Publicado: (2025)

Peeking Behind Closed Doors: Risks of LLM Evaluation by Private Data Curators
por: Bansal, Hritik, et al.
Publicado: (2025)

Rethinking the Chain-of-Thought: The Roles of In-Context Learning and Pre-trained Priors
por: Yang, Hao, et al.
Publicado: (2025)

Leveraging In-Context Learning for Language Model Agents
por: Gupta, Shivanshu, et al.
Publicado: (2025)

Fine-Grained Uncertainty Quantification for Long-Form Language Model Outputs: A Comparative Study
por: Bouchard, Dylan, et al.
Publicado: (2026)

Automated Extraction and Creation of FBS Design Reasoning Knowledge Graphs from Structured Data in Product Catalogues Lacking Contextual Information
por: Sahadevan, Vijayalaxmi, et al.
Publicado: (2024)

Temporal Knowledge Question Answering via Abstract Reasoning Induction
por: Chen, Ziyang, et al.
Publicado: (2023)

Discourse-Aware In-Context Learning for Temporal Expression Normalization
por: Gautam, Akash Kumar, et al.
Publicado: (2024)

Condition-Gated Reasoning for Context-Dependent Biomedical Question Answering
por: Parekh, Jash Rajesh, et al.
Publicado: (2026)

LycheeDecode: Accelerating Long-Context LLM Inference via Hybrid-Head Sparse Decoding
por: Lin, Gang, et al.
Publicado: (2026)

Exploring the Role of Transliteration in In-Context Learning for Low-resource Languages Written in Non-Latin Scripts
por: Ma, Chunlan, et al.
Publicado: (2024)

A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applications
por: Lin, Minhua, et al.
Publicado: (2025)

Learning Probabilistic Temporal Logic Specifications for Stochastic Systems
por: Roy, Rajarshi, et al.
Publicado: (2025)

Few-shot Transfer Learning for Knowledge Base Question Answering: Fusing Supervised Models with In-Context Learning
por: Patidar, Mayur, et al.
Publicado: (2023)

The Role of Diversity in In-Context Learning for Large Language Models
por: Xiao, Wenyang, et al.
Publicado: (2025)

OWL: Overcoming Window Length-Dependence in Speculative Decoding for Long-Context Inputs
por: Lee, Jaeseong, et al.
Publicado: (2025)

Linguistic Structure Induction from Language Models
por: Momen, Omar
Publicado: (2024)

Grammar Induction from Visual, Speech and Text
por: Zhao, Yu, et al.
Publicado: (2024)

Musical Phrase Segmentation via Grammatical Induction
por: Perkins, Reed, et al.
Publicado: (2024)