Copertina

On the Emergence of Induction Heads for In-Context Learning

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Musat, Tiberiu, Pimentel, Tiago, Noci, Lorenzo, Stolfo, Alessandro, Sachan, Mrinmaya, Hofmann, Thomas
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Artificial Intelligence Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Probing for Arithmetic Errors in Language Models
di: Sun, Yucheng, et al.
Pubblicazione: (2025)

Mechanism and Emergence of Stacked Attention Heads in Multi-Layer Transformers
di: Musat, Tiberiu
Pubblicazione: (2024)

Dense SAE Latents Are Features, Not Bugs
di: Sun, Xiaoqing, et al.
Pubblicazione: (2025)

Confidence Regulation Neurons in Language Models
di: Stolfo, Alessandro, et al.
Pubblicazione: (2024)

Do Language Models Exhibit the Same Cognitive Biases in Problem Solving as Human Learners?
di: Opedal, Andreas, et al.
Pubblicazione: (2024)

Beyond Induction Heads: In-Context Meta Learning Induces Multi-Phase Circuit Emergence
di: Minegishi, Gouki, et al.
Pubblicazione: (2025)

Identifying Semantic Induction Heads to Understand In-Context Learning
di: Ren, Jie, et al.
Pubblicazione: (2024)

Temporal Dependencies in In-Context Learning: The Role of Induction Heads
di: Bajaj, Anooshka, et al.
Pubblicazione: (2026)

Fluid Representations in Reasoning Models
di: Kharlapenko, Dmitrii, et al.
Pubblicazione: (2026)

How to Engage Your Readers? Generating Guiding Questions to Promote Active Reading
di: Cui, Peng, et al.
Pubblicazione: (2024)

Improving Large Language Model Safety with Contrastive Representation Learning
di: Simko, Samuel, et al.
Pubblicazione: (2025)

Simulating Students or Sycophantic Problem Solving? On Misconception Faithfulness of LLM Simulators
di: Do, Heejin, et al.
Pubblicazione: (2026)

A Head to Predict and a Head to Question: Pre-trained Uncertainty Quantification Heads for Hallucination Detection in LLM Outputs
di: Shelmanov, Artem, et al.
Pubblicazione: (2025)

Chimera: Diagnosing Shortcut Learning in Visual-Language Understanding
di: Chi, Ziheng, et al.
Pubblicazione: (2025)

The Geometry of Grokking: Norm Minimization on the Zero-Loss Manifold
di: Musat, Tiberiu
Pubblicazione: (2025)

Compose and Fuse: Revisiting the Foundational Bottlenecks in Multimodal Reasoning
di: Wang, Yucheng, et al.
Pubblicazione: (2025)

From Problem-Solving to Teaching Problem-Solving: Aligning LLMs with Pedagogy using Reinforcement Learning
di: Dinucu-Jianu, David, et al.
Pubblicazione: (2025)

Investigating the Zone of Proximal Development of Language Models for In-Context Learning
di: Cui, Peng, et al.
Pubblicazione: (2025)

Multilingual Performance Biases of Large Language Models in Education
di: Gupta, Vansh, et al.
Pubblicazione: (2025)

Generating Pedagogically Meaningful Visuals for Math Word Problems: A New Benchmark and Analysis of Text-to-Image Models
di: Wang, Junling, et al.
Pubblicazione: (2025)

In-Context Learning in Speech Language Models: Analyzing the Role of Acoustic Features, Linguistic Structure, and Induction Heads
di: Pouw, Charlotte, et al.
Pubblicazione: (2026)

Towards Aligning Language Models with Textual Feedback
di: Lloret, Saüc Abadal, et al.
Pubblicazione: (2024)

AFaCTA: Assisting the Annotation of Factual Claim Detection with Reliable LLM Annotators
di: Ni, Jingwei, et al.
Pubblicazione: (2024)

Post-Training Language Models for Crosslingual Consistency
di: Liu, Tianyu, et al.
Pubblicazione: (2026)

Can Vision-Language Models Solve Visual Math Equations?
di: Choudhury, Monjoy Narayan, et al.
Pubblicazione: (2025)

Stepwise Verification and Remediation of Student Reasoning Errors with Large Language Model Tutors
di: Daheim, Nico, et al.
Pubblicazione: (2024)

Retro-li: Small-Scale Retrieval Augmented Generation Supporting Noisy Similarity Searches and Domain Shift Generalization
di: Rashiti, Gentiana, et al.
Pubblicazione: (2024)

MathGAP: Out-of-Distribution Evaluation on Problems with Arbitrarily Complex Proofs
di: Opedal, Andreas, et al.
Pubblicazione: (2024)

Learning to Reason Efficiently with A* Post-Training
di: Opedal, Andreas, et al.
Pubblicazione: (2026)

Corrupted by Reasoning: Reasoning Language Models Become Free-Riders in Public Goods Games
di: Piedrahita, David Guzman, et al.
Pubblicazione: (2025)

PATS: Personality-Aware Teaching Strategies with Large Language Model Tutors
di: Rooein, Donya, et al.
Pubblicazione: (2026)

Learning to Learn from Language Feedback with Social Meta-Learning
di: Cook, Jonathan, et al.
Pubblicazione: (2026)

MathTutorBench: A Benchmark for Measuring Open-ended Pedagogical Capabilities of LLM Tutors
di: Macina, Jakub, et al.
Pubblicazione: (2025)

Pointwise Mutual Information as a Performance Gauge for Retrieval-Augmented Generation
di: Liu, Tianyu, et al.
Pubblicazione: (2024)

DIRAS: Efficient LLM Annotation of Document Relevance in Retrieval Augmented Generation
di: Ni, Jingwei, et al.
Pubblicazione: (2024)

Improving Instruction-Following in Language Models through Activation Steering
di: Stolfo, Alessandro, et al.
Pubblicazione: (2024)

Can LLMs Model Incorrect Student Reasoning? A Case Study on Distractor Generation
di: Zengaffinen, Yanick, et al.
Pubblicazione: (2026)

Exploring the Jungle of Bias: Political Bias Attribution in Language Models via Dependency Analysis
di: Jenny, David F., et al.
Pubblicazione: (2023)

Universal Response and Emergence of Induction in LLMs
di: Luick, Niclas
Pubblicazione: (2024)

Balancing Truthfulness and Informativeness with Uncertainty-Aware Instruction Fine-Tuning
di: Wu, Tianyi, et al.
Pubblicazione: (2025)