Cover Image

Induction Head Toxicity Mechanistically Explains Repetition Curse in Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Shuxun, Yin, Qingyu, Leong, Chak Tou, Zhang, Qiang, Yang, Linyi
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Why Safeguarded Ships Run Aground? Aligned Large Language Models' Safety Mechanisms Tend to Be Anchored in The Template Region
by: Leong, Chak Tou, et al.
Published: (2025)

Probing the Difficulty Perception Mechanism of Large Language Models
by: Lee, Sunbowen, et al.
Published: (2025)

Constrain Alignment with Sparse Autoencoders
by: Yin, Qingyu, et al.
Published: (2024)

Finding RELIEF: Shaping Reasoning Behavior without Reasoning Supervision via Belief Engineering
by: Leong, Chak Tou, et al.
Published: (2026)

E2CL: Exploration-based Error Correction Learning for Embodied Agents
by: Wang, Hanlin, et al.
Published: (2024)

Scaling over Scaling: Exploring Test-Time Scaling Plateau in Large Reasoning Models
by: Wang, Jian, et al.
Published: (2025)

Seeing Isn't Believing: Mitigating Belief Inertia via Active Intervention in Embodied Agents
by: Wang, Hanlin, et al.
Published: (2026)

STeCa: Step-level Trajectory Calibration for LLM Agent Learning
by: Wang, Hanlin, et al.
Published: (2025)

TokenSkip: Controllable Chain-of-Thought Compression in LLMs
by: Xia, Heming, et al.
Published: (2025)

Instruct Once, Chat Consistently in Multiple Rounds: An Efficient Tuning Framework for Dialogue
by: Wang, Jian, et al.
Published: (2024)

An Empirical Analysis of Uncertainty in Large Language Model Evaluations
by: Xie, Qiujie, et al.
Published: (2025)

Characterising Toxicity in Generative Large Language Models
by: Zhang, Zhiyao, et al.
Published: (2026)

A Survey on Evaluation of Large Language Models
by: Chang, Yupeng, et al.
Published: (2023)

Fairness Definitions in Language Models Explained
by: Yin, Zhipeng, et al.
Published: (2024)

Mitigating Reversal Curse in Large Language Models via Semantic-aware Permutation Training
by: Guo, Qingyan, et al.
Published: (2024)

Exploring the Reversal Curse and Other Deductive Logical Reasoning in BERT and GPT-Based Large Language Models
by: Wu, Da, et al.
Published: (2023)

On the Emergence of Induction Heads for In-Context Learning
by: Musat, Tiberiu, et al.
Published: (2025)

AutoPal: Autonomous Adaptation to Users for Personal AI Companionship
by: Cheng, Yi, et al.
Published: (2024)

Rethinking Toxicity Evaluation in Large Language Models: A Multi-Label Perspective
by: Kou, Zhiqiang, et al.
Published: (2025)

Large Language Models as Quasi-crystals: Coherence Without Repetition in Generative Text
by: Guevara-Vela, Jose Manuel
Published: (2025)

Mechanistic Indicators of Understanding in Large Language Models
by: Beckmann, Pierre, et al.
Published: (2025)

Realistic Evaluation of Toxicity in Large Language Models
by: Luong, Tinh Son, et al.
Published: (2024)

Sparsity Induction for Accurate Post-Training Pruning of Large Language Models
by: Jiang, Minhao, et al.
Published: (2026)

Identifying Semantic Induction Heads to Understand In-Context Learning
by: Ren, Jie, et al.
Published: (2024)

Benchmarking Next-Generation Reasoning-Focused Large Language Models in Ophthalmology: A Head-to-Head Evaluation on 5,888 Items
by: Zou, Minjie, et al.
Published: (2025)

Supervised Knowledge Makes Large Language Models Better In-context Learners
by: Yang, Linyi, et al.
Published: (2023)

Just Go Parallel: Improving the Multilingual Capabilities of Large Language Models
by: Qorib, Muhammad Reza, et al.
Published: (2025)

In-Context Learning in Speech Language Models: Analyzing the Role of Acoustic Features, Linguistic Structure, and Induction Heads
by: Pouw, Charlotte, et al.
Published: (2026)

Circumventing Safety Alignment in Large Language Models Through Embedding Space Toxicity Attenuation
by: Zhang, Zhibo, et al.
Published: (2025)

Where Does Toxicity Live? Mechanistic Localization and Targeted Suppression in Language Models
by: Beniwal, Himanshu, et al.
Published: (2026)

CulturePark: Boosting Cross-cultural Understanding in Large Language Models
by: Li, Cheng, et al.
Published: (2024)

Mechanistic Interpretability of Emotion Inference in Large Language Models
by: Tak, Ala N., et al.
Published: (2025)

Mechanistic Decoding of Cognitive Constructs in Large Language Models
by: Shou, Yitong, et al.
Published: (2026)

Efficient Fine-Tuning of Large Language Models for Automated Medical Documentation
by: Leong, Hui Yi, et al.
Published: (2024)

Subtle Errors in Reasoning: Preference Learning via Error-injected Self-editing
by: Xu, Kaishuai, et al.
Published: (2024)

K-ON: Stacking Knowledge On the Head Layer of Large Language Model
by: Guo, Lingbing, et al.
Published: (2025)

Reasoning Models Hallucinate More: Factuality-Aware Reinforcement Learning for Large Reasoning Models
by: Li, Junyi, et al.
Published: (2025)

Temporal Dependencies in In-Context Learning: The Role of Induction Heads
by: Bajaj, Anooshka, et al.
Published: (2026)

Explaining the Reasoning of Large Language Models Using Attribution Graphs
by: Walker, Chase, et al.
Published: (2025)

Parrot Mind: Towards Explaining the Complex Task Reasoning of Pretrained Large Language Models with Template-Content Structure
by: Yang, Haotong, et al.
Published: (2023)