Image de couverture de livre

Single-Position Intervention Fails: Distributed Output Templates Drive In-Context Learning

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Cheng, Bryan, Zhang, Jasper
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

TemplateRL: Structured Template-Guided Reinforcement Learning for LLM Reasoning
par: Wu, Jinyang, et autres
Publié: (2025)

Comparing Template-based and Template-free Language Model Probing
par: Shaier, Sagi, et autres
Publié: (2024)

When In-Distribution Gains Fail: Evaluating Weak-to-Strong Reward Models under Preference Shift
par: Le, Khoi, et autres
Publié: (2026)

LLM Library Learning Fails: A LEGO-Prover Case Study
par: Berlot-Attwell, Ian, et autres
Publié: (2025)

When Refusals Fail: Unstable Safety Mechanisms in Long-Context LLM Agents
par: Hadeliya, Tsimur, et autres
Publié: (2025)

Faithful Bi-Directional Model Steering via Distribution Matching and Distributed Interchange Interventions
par: Bao, Yuntai, et autres
Publié: (2026)

Short Data, Long Context: Distilling Positional Knowledge in Transformers
par: Huber, Patrick, et autres
Publié: (2026)

Extracting Prompts by Inverting LLM Outputs
par: Zhang, Collin, et autres
Publié: (2024)

Copy-Augmented Representation for Structure Invariant Template-Free Retrosynthesis
par: Zhuang, Jiaxi, et autres
Publié: (2025)

Why Semantic Entropy Fails: Geometry-Aware and Calibrated Uncertainty for Policy Optimization
par: Zhang, Zheyuan, et autres
Publié: (2026)

DeMeVa at LeWiDi-2025: Modeling Perspectives with In-Context Learning and Label Distribution Learning
par: Ignatev, Daniil, et autres
Publié: (2025)

Uncertainty Quantification for In-Context Learning of Large Language Models
par: Ling, Chen, et autres
Publié: (2024)

Exploring Context Window of Large Language Models via Decomposed Positional Vectors
par: Dong, Zican, et autres
Publié: (2024)

Failing to Falsify: Evaluating and Mitigating Confirmation Bias in Language Models
par: Jhaveri, Ayush Rajesh, et autres
Publié: (2026)

The Illusion of Certainty: Uncertainty Quantification for LLMs Fails under Ambiguity
par: Tomov, Tim, et autres
Publié: (2025)

Current Agents Fail to Leverage World Model as Tool for Foresight
par: Qian, Cheng, et autres
Publié: (2026)

Theoretical Understanding of In-Context Learning in Shallow Transformers with Unstructured Data
par: Xing, Yue, et autres
Publié: (2024)

APB: Accelerating Distributed Long-Context Inference by Passing Compressed Context Blocks across GPUs
par: Huang, Yuxiang, et autres
Publié: (2025)

Dimensional Collapse in Transformer Attention Outputs: A Challenge for Sparse Dictionary Learning
par: Wang, Junxuan, et autres
Publié: (2025)

Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding
par: Zhang, Zhenyu, et autres
Publié: (2024)

Dynamic Attention-Guided Context Decoding for Mitigating Context Faithfulness Hallucinations in Large Language Models
par: Huang, Yanwen, et autres
Publié: (2025)

AutoPyVerifier: Learning Compact Executable Verifiers for Large Language Model Outputs
par: Pezeshkpour, Pouya, et autres
Publié: (2026)

Calibrate to Discriminate: Improve In-Context Learning with Label-Free Comparative Inference
par: Cheng, Wei, et autres
Publié: (2024)

Improving Diversity in Language Models: When Temperature Fails, Change the Loss
par: Verine, Alexandre, et autres
Publié: (2025)

Brewing Knowledge in Context: Distillation Perspectives on In-Context Learning
par: Li, Chengye, et autres
Publié: (2025)

The Extrapolation Cliff in On-Policy Distillation of Near-Deterministic Structured Outputs
par: Li, Xin, et autres
Publié: (2026)

Exploring Imbalanced Annotations for Effective In-Context Learning
par: Gao, Hongfu, et autres
Publié: (2025)

On the Noise Robustness of In-Context Learning for Text Generation
par: Gao, Hongfu, et autres
Publié: (2024)

PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Training
par: Zhu, Dawei, et autres
Publié: (2023)

Dense Retrievers Can Fail on Simple Queries: Revealing The Granularity Dilemma of Embeddings
par: Xu, Liyan, et autres
Publié: (2025)

RAVEN: In-Context Learning with Retrieval-Augmented Encoder-Decoder Language Models
par: Huang, Jie, et autres
Publié: (2023)

Mitigate Position Bias in Large Language Models via Scaling a Single Dimension
par: Yu, Yijiong, et autres
Publié: (2024)

Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization
par: Hsieh, Cheng-Yu, et autres
Publié: (2024)

Crafting a Good Prompt or Providing Exemplary Dialogues? A Study of In-Context Learning for Persona-based Dialogue Generation
par: Pu, Jiashu, et autres
Publié: (2024)

AI Scientists Fail Without Strong Implementation Capability
par: Zhu, Minjun, et autres
Publié: (2025)

Detection Is Cheap, Routing Is Learned: Why Refusal-Based Alignment Evaluation Fails
par: Frank, Gregory N.
Publié: (2026)

Training and Evaluating Language Models with Template-based Data Generation
par: Zhang, Yifan
Publié: (2024)

RePo: Language Models with Context Re-Positioning
par: Li, Huayang, et autres
Publié: (2025)

MMUTF: Multimodal Multimedia Event Argument Extraction with Unified Template Filling
par: Seeberger, Philipp, et autres
Publié: (2024)

Evaluating the Robustness of Adverse Drug Event Classification Models Using Templates
par: MacPhail, Dorothea, et autres
Publié: (2024)