Cover Image

Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Hsieh, Cheng-Yu, Chuang, Yung-Sung, Li, Chun-Liang, Wang, Zifeng, Le, Long T., Kumar, Abhishek, Glass, James, Ratner, Alexander, Lee, Chen-Yu, Krishna, Ranjay, Pfister, Tomas
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps
by: Chuang, Yung-Sung, et al.
Published: (2024)

The Hard Positive Truth about Vision-Language Compositionality
by: Kamath, Amita, et al.
Published: (2024)

Don't "Overthink" Passage Reranking: Is Reasoning Truly Necessary?
by: Jedidi, Nour, et al.
Published: (2025)

Zero-Shot Dense Retrieval with Embeddings from Relevance Feedback
by: Jedidi, Nour, et al.
Published: (2024)

Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding
by: Zhang, Zhenyu, et al.
Published: (2024)

Learning to Attribute with Attention
by: Cohen-Wang, Benjamin, et al.
Published: (2025)

CodecLM: Aligning Language Models with Tailored Synthetic Data
by: Wang, Zifeng, et al.
Published: (2024)

Towards Acyclic Preference Evaluation of Language Models via Multiple Evaluators
by: Hu, Zhengyu, et al.
Published: (2024)

CaLM: Contrasting Large and Small Language Models to Verify Grounded Generation
by: Hsu, I-Hung, et al.
Published: (2024)

Is C4 Dataset Optimal for Pruning? An Investigation of Calibration Data for LLM Pruning
by: Bandari, Abhinav, et al.
Published: (2024)

A$^2$RD: Agentic Autoregressive Diffusion for Long Video Consistency
by: Long, Do Xuan, et al.
Published: (2026)

DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models
by: Chuang, Yung-Sung, et al.
Published: (2023)

Cleansing Jewel: A Neural Spelling Correction Model Built On Google OCR-ed Tibetan Manuscripts
by: Luo, Queenie, et al.
Published: (2023)

Design and Implementation of a Doppler Ultrasound Transducer Utilizing an Empirical Physical‐Based Model for Carotid Artery Stenosis
by: Wei‐Cheng Lin, et al.
Published: (2025)

Unraveling Dual Mechanisms in Quasi‐Layered Bi2O2Se via Defect Modulation for High‐Performance Aqueous Zn‐Ion Batteries
by: Yi‐Yen Hsieh, et al.
Published: (2024)

The Unmet Promise of Synthetic Training Images: Using Retrieved Real Images Performs Better
by: Geng, Scott, et al.
Published: (2024)

Logit-Attention Divergence: Mitigating Position Bias in Multi-Image Retrieval via Attention-Guided Calibration
by: Xian, Mingtao, et al.
Published: (2026)

An Intelligent Mobile Application to Monitor and Correct Sitting Posture Using Raspberry Pi and MediaPipe Pose Detection
by: Yung-Chen, et al.
Published: (2025)

An Efficient Recipe for Long Context Extension via Middle-Focused Positional Encoding
by: Wu, Tong, et al.
Published: (2024)

Chain of Agents: Large Language Models Collaborating on Long-Context Tasks
by: Zhang, Yusen, et al.
Published: (2024)

SPARO: Selective Attention for Robust and Compositional Transformer Encodings for Vision
by: Vani, Ankit, et al.
Published: (2024)

SelfCite: Self-Supervised Alignment for Context Attribution in Large Language Models
by: Chuang, Yung-Sung, et al.
Published: (2025)

Atlas: Multi-Scale Attention Improves Long Context Image Modeling
by: Agrawal, Kumar Krishna, et al.
Published: (2025)

Speculative Knowledge Distillation: Bridging the Teacher-Student Gap Through Interleaved Sampling
by: Xu, Wenda, et al.
Published: (2024)

Beyond Position Bias: Shifting Context Compression from Position-Driven to Semantic-Driven
by: Tang, Jiwei, et al.
Published: (2026)

Structured Packing in LLM Training Improves Long Context Utilization
by: Staniszewski, Konrad, et al.
Published: (2023)

FocalLens: Instruction Tuning Enables Zero-Shot Conditional Image Representations
by: Hsieh, Cheng-Yu, et al.
Published: (2025)

RubricEM: Meta-RL with Rubric-guided Policy Decomposition beyond Verifiable Rewards
by: Li, Gaotang, et al.
Published: (2026)

In Prospect and Retrospect: Reflective Memory Management for Long-term Personalized Dialogue Agents
by: Tan, Zhen, et al.
Published: (2025)

$π$-Attention: Periodic Sparse Transformers for Efficient Long-Context Modeling
by: Liu, Dong, et al.
Published: (2025)

Graph-Based Captioning: Enhancing Visual Descriptions by Interconnecting Region Captions
by: Hsieh, Yu-Guan, et al.
Published: (2024)

Never Lost in the Middle: Mastering Long-Context Question Answering with Position-Agnostic Decompositional Training
by: He, Junqing, et al.
Published: (2023)

Perception Tokens Enhance Visual Reasoning in Multimodal Language Models
by: Bigverdi, Mahtab, et al.
Published: (2024)

Self-Consistency Falls Short! The Adverse Effects of Positional Bias on Long-Context Problems
by: Byerly, Adam, et al.
Published: (2024)

PolicyBank: Evolving Policy Understanding for LLM Agents
by: Choi, Jihye, et al.
Published: (2026)

Lost in the Middle at Birth: An Exact Theory of Transformer Position Bias
by: Chowdhury, Borun D
Published: (2026)

Challenges in the Socioeconomic and Environmental Context: A Multidimensional Analysis of Delayed Childhood Development
by: Shih‐Yung Chiu, et al.
Published: (2026)

Template-free search for gravitational wave events using coincident anomaly detection
by: Ratner, Daniel
Published: (2026)

Phagocytosis and encapsulation : cellular immune responses in arthropoda / Stuart Ratner
by: Ratner, Stuart
Published: (1983)

National Museum of American History Branch Library.
by: Ratner, Rhoda S.
Published: (1986)