Cover Image

Equipping Transformer with Random-Access Reading for Long-Context Understanding

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yang, Chenghao, Yang, Zi, Hua, Nan
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Attendre: Wait To Attend By Retrieval With Evicted Queries in Memory-Based Transformers for Long Context Processing
by: Yang, Zi, et al.
Published: (2024)

Retrieval Or Holistic Understanding? Dolce: Differentiate Our Long Context Evaluation Tasks
by: Yang, Zi
Published: (2024)

Your Context Is Not an Array: Unveiling Random Access Limitations in Transformers
by: Ebrahimi, MohammadReza, et al.
Published: (2024)

ETT: Expanding the Long Context Understanding Capability of LLMs at Test-Time
by: Zahirnia, Kiarash, et al.
Published: (2025)

ReMamba: Equip Mamba with Effective Long-Sequence Modeling
by: Yuan, Danlong, et al.
Published: (2024)

Equipping LLM with Directional Multi-Talker Speech Understanding Capabilities
by: Lin, Ju, et al.
Published: (2026)

Quantization-Aware and Tensor-Compressed Training of Transformers for Natural Language Understanding
by: Yang, Zi, et al.
Published: (2023)

From Text to Pixel: Advancing Long-Context Understanding in MLLMs
by: Lu, Yujie, et al.
Published: (2024)

LongEmbed: Extending Embedding Models for Long Context Retrieval
by: Zhu, Dawei, et al.
Published: (2024)

LIFT: A Novel Framework for Enhancing Long-Context Understanding of LLMs via Long Input Fine-Tuning
by: Mao, Yansheng, et al.
Published: (2025)

LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding
by: Bai, Yushi, et al.
Published: (2023)

LooGLE: Can Long-Context Language Models Understand Long Contexts?
by: Li, Jiaqi, et al.
Published: (2023)

Long-Context Encoder Models for Polish Language Understanding
by: Dadas, Sławomir, et al.
Published: (2026)

RepoQA: Evaluating Long Context Code Understanding
by: Liu, Jiawei, et al.
Published: (2024)

Beyond Isolated Capabilities: Bridging Long CoT Reasoning and Long-Context Understanding
by: Wang, Yifei
Published: (2025)

Latent-Condensed Transformer for Efficient Long Context Modeling
by: You, Zeng, et al.
Published: (2026)

Dynamic Chunking and Selection for Reading Comprehension of Ultra-Long Context in Large Language Models
by: Sheng, Boheng, et al.
Published: (2025)

Self-Taught Agentic Long Context Understanding
by: Zhuang, Yufan, et al.
Published: (2025)

Hello Again! LLM-powered Personalized Agent for Long-term Dialogue
by: Li, Hao, et al.
Published: (2024)

Membership Inference Attack against Long-Context Large Language Models
by: Wang, Zixiong, et al.
Published: (2024)

Core Context Aware Transformers for Long Context Language Modeling
by: Chen, Yaofo, et al.
Published: (2024)

FltLM: An Intergrated Long-Context Large Language Model for Effective Context Filtering and Understanding
by: Deng, Jingyang, et al.
Published: (2024)

LongReason: A Synthetic Long-Context Reasoning Benchmark via Context Expansion
by: Ling, Zhan, et al.
Published: (2025)

PaceLLM: Brain-Inspired Large Language Models for Long-Context Understanding
by: Li, Kangcong, et al.
Published: (2025)

Mixture of In-Context Experts Enhance LLMs' Long Context Awareness
by: Lin, Hongzhan, et al.
Published: (2024)

XL$^2$Bench: A Benchmark for Extremely Long Context Understanding with Long-range Dependencies
by: Ni, Xuanfan, et al.
Published: (2024)

MiniLongBench: The Low-cost Long Context Understanding Benchmark for Large Language Models
by: Huang, Zhongzhan, et al.
Published: (2025)

Understanding the RoPE Extensions of Long-Context LLMs: An Attention Perspective
by: Zhong, Meizhi, et al.
Published: (2024)

MiA-Signature: Approximating Global Activation for Long-Context Understanding
by: Li, Yuqing, et al.
Published: (2026)

Facilitating Long Context Understanding via Supervised Chain-of-Thought Reasoning
by: Lin, Jingyang, et al.
Published: (2025)

Mindscape-Aware Retrieval Augmented Generation for Improved Long Context Understanding
by: Li, Yuqing, et al.
Published: (2025)

ScaleFormer: Span Representation Cumulation for Long-Context Transformer
by: Du, Jiangshu, et al.
Published: (2025)

Learning When to Attend: Conditional Memory Access for Long-Context LLMs
by: Choudhary, Sakshi, et al.
Published: (2026)

OCR-Memory: Optical Context Retrieval for Long-Horizon Agent Memory
by: Li, Jinze, et al.
Published: (2026)

Context as a Tool: Context Management for Long-Horizon SWE-Agents
by: Liu, Shukai, et al.
Published: (2025)

Long Context is Not Long at All: A Prospector of Long-Dependency Data for Large Language Models
by: Chen, Longze, et al.
Published: (2024)

When Hindsight is Not 20/20: Testing Limits on Reflective Thinking in Large Language Models
by: Li, Yanhong, et al.
Published: (2024)

LIFT: Improving Long Context Understanding Through Long Input Fine-Tuning
by: Mao, Yansheng, et al.
Published: (2024)

Generalization or Hallucination? Understanding Out-of-Context Reasoning in Transformers
by: Huang, Yixiao, et al.
Published: (2025)

Efficient Prompt Compression with Evaluator Heads for Long-Context Transformer Inference
by: Fei, Weizhi, et al.
Published: (2025)