Imagen de Portada

Your Context Is Not an Array: Unveiling Random Access Limitations in Transformers

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Ebrahimi, MohammadReza, Panchal, Sunny, Memisevic, Roland
Formato:	Preprint
Publicado:	2024
Materias:	Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

On the "Induction Bias" in Sequence Models
por: Ebrahimi, M. Reza, et al.
Publicado: (2026)

Revisiting Bi-Linear State Transitions in Recurrent Neural Networks
por: Ebrahimi, M. Reza, et al.
Publicado: (2025)

Enhancing Hallucination Detection through Noise Injection
por: Liu, Litian, et al.
Publicado: (2025)

Multi-Draft Speculative Sampling: Canonical Decomposition and Theoretical Limits
por: Khisti, Ashish, et al.
Publicado: (2024)

Can Vision-Language Models Answer Face to Face Questions in the Real-World?
por: Pourreza, Reza, et al.
Publicado: (2025)

Look, Remember and Reason: Grounded reasoning in videos with language models
por: Bhattacharyya, Apratim, et al.
Publicado: (2023)

From Out-of-Distribution Detection to Hallucination Detection: A Geometric View
por: Liu, Litian, et al.
Publicado: (2026)

Equipping Transformer with Random-Access Reading for Long-Context Understanding
por: Yang, Chenghao, et al.
Publicado: (2024)

Delayed Attention Training Improves Length Generalization in Transformer--RNN Hybrids
por: Phan, Buu, et al.
Publicado: (2025)

Can Multi-Modal LLMs Provide Live Step-by-Step Task Guidance?
por: Bhattacharyya, Apratim, et al.
Publicado: (2025)

Evidence-Grounded Subspecialty Reasoning: Evaluating a Curated Clinical Intelligence Layer on the 2025 Endocrinology Board-Style Examination
por: Hosseinian, Amir, et al.
Publicado: (2026)

Context is Enough: Empirical Validation of $\textit{Sequentiality}$ on Essays
por: Sunny, Amal, et al.
Publicado: (2025)

Unveiling Simplicities of Attention: Adaptive Long-Context Head Identification
por: Donhauser, Konstantin, et al.
Publicado: (2025)

Advancements in Continuous Glucose Monitoring: Integrating Deep Learning and ECG Signal
por: Hosseinzadehketilateh, MohammadReza, et al.
Publicado: (2024)

How Persuasive is Your Context?
por: Nguyen, Tu, et al.
Publicado: (2025)

RULER: What's the Real Context Size of Your Long-Context Language Models?
por: Hsieh, Cheng-Ping, et al.
Publicado: (2024)

Bootstrap Your Own Context Length
por: Wang, Liang, et al.
Publicado: (2024)

Make Your LLM Fully Utilize the Context
por: An, Shengnan, et al.
Publicado: (2024)

Know Your Limits: A Survey of Abstention in Large Language Models
por: Wen, Bingbing, et al.
Publicado: (2024)

Mind Your Format: Towards Consistent Evaluation of In-Context Learning Improvements
por: Voronov, Anton, et al.
Publicado: (2024)

Hear Your Code Fail, Voice-Assisted Debugging for Python
por: Amiri, Sayed Mahbub Hasan, et al.
Publicado: (2025)

What to Say and When to Say it: Live Fitness Coaching as a Testbed for Situated Interaction
por: Panchal, Sunny, et al.
Publicado: (2024)

Context-Free Recognition with Transformers
por: Jerad, Selim, et al.
Publicado: (2026)

The Curse of CoT: On the Limitations of Chain-of-Thought in In-Context Learning
por: Zheng, Tianshi, et al.
Publicado: (2025)

Indic-TunedLens: Interpreting Multilingual Models in Indian Languages
por: Panchal, Mihir, et al.
Publicado: (2026)

Unveiling In-Context Learning: A Coordinate System to Understand Its Working Mechanism
por: Zhao, Anhao, et al.
Publicado: (2024)

Your Transformer is Secretly Linear
por: Razzhigaev, Anton, et al.
Publicado: (2024)

Focus Directions Make Your Language Models Pay More Attention to Relevant Contexts
por: Zhu, Youxiang, et al.
Publicado: (2025)

InfiniteICL: Breaking the Limit of Context Window Size via Long Short-term Memory Transformation
por: Cao, Bowen, et al.
Publicado: (2025)

Explainable Deep Learning for Cataract Detection in Retinal Images: A Dual-Eye and Knowledge Distillation Approach
por: Soflaei, MohammadReza Abbaszadeh Bavil, et al.
Publicado: (2025)

Aligning to What? Limits to RLHF Based Alignment
por: Barnhart, Logan, et al.
Publicado: (2025)

DrugRAG: Enhancing Pharmacy LLM Performance Through A Novel Retrieval-Augmented Generation Pipeline
por: Kazemzadeh, Houman, et al.
Publicado: (2025)

Unveiling Cultural Blind Spots: Analyzing the Limitations of mLLMs in Procedural Text Comprehension
por: Yari, Amir Hossein, et al.
Publicado: (2025)

Since the Scientific Literature Is Multilingual, Our Models Should Be Too
por: Ebrahimi, Abteen, et al.
Publicado: (2024)

Mechanisms of Symbol Processing for In-Context Learning in Transformer Networks
por: Smolensky, Paul, et al.
Publicado: (2024)

RepMatch: Quantifying Cross-Instance Similarities in Representation Space
por: Modarres, Mohammad Reza, et al.
Publicado: (2024)

NormXLogit: The Head-on-Top Never Lies
por: Abbasi, Sina, et al.
Publicado: (2024)

Large Language Models are Limited in Out-of-Context Knowledge Reasoning
por: Hu, Peng, et al.
Publicado: (2024)

Positional Biases Shift as Inputs Approach Context Window Limits
por: Veseli, Blerta, et al.
Publicado: (2025)

Beyond Context Limits: Subconscious Threads for Long-Horizon Reasoning
por: Luo, Hongyin, et al.
Publicado: (2025)