Cover Image

Position: The Turing-Completeness of Autoregressive Transformers Relies Heavily on Context Management

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Cui, Guanyu, Wei, Zhewei, He, Kun
Format:	Preprint
Published:	2026
Subjects:	Artificial Intelligence Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

SwiftKV: Fast Prefill-Optimized Inference with Knowledge-Preserving Model Transformation
by: Qiao, Aurick, et al.
Published: (2024)

Transformer-Based Multimodal Knowledge Graph Completion with Link-Aware Contexts
by: Ma, Haodi, et al.
Published: (2025)

Context Dependence and Reliability in Autoregressive Language Models
by: Sengupta, Poushali, et al.
Published: (2026)

How do Language Models Generate Slang: A Systematic Comparison between Human and Machine-Generated Slang Usages
by: Wu, Siyang, et al.
Published: (2025)

CoCA: Fusing Position Embedding with Collinear Constrained Attention in Transformers for Long Context Window Extending
by: Zhu, Shiyi, et al.
Published: (2023)

Neural Paging: Learning Context Management Policies for Turing-Complete Agents
by: Chen, Liang, et al.
Published: (2026)

Scalable Best-of-N Selection for Large Language Models via Self-Certainty
by: Kang, Zhewei, et al.
Published: (2025)

Rotary Positional Embeddings as Phase Modulation: Theoretical Bounds on the RoPE Base for Long-Context Transformers
by: Liu, Feilong
Published: (2026)

Efficient Relational Context Perception for Knowledge Graph Completion
by: Tu, Wenkai, et al.
Published: (2024)

Learning to Hint for Reinforcement Learning
by: Xia, Yu, et al.
Published: (2026)

The Generalized Turing Test: A Foundation for Comparing Intelligence
by: Mitropolsky, Daniel, et al.
Published: (2026)

How Hard Is It for Message-Passing GNNs to Simulate One Weisfeiler-Lehman Color-Refinement Step?
by: Cui, Guanyu, et al.
Published: (2024)

RePo: Language Models with Context Re-Positioning
by: Li, Huayang, et al.
Published: (2025)

Projected Autoregression: Autoregressive Language Generation in Continuous State Space
by: Naparstek, Oshri
Published: (2026)

SeqPE: Transformer with Sequential Position Encoding
by: Li, Huayang, et al.
Published: (2025)

Hierarchical Autoregressive Transformers: Combining Byte- and Word-Level Processing for Robust, Adaptable Language Models
by: Neitemeier, Pit, et al.
Published: (2025)

Sensitivity-Positional Co-Localization in GQA Transformers
by: Rao, Manoj Chandrashekar
Published: (2026)

HoPE: A Novel Positional Encoding Without Long-Term Decay for Enhanced Context Awareness and Extrapolation
by: Chen, Yuhan, et al.
Published: (2024)

Advancing Sequential Numerical Prediction in Autoregressive Models
by: Fei, Xiang, et al.
Published: (2025)

A Pseudo-Semantic Loss for Autoregressive Models with Logical Constraints
by: Ahmed, Kareem, et al.
Published: (2023)

SampleAttention: Near-Lossless Acceleration of Long Context LLM Inference with Adaptive Structured Sparse Attention
by: Zhu, Qianchao, et al.
Published: (2024)

ALPINE: Unveiling the Planning Capability of Autoregressive Learning in Language Models
by: Wang, Siwei, et al.
Published: (2024)

RoPE Distinguishes Neither Positions Nor Tokens in Long Contexts, Provably
by: Du, Yufeng, et al.
Published: (2026)

Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization
by: Hsieh, Cheng-Yu, et al.
Published: (2024)

Universal Approximation of Visual Autoregressive Transformers
by: Chen, Yifang, et al.
Published: (2025)

Continuous Autoregressive Language Models
by: Shao, Chenze, et al.
Published: (2025)

Lost in the Middle at Birth: An Exact Theory of Transformer Position Bias
by: Chowdhury, Borun D
Published: (2026)

ReFusion: A Diffusion Large Language Model with Parallel Autoregressive Decoding
by: Li, Jia-Nan, et al.
Published: (2025)

On the Robustness of Transformers against Context Hijacking for Linear Classification
by: Li, Tianle, et al.
Published: (2025)

Do pretrained Transformers Learn In-Context by Gradient Descent?
by: Shen, Lingfeng, et al.
Published: (2023)

Diagnosing Retrieval Bias Under Multiple In-Context Knowledge Updates in Large Language Models
by: Qiao, Boyu, et al.
Published: (2026)

Alice in Wonderland: Simple Tasks Showing Complete Reasoning Breakdown in State-Of-the-Art Large Language Models
by: Nezhurina, Marianna, et al.
Published: (2024)

Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learning
by: Wang, Zhaoyang, et al.
Published: (2026)

Planner and Executor: Collaboration between Discrete Diffusion And Autoregressive Models in Reasoning
by: Berrayana, Lina, et al.
Published: (2025)

Positional Failures in Long-Context LLMs: A Blind Spot in Reasoning Benchmarks
by: Zhang, Chuyifei, et al.
Published: (2026)

Improving Autoregressive Training with Dynamic Oracles
by: Yang, Jianing, et al.
Published: (2024)

In-Context Learning with Transformers: Softmax Attention Adapts to Function Lipschitzness
by: Collins, Liam, et al.
Published: (2024)

Position Engineering: Boosting Large Language Models through Positional Information Manipulation
by: He, Zhiyuan, et al.
Published: (2024)

AgentFold: Long-Horizon Web Agents with Proactive Context Management
by: Ye, Rui, et al.
Published: (2025)

Sculptor: Empowering LLMs with Cognitive Agency via Active Context Management
by: Li, Mo, et al.
Published: (2025)