Cover Image

Attention as Binding: A Vector-Symbolic Perspective on Transformer Reasoning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Dhayalkar, Sahil Rajesh
Format:	Preprint
Published:	2025
Subjects:	Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Reasoning Stabilization Point: A Training-Time Signal for Stable Evidence and Shortcut Reliance
by: Dhayalkar, Sahil Rajesh
Published: (2026)

Dynamic Context Adaptation and Information Flow Control in Transformers: Introducing the Evaluator Adjuster Unit and Gated Residual Connections
by: Dhayalkar, Sahil Rajesh
Published: (2024)

A Combinatorial Theory of Dropout: Subnetworks, Graph Geometry, and Generalization
by: Dhayalkar, Sahil Rajesh
Published: (2025)

The Geometry of ReLU Networks through the ReLU Transition Graph
by: Dhayalkar, Sahil Rajesh
Published: (2025)

Neural Networks as Universal Finite-State Machines: A Constructive Deterministic Finite Automaton Theory
by: Dhayalkar, Sahil Rajesh
Published: (2025)

Symbolic Feedforward Networks for Probabilistic Finite Automata: Exact Simulation and Learnability
by: Dhayalkar, Sahil Rajesh
Published: (2025)

Discrete Functional Geometry of ReLU Networks via ReLU Transition Graphs
by: Dhayalkar, Sahil Rajesh
Published: (2025)

Logic-Gated Time-Shared Feedforward Networks for Alternating Finite Automata: Exact Simulation and Learnability
by: Dhayalkar, Sahil Rajesh
Published: (2026)

A Constructive Framework for Nondeterministic Automata via Time-Shared, Depth-Unrolled Feedforward Networks
by: Dhayalkar, Sahil Rajesh
Published: (2025)

Human Activity Recognition using Smartphones
by: Sonawane, Mayur, et al.
Published: (2024)

Vector Symbolic Algebras for the Abstraction and Reasoning Corpus
by: Joffe, Isaac, et al.
Published: (2025)

RESOLVE: Relational Reasoning with Symbolic and Object-Level Features Using Vector Symbolic Processing
by: Mejri, Mohamed, et al.
Published: (2024)

How Do Transformers Learn Variable Binding in Symbolic Programs?
by: Wu, Yiwei, et al.
Published: (2025)

Vectorized Attention with Learnable Encoding for Quantum Transformer
by: Guo, Ziqing, et al.
Published: (2025)

A Walsh Hadamard Derived Linear Vector Symbolic Architecture
by: Alam, Mohammad Mahmudul, et al.
Published: (2024)

GPT-2 Through the Lens of Vector Symbolic Architectures
by: Knittel, Johannes, et al.
Published: (2024)

LARS-VSA: A Vector Symbolic Architecture For Learning with Abstract Rules
by: Mejri, Mohamed, et al.
Published: (2024)

Symbol Grounding in Neuro-Symbolic AI: A Gentle Introduction to Reasoning Shortcuts
by: Marconato, Emanuele, et al.
Published: (2025)

RACE Attention: A Strictly Linear-Time Attention Layer for Training on Outrageously Large Contexts
by: Joshi, Sahil, et al.
Published: (2025)

Fusing Memory and Attention: A study on LSTM, Transformer and Hybrid Architectures for Symbolic Music Generation
by: Ghoshal, Soudeep, et al.
Published: (2026)

Symbol-Equivariant Recurrent Reasoning Models
by: Freinschlag, Richard, et al.
Published: (2026)

Singular Vectors of Attention Heads Align with Features
by: Franco, Gabriel, et al.
Published: (2026)

Scheduled Knowledge Acquisition on Lightweight Vector Symbolic Architectures for Brain-Computer Interfaces
by: Liu, Yejia, et al.
Published: (2024)

Disentangling Recall and Reasoning in Transformer Models through Layer-wise Attention and Activation Analysis
by: Fartale, Harshwardhan, et al.
Published: (2025)

The Role of Foundation Models in Neuro-Symbolic Learning and Reasoning
by: Cunnington, Daniel, et al.
Published: (2024)

A Neuro-Symbolic Benchmark Suite for Concept Quality and Reasoning Shortcuts
by: Bortolotti, Samuele, et al.
Published: (2024)

Curse of Attention: A Kernel-Based Perspective for Why Transformers Fail to Generalize on Time Series Forecasting and Beyond
by: Ke, Yekun, et al.
Published: (2024)

BEARS Make Neuro-Symbolic Models Aware of their Reasoning Shortcuts
by: Marconato, Emanuele, et al.
Published: (2024)

Efficient Rectification of Neuro-Symbolic Reasoning Inconsistencies by Abductive Reflection
by: Hu, Wen-Chao, et al.
Published: (2024)

Building Expressive and Tractable Probabilistic Generative Models: A Review
by: Sidheekh, Sahil, et al.
Published: (2024)

Scout Before You Attend: Sketch-and-Walk Sparse Attention for Efficient LLM Inference
by: Le, Hoang Anh Duy, et al.
Published: (2026)

Geometric Attention: A Regime-Explicit Operator Semantics for Transformer Attention
by: Freytes, Luis Rosario
Published: (2026)

Understanding the Language Model to Solve the Symbolic Multi-Step Reasoning Problem from the Perspective of Buffer Mechanism
by: Wang, Zhiwei, et al.
Published: (2024)

SeerAttention-R: Sparse Attention Adaptation for Long Reasoning
by: Gao, Yizhao, et al.
Published: (2025)

Comprehension Without Competence: Architectural Limits of LLMs in Symbolic Computation and Reasoning
by: Zhang, Zheng
Published: (2025)

Advanced Weakly-Supervised Formula Exploration for Neuro-Symbolic Mathematical Reasoning
by: Wu, Yuxuan, et al.
Published: (2025)

GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models
by: Mirzadeh, Iman, et al.
Published: (2024)

The KANDY Benchmark: Incremental Neuro-Symbolic Learning and Reasoning with Kandinsky Patterns
by: Lorello, Luca Salvatore, et al.
Published: (2024)

Decoupling Knowledge and Reasoning in Transformers: A Modular Architecture with Generalized Cross-Attention
by: Guo, Zhenyu, et al.
Published: (2025)

The Bayesian Geometry of Transformer Attention
by: Agarwal, Naman, et al.
Published: (2025)