Cover Image

Provably Learning Attention with Queries

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Bhattamishra, Satwik, Shah, Kulin, Hahn, Michael, Kanade, Varun
Format:	Preprint
Published:	2026
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Separations in the Representational Capabilities of Transformers and Recurrent Architectures
by: Bhattamishra, Satwik, et al.
Published: (2024)

Hardness of Learning Regular Languages in the Next Symbol Prediction Setting
by: Bhattamishra, Satwik, et al.
Published: (2025)

Discovering Interpretable Algorithms by Decompiling Transformers to RASP
by: Huang, Xinting, et al.
Published: (2026)

Benefits and Limitations of Communication in Multi-Agent Reasoning
by: Rizvi-Martel, Michael, et al.
Published: (2025)

A Formal Framework for Understanding Length Generalization in Transformers
by: Huang, Xinting, et al.
Published: (2024)

Pause Tokens Strictly Increase the Expressivity of Constant-Depth Transformers
by: London, Charles, et al.
Published: (2025)

Inshrinkerator: Compressing Deep Learning Training Checkpoints via Dynamic Quantization
by: Agrawal, Amey, et al.
Published: (2023)

Learning general Gaussian mixtures with efficient score matching
by: Chen, Sitan, et al.
Published: (2024)

Masked Diffusion for Generative Recommendation
by: Shah, Kulin, et al.
Published: (2025)

Gating Enables Curvature: A Geometric Expressivity Gap in Attention
by: Bathula, Satwik, et al.
Published: (2026)

How Global Calibration Strengthens Multiaccuracy
by: Casacuberta, Sílvia, et al.
Published: (2025)

ReGuidance: A Simple Diffusion Wrapper for Boosting Sample Quality on Hard Inverse Problems
by: Karan, Aayush, et al.
Published: (2025)

Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion
by: Shah, Kulin, et al.
Published: (2025)

Causal Language Modeling Can Elicit Search and Reasoning Capabilities on Logic Puzzles
by: Shah, Kulin, et al.
Published: (2024)

Simple Mechanisms for Representing, Indexing and Manipulating Concepts
by: Li, Yuanzhi, et al.
Published: (2023)

The Transformer Cookbook
by: Yang, Andy, et al.
Published: (2025)

Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions
by: Kim, Jaeyeon, et al.
Published: (2025)

DEEDEE: Fast and Scalable Out-of-Distribution Dynamics Detection
by: Aljaafari, Tala, et al.
Published: (2025)

Unrolled denoising networks provably learn optimal Bayesian inference
by: Karan, Aayush, et al.
Published: (2024)

Continual Learning with Query-Only Attention
by: Bekal, Gautham, et al.
Published: (2025)

Verifiable and Provably Secure Machine Unlearning
by: Eisenhofer, Thorsten, et al.
Published: (2022)

Security Concerns in Quantum Machine Learning as a Service
by: Kundu, Satwik, et al.
Published: (2024)

Explainable Machine Learning for Pediatric Dental Risk Stratification Using Socio-Demographic Determinants
by: Kanade, Manasi, et al.
Published: (2026)

Minimalist Softmax Attention Provably Learns Constrained Boolean Functions
by: Hu, Jerry Yao-Chieh, et al.
Published: (2025)

STIQ: Safeguarding Training and Inferencing of Quantum Neural Networks from Untrusted Cloud
by: Kundu, Satwik, et al.
Published: (2024)

Inverse-Transpilation: Reverse-Engineering Quantum Compiler Optimization Passes from Circuit Snapshots
by: Kundu, Satwik, et al.
Published: (2025)

Provably Shorter Scratchpads in Hybrid DeltaNet-Attention Decoders
by: Steifer, Tomasz
Published: (2026)

Adversarial Threats in Quantum Machine Learning: A Survey of Attacks and Defenses
by: Ghosh, Archisman, et al.
Published: (2025)

Not All Queries Need Rewriting: When Prompt-Only LLM Refinement Helps and Hurts Dense Retrieval
by: Kotte, Varun
Published: (2026)

KQ-SVD: Compressing the KV Cache with Provable Guarantees on Attention Fidelity
by: Lesens, Damien, et al.
Published: (2025)

Attention with Trained Embeddings Provably Selects Important Tokens
by: Wu, Diyuan, et al.
Published: (2025)

AttackGNN: Red-Teaming GNNs in Hardware Security Using Reinforcement Learning
by: Gohil, Vasudev, et al.
Published: (2024)

Robust Learning of Diverse Code Edits
by: Aggarwal, Tushar, et al.
Published: (2025)

SpectraLDS: Provable Distillation for Linear Dynamical Systems
by: Shah, Devan, et al.
Published: (2025)

Optimised Grouped-Query Attention Mechanism for Transformers
by: Chen, Yuang, et al.
Published: (2024)

Sparse Query Attention (SQA): A Computationally Efficient Attention Mechanism with Query Heads Reduction
by: Filipek, Adam
Published: (2025)

Recoverability Has a Law: The ERR Measure for Tool-Augmented Agents
by: Vuddanti, Sri Vatsa, et al.
Published: (2026)

DyPP: Dynamic Parameter Prediction to Accelerate Convergence of Variational Quantum Algorithms
by: Kundu, Satwik, et al.
Published: (2023)

On Provable Benefits of Muon in Federated Learning
by: Zhang, Xinwen, et al.
Published: (2025)

Provable Training for Graph Contrastive Learning
by: Yu, Yue, et al.
Published: (2023)