Image de couverture de livre

Learning to Recall with Transformers Beyond Orthogonal Embeddings

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Vural, Nuri Mert, Bietti, Alberto, Soltanolkotabi, Mahdi, Wu, Denny
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Learning quadratic neural networks in high dimensions: SGD dynamics and scaling laws
par: Arous, Gérard Ben, et autres
Publié: (2025)

Pruning is Optimal for Learning Sparse Features in High-Dimensions
par: Vural, Nuri Mert, et autres
Publié: (2024)

Understanding Factual Recall in Transformers via Associative Memories
par: Nichani, Eshaan, et autres
Publié: (2024)

Full-Batch Gradient Descent Outperforms One-Pass SGD: Sample Complexity Separation in Single-Index Learning
par: Kovačević, Filip, et autres
Publié: (2026)

Understanding the Mechanisms of Fast Hyperparameter Transfer
par: Ghosh, Nikhil, et autres
Publié: (2025)

Learning Compositional Functions with Transformers from Easy-to-Hard Data
par: Wang, Zixuan, et autres
Publié: (2025)

Understanding Contextual Recall in Transformers: How Finetuning Enables In-Context Reasoning over Pretraining Knowledge
par: Vasudeva, Bhavya, et autres
Publié: (2026)

Sharp Capacity Scaling of Spectral Optimizers in Learning Associative Memory
par: Kim, Juno, et autres
Publié: (2026)

Asymmetric Prompt Weighting for Reinforcement Learning with Verifiable Rewards
par: Heckel, Reinhard, et autres
Publié: (2026)

Theoretical Insights into Overparameterized Models in Multi-Task and Replay-Based Continual Learning
par: Banayeeanzade, Amin, et autres
Publié: (2024)

From Shortcut to Induction Head: How Data Diversity Shapes Algorithm Selection in Transformers
par: Kawata, Ryotaro, et autres
Publié: (2025)

Training Dynamics of Softmax Self-Attention: Fast Global Convergence via Preconditioning
par: Goel, Gautam, et autres
Publié: (2026)

FoNE: Precise Single-Token Number Embeddings via Fourier Features
par: Zhou, Tianyi, et autres
Publié: (2025)

Provable Multi-Task Representation Learning by Two-Layer ReLU Neural Networks
par: Collins, Liam, et autres
Publié: (2023)

Test-Time Training Provably Improves Transformers as In-context Learners
par: Gozeten, Halil Alperen, et autres
Publié: (2025)

MosaicMRI: A Diverse Dataset and Benchmark for Raw Musculoskeletal MRI
par: Arguello, Paula, et autres
Publié: (2026)

Learning Associative Memories with Gradient Descent
par: Cabannes, Vivien, et autres
Publié: (2024)

Adapt and Diffuse: Sample-adaptive Reconstruction via Latent Diffusion Models
par: Fabian, Zalan, et autres
Publié: (2023)

The Rich and the Simple: On the Implicit Bias of Adam and SGD
par: Vasudeva, Bhavya, et autres
Publié: (2025)

Assign and Add: A Mechanistic Study of Compositional Arithmetic
par: Exoo, Brady, et autres
Publié: (2026)

CrispEdit: Low-Curvature Projections for Scalable Non-Destructive LLM Editing
par: Ikram, Zarif, et autres
Publié: (2026)

Emergence and Evolution of Interpretable Concepts in Diffusion Models
par: Tinaz, Berk, et autres
Publié: (2025)

Geometric Factual Recall in Transformers
par: Ravfogel, Shauli, et autres
Publié: (2026)

DiracDiffusion: Denoising and Incremental Reconstruction with Assured Data-Consistency
par: Fabian, Zalan, et autres
Publié: (2023)

LightSplit: Practical Privacy-Preserving Split Learning via Orthogonal Projections
par: Cihangiroglu, Mert, et autres
Publié: (2026)

Selective Embedding for Deep Learning
par: Sehri, Mert, et autres
Publié: (2025)

CryptoMamba: Leveraging State Space Models for Accurate Bitcoin Price Prediction
par: Sepehri, Mohammad Shahab, et autres
Publié: (2025)

Negotiated Representations to Prevent Overfitting in Machine Learning Applications
par: Korhan, Nuri, et autres
Publié: (2023)

Multimodal Alignment and Preference Optimization for Zero-Shot Conditional RNA Generation
par: Klypa, Roman, et autres
Publié: (2026)

BAnG: Bidirectional Anchored Generation for Conditional RNA Design
par: Klypa, Roman, et autres
Publié: (2025)

Serpent: Scalable and Efficient Image Restoration via Multi-scale Structured State Space Models
par: Sepehri, Mohammad Shahab, et autres
Publié: (2024)

Fine-Tuning Dynamics of In-Context Factual Recall in Transformers
par: Huang, Ruomin, et autres
Publié: (2026)

Recall: Empowering Multimodal Embedding for Edge Devices
par: Cai, Dongqi, et autres
Publié: (2024)

Counterfactual Learning of Stochastic Policies with Continuous Actions
par: Zenati, Houssam, et autres
Publié: (2020)

Level Set Teleportation: An Optimization Perspective
par: Mishkin, Aaron, et autres
Publié: (2024)

Distributional Associations vs In-Context Reasoning: A Study of Feed-forward and Attention Layers
par: Chen, Lei, et autres
Publié: (2024)

ConceptMix++: Leveling the Playing Field in Text-to-Image Benchmarking via Iterative Prompt Optimization
par: Gan, Haosheng, et autres
Publié: (2025)

Probably Approximately Precision and Recall Learning
par: Cohen, Lee, et autres
Publié: (2024)

RoBERTurk: Adjusting RoBERTa for Turkish
par: Tas, Nuri
Publié: (2024)

How Transformers Learn In-Context Recall Tasks? Optimality, Training Dynamics and Generalization
par: Nguyen, Quan, et autres
Publié: (2025)