Cover Image

Self-Attention as a Covariance Readout: A Unified View of In-Context Learning and Repetition

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Xu, Haoren, Fang, Guanhua
Format:	Preprint
Published:	2026
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Multi-Item-Query Attention for Stable Sequential Recommendation
by: Xu, Mingshi, et al.
Published: (2025)

Transformers as Unsupervised Learning Algorithms: A study on Gaussian Mixtures
by: Chen, Zhiheng, et al.
Published: (2025)

Learning under Commission and Omission Event Outliers
by: Zhang, Yuecheng, et al.
Published: (2025)

Unifying Perplexing Behaviors in Modified BP Attributions through Alignment Perspective
by: Zheng, Guanhua, et al.
Published: (2025)

General and Estimable Learning Bound Unifying Covariate and Concept Shifts
by: Chen, Hongbo, et al.
Published: (2025)

Self-Supervised Learning for Covariance Estimation
by: Diskin, Tzvi, et al.
Published: (2024)

Zero-Shot Time Series Forecasting with Covariates via In-Context Learning
by: Auer, Andreas, et al.
Published: (2025)

Self-Attention as Distributional Projection: A Unified Interpretation of Transformer Architecture
by: Mehta, Nihal
Published: (2025)

Continual Learning in the Presence of Repetition
by: Hemati, Hamed, et al.
Published: (2024)

On Non-asymptotic Theory of Recurrent Neural Networks in Temporal Point Processes
by: Chen, Zhiheng, et al.
Published: (2024)

Context-Enhanced Multi-View Trajectory Representation Learning: Bridging the Gap through Self-Supervised Models
by: Qian, Tangwen, et al.
Published: (2024)

Unifying Sparse Attention with Hierarchical Memory for Scalable Long-Context LLM Serving
by: Zhao, Zihan, et al.
Published: (2026)

Self-Supervised Learning for Neural Topic Models with Variance-Invariance-Covariance Regularization
by: Xu, Weiran, et al.
Published: (2025)

Attention as In-Context Empirical Bayes: A Two-Stage View via Particle Dynamics
by: Smart, Matthew, et al.
Published: (2026)

Are Independently Estimated View Uncertainties Comparable? Unified Routing for Trusted Multi-View Classification
by: Zhang, Yilin, et al.
Published: (2026)

Baguan-TS: A Sequence-Native In-Context Learning Model for Time Series Forecasting with Covariates
by: Yang, Linxiao, et al.
Published: (2026)

Contrastive Graph Condensation: Advancing Data Versatility through Self-Supervised Learning
by: Gao, Xinyi, et al.
Published: (2024)

Training Dynamics of In-Context Learning in Linear Attention
by: Zhang, Yedi, et al.
Published: (2025)

USP: A Unified Sequence Parallelism Approach for Long Context Generative AI
by: Fang, Jiarui, et al.
Published: (2024)

A Unified View on Solving Objective Mismatch in Model-Based Reinforcement Learning
by: Wei, Ran, et al.
Published: (2023)

Learning Repetition-Invariant Representations for Polymer Informatics
by: Zhu, Yihan, et al.
Published: (2025)

SPECTRE: An FFT-Based Efficient Drop-In Replacement to Self-Attention for Long Contexts
by: Fein-Ashley, Jacob, et al.
Published: (2025)

CodeSAM: Source Code Representation Learning by Infusing Self-Attention with Multi-Code-View Graphs
by: Mathai, Alex, et al.
Published: (2024)

On Understanding Attention-Based In-Context Learning for Categorical Data
by: Wang, Aaron T., et al.
Published: (2024)

Analyzing and Mitigating Repetitions in Trip Recommendation
by: Shu, Wenzheng, et al.
Published: (2025)

Self-Attention Mechanism in Multimodal Context for Banking Transaction Flow
by: Delestre, Cyrile, et al.
Published: (2024)

Datasets, Documents, and Repetitions: The Practicalities of Unequal Data Quality
by: Fang, Alex, et al.
Published: (2025)

Breaking Symmetry Bottlenecks in GNN Readouts
by: Talhi, Mouad, et al.
Published: (2026)

ID and Graph View Contrastive Learning with Multi-View Attention Fusion for Sequential Recommendation
by: Zhou, Xiaofan, et al.
Published: (2026)

Unified View Imputation and Feature Selection Learning for Incomplete Multi-view Data
by: Huang, Yanyong, et al.
Published: (2024)

MCCE: Missingness-aware Causal Concept Explainer
by: Gao, Jifan, et al.
Published: (2024)

Inherent Trade-Offs between Diversity and Stability in Multi-Task Benchmarks
by: Zhang, Guanhua, et al.
Published: (2024)

Sketching the Readout of Large Language Models for Scalable Data Attribution and Valuation
by: Ran, Yide, et al.
Published: (2026)

Scaling Laws and In-Context Learning: A Unified Theoretical Framework
by: Mehta, Sushant, et al.
Published: (2025)

Unified View of Grokking, Double Descent and Emergent Abilities: A Perspective from Circuits Competition
by: Huang, Yufei, et al.
Published: (2024)

Intelligent Repetition Counting for Unseen Exercises: A Few-Shot Learning Approach with Sensor Signals
by: Lim, Yooseok, et al.
Published: (2024)

Retrieval Backward Attention without Additional Training: Enhance Embeddings of Large Language Models via Repetition
by: Duan, Yifei, et al.
Published: (2025)

Double Self-weighted Multi-view Clustering via Adaptive View Fusion
by: Fang, Xiang, et al.
Published: (2020)

A Unified Density Operator View of Flow Control and Merging
by: De Santi, Riccardo, et al.
Published: (2026)

Iterative Amortized Inference: Unifying In-Context Learning and Learned Optimizers
by: Mittal, Sarthak, et al.
Published: (2025)