Cover Image

On the Learn-to-Optimize Capabilities of Transformers in In-Context Sparse Recovery

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liu, Renpu, Zhou, Ruida, Shen, Cong, Yang, Jing
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

A Shared Low-Rank Adaptation Approach to Personalized RLHF
by: Liu, Renpu, et al.
Published: (2025)

Data-adaptive Differentially Private Prompt Synthesis for In-Context Learning
by: Gao, Fengyu, et al.
Published: (2024)

Unlabeled Data Can Provably Enhance In-Context Learning of Transformers
by: Liu, Renpu, et al.
Published: (2026)

Federated Representation Learning in the Under-Parameterized Regime
by: Liu, Renpu, et al.
Published: (2024)

Investigation into In-Context Learning Capabilities of Transformers
by: Chandrupatla, Rushil, et al.
Published: (2026)

On the Training Convergence of Transformers for In-Context Classification of Gaussian Mixtures
by: Shen, Wei, et al.
Published: (2024)

Relational Learning in Pre-Trained Models: A Theory from Hypergraph Recovery Perspective
by: Chen, Yang, et al.
Published: (2024)

Efficient Multi-objective Prompt Optimization via Pure-exploration Bandits
by: Li, Donghao, et al.
Published: (2026)

FastGAS: Fast Graph-based Annotation Selection for In-Context Learning
by: Chen, Zihan, et al.
Published: (2024)

Path-Guided Particle-based Sampling
by: Fan, Mingzhou, et al.
Published: (2024)

Reinforcement Learning With Sparse-Executing Actions via Sparsity Regularization
by: Pang, Jing-Cheng, et al.
Published: (2021)

HiPER: Hierarchical Reinforcement Learning with Explicit Credit Assignment for Large Language Model Agents
by: Peng, Jiangweizhi, et al.
Published: (2026)

MISA: Mixture of Indexer Sparse Attention for Long-Context LLM Inference
by: Zhou, Ruijie, et al.
Published: (2026)

Do pretrained Transformers Learn In-Context by Gradient Descent?
by: Shen, Lingfeng, et al.
Published: (2023)

On the Generalization Capability of Temporal Graph Learning Algorithms: Theoretical Insights and a Simpler Method
by: Cong, Weilin, et al.
Published: (2024)

Spectral Unforgetting: Post-Hoc Recovery of Damaged Capabilities Without Retraining
by: Abro, Aarash, et al.
Published: (2026)

Exploring the In-Context Learning Capabilities of LLMs for Money Laundering Detection in Financial Graphs
by: Pirmorad, Erfan
Published: (2025)

Quantifying Multimodal Capabilities: Formal Generalization Guarantees in Pairwise Metric Learning
by: Zhou, Richeng, et al.
Published: (2026)

Efficient Prompt Optimization Through the Lens of Best Arm Identification
by: Shi, Chengshuai, et al.
Published: (2024)

Orion-MSP: Multi-Scale Sparse Attention for Tabular In-Context Learning
by: Bouadi, Mohamed, et al.
Published: (2025)

GAR: Generative Adversarial Reinforcement Learning for Formal Theorem Proving
by: Wang, Ruida, et al.
Published: (2025)

Theoretical Analysis of Sparse Optimization with Reparameterization, Weight Decay, and Adaptive Learning Rate
by: Xu, Huangyu, et al.
Published: (2026)

In-Context Decision Transformer: Reinforcement Learning via Hierarchical Chain-of-Thought
by: Huang, Sili, et al.
Published: (2024)

Self-Tuning Sparse Attention: Multi-Fidelity Hyperparameter Optimization for Transformer Acceleration
by: Dev, Arundhathi, et al.
Published: (2026)

Beyond Binary Preferences: A Principled Framework for Reward Modeling with Ordinal Feedback
by: Afsharrad, Amirhossein, et al.
Published: (2026)

Meta-Learning Transformers to Improve In-Context Generalization
by: Braccaioli, Lorenzo, et al.
Published: (2025)

Reflective Context Learning: Studying the Optimization Primitives of Context Space
by: Vassilyev, Nikita, et al.
Published: (2026)

Reframing Data Value for Large Language Models Through the Lens of Plausibility
by: Rammal, Mohamad Rida, et al.
Published: (2024)

Learning in Context, Guided by Choice: A Reward-Free Paradigm for Reinforcement Learning with Transformers
by: Dong, Juncheng, et al.
Published: (2026)

Learning Spatio-Temporal Dynamics for Trajectory Recovery via Time-Aware Transformer
by: Sun, Tian, et al.
Published: (2025)

Transform then Explore: a Simple and Effective Technique for Exploratory Combinatorial Optimization with Reinforcement Learning
by: Pu, Tianle, et al.
Published: (2024)

SparseBalance: Load-Balanced Long Context Training with Dynamic Sparse Attention
by: Xu, Hongtao, et al.
Published: (2026)

Context-aware Diversity Enhancement for Neural Multi-Objective Combinatorial Optimization
by: Lu, Yongfan, et al.
Published: (2024)

Transformers Don't In-Context Learn Least Squares Regression
by: Hill, Joshua, et al.
Published: (2025)

Heuristic Transformer: Belief Augmented In-Context Reinforcement Learning
by: Dippel, Oliver, et al.
Published: (2025)

In-Context Algorithm Emulation in Fixed-Weight Transformers
by: Hu, Jerry Yao-Chieh, et al.
Published: (2025)

FLUID: Continuous-Time Hyperconnected Sparse Transformer for Sink-Free Learning
by: Razzaq, Waleed, et al.
Published: (2026)

To Theoretically Understand Transformer-Based In-Context Learning for Optimizing CSMA
by: Hao, Shugang, et al.
Published: (2025)

Generative Pre-Trained Transformer for Symbolic Regression Base In-Context Reinforcement Learning
by: Li, Yanjie, et al.
Published: (2024)

Iterative Amortized Inference: Unifying In-Context Learning and Learned Optimizers
by: Mittal, Sarthak, et al.
Published: (2025)