Cover Image

In-Context Compositional Learning via Sparse Coding Transformer

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chen, Wei, Yu, Jingxi, Miao, Zichen, Qiu, Qiang
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Sparse Fine-Tuning of Transformers for Generative Tasks
by: Chen, Wei, et al.
Published: (2025)

Large Convolutional Model Tuning via Filter Subspace
by: Chen, Wei, et al.
Published: (2024)

Training Bayesian Neural Networks with Sparse Subspace Variational Inference
by: Li, Junbo, et al.
Published: (2024)

Robot Learning with Sparsity and Scarcity
by: Xu, Jingxi
Published: (2025)

Extra Clients at No Extra Cost: Overcome Data Heterogeneity in Federated Learning with Filter Decomposition
by: Chen, Wei, et al.
Published: (2025)

On the Learn-to-Optimize Capabilities of Transformers in In-Context Sparse Recovery
by: Liu, Renpu, et al.
Published: (2024)

How Transformers Utilize Multi-Head Attention in In-Context Learning? A Case Study on Sparse Linear Regression
by: Chen, Xingwu, et al.
Published: (2024)

Calibrating Transformers via Sparse Gaussian Processes
by: Chen, Wenlong, et al.
Published: (2023)

Transformers Meet In-Context Learning: A Universal Approximation Theory
by: Li, Gen, et al.
Published: (2025)

In-Context Deep Learning via Transformer Models
by: Wu, Weimin, et al.
Published: (2024)

How Do Nonlinear Transformers Learn and Generalize in In-Context Learning?
by: Li, Hongkang, et al.
Published: (2024)

Locality Sensitive Sparse Encoding for Learning World Models Online
by: Liu, Zichen, et al.
Published: (2024)

Learning on Transformers is Provable Low-Rank and Sparse: A One-layer Analysis
by: Li, Hongkang, et al.
Published: (2024)

SFi-Former: Sparse Flow Induced Attention for Graph Transformer
by: Li, Zhonghao, et al.
Published: (2025)

Learning Sparse Codes with Entropy-Based ELBOs
by: Velychko, Dmytro, et al.
Published: (2023)

Unifying Sparse Attention with Hierarchical Memory for Scalable Long-Context LLM Serving
by: Zhao, Zihan, et al.
Published: (2026)

In-Context In-Context Learning with Transformer Neural Processes
by: Ashman, Matthew, et al.
Published: (2024)

Sparse Mean Estimation in Adversarial Settings via Incremental Learning
by: Ma, Jianhao, et al.
Published: (2023)

Can Transformers Break Encryption Schemes via In-Context Learning?
by: Korrapati, Jathin, et al.
Published: (2025)

Dimensional Collapse in Transformer Attention Outputs: A Challenge for Sparse Dictionary Learning
by: Wang, Junxuan, et al.
Published: (2025)

In-Context Decision Transformer: Reinforcement Learning via Hierarchical Chain-of-Thought
by: Huang, Sili, et al.
Published: (2024)

Adaptive Sparse Möbius Transforms for Learning Polynomials
by: Erginbas, Yigit Efe, et al.
Published: (2026)

TabNSA: Native Sparse Attention for Efficient Tabular Data Learning
by: Eslamian, Ali, et al.
Published: (2025)

Transformers Learn Latent Mixture Models In-Context via Mirror Descent
by: D'Angelo, Francesco, et al.
Published: (2026)

Stop Probing, Start Coding: Why Linear Probes and Sparse Autoencoders Fail at Compositional Generalisation
by: Pacela, Vitória Barin, et al.
Published: (2026)

Understanding the Generalization of In-Context Learning in Transformers: An Empirical Study
by: Zhang, Xingxuan, et al.
Published: (2025)

Scalable Structure Learning for Sparse Context-Specific Systems
by: Rios, Felix Leopoldo, et al.
Published: (2024)

Towards Theoretical Understanding of Transformer Test-Time Computing: Investigation on In-Context Linear Regression
by: Chen, Xingwu, et al.
Published: (2025)

Free Energy Surface Sampling via Reduced Flow Matching
by: Liu, Zichen, et al.
Published: (2026)

Learning Sparse Compositional Functions with Norm-Constrained Neural Networks
by: Huang, Shuo, et al.
Published: (2026)

Transformers Provably Learn Sparse XOR with Polylogarithmic Parameters
by: Han, Yaomengxi, et al.
Published: (2025)

Binary Sparse Coding for Interpretability
by: Quirke, Lucia, et al.
Published: (2025)

Provable In-Context Learning of Nonlinear Regression with Transformers
by: Li, Hongbo, et al.
Published: (2025)

Automatic Domain Adaptation by Transformers in In-Context Learning
by: Hataya, Ryuichiro, et al.
Published: (2024)

In-Context Learning Enhanced Credibility Transformer
by: Padayachy, Kishan, et al.
Published: (2025)

SURGE: On the Potential of Large Language Models as General-Purpose Surrogate Code Executors
by: Lyu, Bohan, et al.
Published: (2025)

Transformer Learns Optimal Variable Selection in Group-Sparse Classification
by: Zhang, Chenyang, et al.
Published: (2025)

Transformers Implement Functional Gradient Descent to Learn Non-Linear Functions In Context
by: Cheng, Xiang, et al.
Published: (2023)

SURE-RAG: Sufficiency and Uncertainty-Aware Evidence Verification for Selective Retrieval-Augmented Generation
by: Qiu, Jingxi, et al.
Published: (2026)

Exact Conversion of In-Context Learning to Model Weights in Linearized-Attention Transformers
by: Chen, Brian K, et al.
Published: (2024)