Imagen de Portada

Investigation into In-Context Learning Capabilities of Transformers

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Chandrupatla, Rushil, Bangayan, Leo, Leng, Sebastian
Formato:	Preprint
Publicado:	2026
Materias:	Machine Learning Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

On the Learn-to-Optimize Capabilities of Transformers in In-Context Sparse Recovery
por: Liu, Renpu, et al.
Publicado: (2024)

Offline and Online KL-Regularized RLHF under Differential Privacy
por: Wu, Yulian, et al.
Publicado: (2025)

Exploring the In-Context Learning Capabilities of LLMs for Money Laundering Detection in Financial Graphs
por: Pirmorad, Erfan
Publicado: (2025)

Towards Theoretical Understanding of Transformer Test-Time Computing: Investigation on In-Context Linear Regression
por: Chen, Xingwu, et al.
Publicado: (2025)

Meta-Learning Transformers to Improve In-Context Generalization
por: Braccaioli, Lorenzo, et al.
Publicado: (2025)

Transformers Don't In-Context Learn Least Squares Regression
por: Hill, Joshua, et al.
Publicado: (2025)

Heuristic Transformer: Belief Augmented In-Context Reinforcement Learning
por: Dippel, Oliver, et al.
Publicado: (2025)

Transformers Learn Robust In-Context Regression under Distributional Uncertainty
por: Cao, Hoang T. H., et al.
Publicado: (2026)

Transformers Provably Implement In-Context Reinforcement Learning with Policy Improvement
por: Liang, Haodong, et al.
Publicado: (2026)

On the Role of Transformer Feed-Forward Layers in Nonlinear In-Context Learning
por: Sun, Haoyuan, et al.
Publicado: (2025)

MAC: Multi-Agent Constitution Learning
por: Thareja, Rushil, et al.
Publicado: (2026)

Scaling Agentic Capabilities, Not Context: Efficient Reinforcement Finetuning for Large Toolspaces
por: Gupta, Karan, et al.
Publicado: (2026)

Understanding Transformer Reasoning Capabilities via Graph Algorithms
por: Sanford, Clayton, et al.
Publicado: (2024)

Tight Lower Bounds and Improved Convergence in Performative Prediction
por: Khorsandi, Pedram, et al.
Publicado: (2024)

Interpreting Context Look-ups in Transformers: Investigating Attention-MLP Interactions
por: Neo, Clement, et al.
Publicado: (2024)

ContextBench: Modifying Contexts for Targeted Latent Activation
por: Graham, Robert, et al.
Publicado: (2025)

Self-supervised Graph Transformer with Contrastive Learning for Brain Connectivity Analysis towards Improving Autism Detection
por: Leng, Yicheng, et al.
Publicado: (2025)

Integrating Temporal and Structural Context in Graph Transformers for Relational Deep Learning
por: Lachi, Divyansha, et al.
Publicado: (2025)

In-Context Decision Transformer: Reinforcement Learning via Hierarchical Chain-of-Thought
por: Huang, Sili, et al.
Publicado: (2024)

Learning in Context, Guided by Choice: A Reward-Free Paradigm for Reinforcement Learning with Transformers
por: Dong, Juncheng, et al.
Publicado: (2026)

Honesty to Subterfuge: In-Context Reinforcement Learning Can Make Honest Models Reward Hack
por: McKee-Reid, Leo, et al.
Publicado: (2024)

DP-Fusion: Token-Level Differentially Private Inference for Large Language Models
por: Thareja, Rushil, et al.
Publicado: (2025)

Algorithmic Capabilities of Random Transformers
por: Zhong, Ziqian, et al.
Publicado: (2024)

IAA: Inner-Adaptor Architecture Empowers Frozen Large Language Model with Multimodal Capabilities
por: Wang, Bin, et al.
Publicado: (2024)

Transforming Behavioral Neuroscience Discovery with In-Context Learning and AI-Enhanced Tensor Methods
por: Goulart, Paimon, et al.
Publicado: (2026)

Is More Context Always Better? Examining LLM Reasoning Capability for Time Interval Prediction
por: Cao, Yanan, et al.
Publicado: (2026)

Generative Pre-Trained Transformer for Symbolic Regression Base In-Context Reinforcement Learning
por: Li, Yanjie, et al.
Publicado: (2024)

How Transformers Learn In-Context Recall Tasks? Optimality, Training Dynamics and Generalization
por: Nguyen, Quan, et al.
Publicado: (2025)

A Survey of Explainable Reinforcement Learning: Targets, Methods and Needs
por: Saulières, Léo
Publicado: (2025)

Eliciting Fine-Tuned Transformer Capabilities via Inference-Time Techniques
por: Sharma, Asankhaya
Publicado: (2025)

Streamlining Admission with LOR Insights: AI-Based Leadership Assessment in Online Master's Program
por: Soylu, Meryem Yilmaz, et al.
Publicado: (2025)

Scaling In-Context Online Learning Capability of LLMs via Cross-Episode Meta-RL
por: Lin, Xiaofeng, et al.
Publicado: (2026)

The Geometric Anatomy of Capability Acquisition in Transformers
por: Billa, Jayadev
Publicado: (2026)

Inner Loop Inference for Pretrained Transformers: Unlocking Latent Capabilities Without Training
por: Lys, Jonathan, et al.
Publicado: (2026)

Graph Diffusion Transformers are In-Context Molecular Designers
por: Liu, Gang, et al.
Publicado: (2025)

In-Context Algorithm Emulation in Fixed-Weight Transformers
por: Hu, Jerry Yao-Chieh, et al.
Publicado: (2025)

A Proposal for Networks Capable of Continual Learning
por: Erden, Zeki Doruk, et al.
Publicado: (2025)

Generalization Capability for Imitation Learning
por: Wang, Yixiao
Publicado: (2025)

Do pretrained Transformers Learn In-Context by Gradient Descent?
por: Shen, Lingfeng, et al.
Publicado: (2023)

Transformers Learn Low Sensitivity Functions: Investigations and Implications
por: Vasudeva, Bhavya, et al.
Publicado: (2024)