Image de couverture de livre

Investigating the Pre-Training Dynamics of In-Context Learning: Task Recognition vs. Task Learning

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Wang, Xiaolei, Tang, Xinyu, Zhao, Wayne Xin, Wen, Ji-Rong
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

DAWN-ICL: Strategic Planning of Problem-solving Trajectories for Zero-Shot In-Context Learning
par: Tang, Xinyu, et autres
Publié: (2024)

Learning Task Representations from In-Context Learning
par: Saglam, Baturay, et autres
Publié: (2025)

Exploring Context Window of Large Language Models via Decomposed Positional Vectors
par: Dong, Zican, et autres
Publié: (2024)

Towards High Data Efficiency in Reinforcement Learning with Verifiable Reward
par: Tang, Xinyu, et autres
Publié: (2025)

How does Multi-Task Training Affect Transformer In-Context Capabilities? Investigations with Function Classes
par: Bhasin, Harmon, et autres
Publié: (2024)

From Unstructured Data to In-Context Learning: Exploring What Tasks Can Be Learned and When
par: Wibisono, Kevin Christian, et autres
Publié: (2024)

In-Context Transfer Learning: Demonstration Synthesis by Transferring Similar Tasks
par: Wang, Dingzirui, et autres
Publié: (2024)

MultiGPrompt for Multi-Task Pre-Training and Prompting on Graphs
par: Yu, Xingtong, et autres
Publié: (2023)

Low-rank Optimization Trajectories Modeling for LLM RLVR Acceleration
par: Chen, Zhipeng, et autres
Publié: (2026)

What Do Language Models Learn in Context? The Structured Task Hypothesis
par: Li, Jiaoda, et autres
Publié: (2024)

Task Schema and Binding: A Double Dissociation Study of In-Context Learning
par: Kim, Chaeha
Publié: (2025)

Quiet Feature Learning in Algorithmic Tasks
par: Naidu, Prudhviraj, et autres
Publié: (2025)

Rethinking the Evaluation for Conversational Recommendation in the Era of Large Language Models
par: Wang, Xiaolei, et autres
Publié: (2023)

Learning Dynamics in Continual Pre-Training for Large Language Models
par: Wang, Xingjin, et autres
Publié: (2025)

Spoken Language Understanding on Unseen Tasks With In-Context Learning
par: Agrawal, Neeraj, et autres
Publié: (2025)

Emergence and Effectiveness of Task Vectors in In-Context Learning: An Encoder Decoder Perspective
par: Han, Seungwook, et autres
Publié: (2024)

Dynamic Prompt Fusion for Multi-Task and Cross-Domain Adaptation in LLMs
par: Hu, Xin, et autres
Publié: (2025)

Unleashing the Potential of Large Language Models as Prompt Optimizers: Analogical Analysis with Gradient-based Model Optimizers
par: Tang, Xinyu, et autres
Publié: (2024)

OrchMoE: Efficient Multi-Adapter Learning with Task-Skill Synergy
par: Wang, Haowen, et autres
Publié: (2024)

Optimizing Multi-Task Learning for Enhanced Performance in Large Language Models
par: Qi, Zhen, et autres
Publié: (2024)

Towards Understanding Multi-Task Learning (Generalization) of LLMs via Detecting and Exploring Task-Specific Neurons
par: Leng, Yongqi, et autres
Publié: (2024)

Reinforcement Learning on Pre-Training Data
par: Li, Siheng, et autres
Publié: (2025)

Everything Everywhere All at Once: LLMs can In-Context Learn Multiple Tasks in Superposition
par: Xiong, Zheyang, et autres
Publié: (2024)

Dynamic Embeddings with Task-Oriented prompting
par: Balloccu, Allmin, et autres
Publié: (2024)

Parallel Structures in Pre-training Data Yield In-Context Learning
par: Chen, Yanda, et autres
Publié: (2024)

MoTe: Learning Motion-Text Diffusion Model for Multiple Generation Tasks
par: Wu, Yiming, et autres
Publié: (2024)

Benchmarking Large Language Models for Math Reasoning Tasks
par: Seßler, Kathrin, et autres
Publié: (2024)

Time-MQA: Time Series Multi-Task Question Answering with Context Enhancement
par: Kong, Yaxuan, et autres
Publié: (2025)

Ltri-LLM: Streaming Long Context Inference for LLMs with Training-Free Dynamic Triangular Attention Pattern
par: Tang, Hongyin, et autres
Publié: (2024)

How Transformers Learn Regular Language Recognition: A Theoretical Study on Training Dynamics and Implicit Bias
par: Huang, Ruiquan, et autres
Publié: (2025)

Can we Soft Prompt LLMs for Graph Learning Tasks?
par: Liu, Zheyuan, et autres
Publié: (2024)

Layer by Layer: Uncovering Where Multi-Task Learning Happens in Instruction-Tuned Large Language Models
par: Zhao, Zheng, et autres
Publié: (2024)

HMS-BERT: Hybrid Multi-Task Self-Training for Multilingual and Multi-Label Cyberbullying Detection
par: Feng, Zixin, et autres
Publié: (2026)

Predicting Task Performance with Context-aware Scaling Laws
par: Montgomery, Kyle, et autres
Publié: (2025)

Multimodal Task Vectors Enable Many-Shot Multimodal In-Context Learning
par: Huang, Brandon, et autres
Publié: (2024)

WSM: Decay-Free Learning Rate Schedule via Checkpoint Merging for LLM Pre-training
par: Tian, Changxin, et autres
Publié: (2025)

NeuroLoRA: Context-Aware Neuromodulation for Parameter-Efficient Multi-Task Adaptation
par: Yang, Yuxin, et autres
Publié: (2026)

Towards Unified Task Embeddings Across Multiple Models: Bridging the Gap for Prompt-Based Large Language Models and Beyond
par: Wang, Xinyu, et autres
Publié: (2024)

KV Cache Offloading for Context-Intensive Tasks
par: Bocharnikov, Andrey, et autres
Publié: (2026)

Understanding Emergent In-Context Learning from a Kernel Regression Perspective
par: Han, Chi, et autres
Publié: (2023)