Copertina

Investigating the Pre-Training Dynamics of In-Context Learning: Task Recognition vs. Task Learning

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Wang, Xiaolei, Tang, Xinyu, Zhao, Wayne Xin, Wen, Ji-Rong
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Machine Learning Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

DAWN-ICL: Strategic Planning of Problem-solving Trajectories for Zero-Shot In-Context Learning
di: Tang, Xinyu, et al.
Pubblicazione: (2024)

Learning Task Representations from In-Context Learning
di: Saglam, Baturay, et al.
Pubblicazione: (2025)

Exploring Context Window of Large Language Models via Decomposed Positional Vectors
di: Dong, Zican, et al.
Pubblicazione: (2024)

Towards High Data Efficiency in Reinforcement Learning with Verifiable Reward
di: Tang, Xinyu, et al.
Pubblicazione: (2025)

How does Multi-Task Training Affect Transformer In-Context Capabilities? Investigations with Function Classes
di: Bhasin, Harmon, et al.
Pubblicazione: (2024)

From Unstructured Data to In-Context Learning: Exploring What Tasks Can Be Learned and When
di: Wibisono, Kevin Christian, et al.
Pubblicazione: (2024)

In-Context Transfer Learning: Demonstration Synthesis by Transferring Similar Tasks
di: Wang, Dingzirui, et al.
Pubblicazione: (2024)

MultiGPrompt for Multi-Task Pre-Training and Prompting on Graphs
di: Yu, Xingtong, et al.
Pubblicazione: (2023)

Low-rank Optimization Trajectories Modeling for LLM RLVR Acceleration
di: Chen, Zhipeng, et al.
Pubblicazione: (2026)

What Do Language Models Learn in Context? The Structured Task Hypothesis
di: Li, Jiaoda, et al.
Pubblicazione: (2024)

Task Schema and Binding: A Double Dissociation Study of In-Context Learning
di: Kim, Chaeha
Pubblicazione: (2025)

Quiet Feature Learning in Algorithmic Tasks
di: Naidu, Prudhviraj, et al.
Pubblicazione: (2025)

Rethinking the Evaluation for Conversational Recommendation in the Era of Large Language Models
di: Wang, Xiaolei, et al.
Pubblicazione: (2023)

Learning Dynamics in Continual Pre-Training for Large Language Models
di: Wang, Xingjin, et al.
Pubblicazione: (2025)

Spoken Language Understanding on Unseen Tasks With In-Context Learning
di: Agrawal, Neeraj, et al.
Pubblicazione: (2025)

Emergence and Effectiveness of Task Vectors in In-Context Learning: An Encoder Decoder Perspective
di: Han, Seungwook, et al.
Pubblicazione: (2024)

Dynamic Prompt Fusion for Multi-Task and Cross-Domain Adaptation in LLMs
di: Hu, Xin, et al.
Pubblicazione: (2025)

Unleashing the Potential of Large Language Models as Prompt Optimizers: Analogical Analysis with Gradient-based Model Optimizers
di: Tang, Xinyu, et al.
Pubblicazione: (2024)

OrchMoE: Efficient Multi-Adapter Learning with Task-Skill Synergy
di: Wang, Haowen, et al.
Pubblicazione: (2024)

Optimizing Multi-Task Learning for Enhanced Performance in Large Language Models
di: Qi, Zhen, et al.
Pubblicazione: (2024)

Towards Understanding Multi-Task Learning (Generalization) of LLMs via Detecting and Exploring Task-Specific Neurons
di: Leng, Yongqi, et al.
Pubblicazione: (2024)

Reinforcement Learning on Pre-Training Data
di: Li, Siheng, et al.
Pubblicazione: (2025)

Everything Everywhere All at Once: LLMs can In-Context Learn Multiple Tasks in Superposition
di: Xiong, Zheyang, et al.
Pubblicazione: (2024)

Dynamic Embeddings with Task-Oriented prompting
di: Balloccu, Allmin, et al.
Pubblicazione: (2024)

Parallel Structures in Pre-training Data Yield In-Context Learning
di: Chen, Yanda, et al.
Pubblicazione: (2024)

MoTe: Learning Motion-Text Diffusion Model for Multiple Generation Tasks
di: Wu, Yiming, et al.
Pubblicazione: (2024)

Benchmarking Large Language Models for Math Reasoning Tasks
di: Seßler, Kathrin, et al.
Pubblicazione: (2024)

Time-MQA: Time Series Multi-Task Question Answering with Context Enhancement
di: Kong, Yaxuan, et al.
Pubblicazione: (2025)

Ltri-LLM: Streaming Long Context Inference for LLMs with Training-Free Dynamic Triangular Attention Pattern
di: Tang, Hongyin, et al.
Pubblicazione: (2024)

How Transformers Learn Regular Language Recognition: A Theoretical Study on Training Dynamics and Implicit Bias
di: Huang, Ruiquan, et al.
Pubblicazione: (2025)

Can we Soft Prompt LLMs for Graph Learning Tasks?
di: Liu, Zheyuan, et al.
Pubblicazione: (2024)

Layer by Layer: Uncovering Where Multi-Task Learning Happens in Instruction-Tuned Large Language Models
di: Zhao, Zheng, et al.
Pubblicazione: (2024)

HMS-BERT: Hybrid Multi-Task Self-Training for Multilingual and Multi-Label Cyberbullying Detection
di: Feng, Zixin, et al.
Pubblicazione: (2026)

Predicting Task Performance with Context-aware Scaling Laws
di: Montgomery, Kyle, et al.
Pubblicazione: (2025)

Multimodal Task Vectors Enable Many-Shot Multimodal In-Context Learning
di: Huang, Brandon, et al.
Pubblicazione: (2024)

WSM: Decay-Free Learning Rate Schedule via Checkpoint Merging for LLM Pre-training
di: Tian, Changxin, et al.
Pubblicazione: (2025)

NeuroLoRA: Context-Aware Neuromodulation for Parameter-Efficient Multi-Task Adaptation
di: Yang, Yuxin, et al.
Pubblicazione: (2026)

Towards Unified Task Embeddings Across Multiple Models: Bridging the Gap for Prompt-Based Large Language Models and Beyond
di: Wang, Xinyu, et al.
Pubblicazione: (2024)

KV Cache Offloading for Context-Intensive Tasks
di: Bocharnikov, Andrey, et al.
Pubblicazione: (2026)

Understanding Emergent In-Context Learning from a Kernel Regression Perspective
di: Han, Chi, et al.
Pubblicazione: (2023)