Cover Image

Probing In-Context Learning: Impact of Task Complexity and Model Architecture on Generalization and Efficiency

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liu, Binwen, Xu, Peiyu, Yuan, Quan, Chen, Yihong
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

How Transformers Learn In-Context Recall Tasks? Optimality, Training Dynamics and Generalization
by: Nguyen, Quan, et al.
Published: (2025)

Can Custom Models Learn In-Context? An Exploration of Hybrid Architecture Performance on In-Context Learning Tasks
by: Campbell, Ryan, et al.
Published: (2024)

ComplexityNet: Increasing LLM Inference Efficiency by Learning Task Complexity
by: Bae, Henry, et al.
Published: (2023)

Investigating the Impact of Data Selection Strategies on Language Model Performance
by: Gu, Jiayao, et al.
Published: (2025)

Pretrain-Test Task Alignment Governs Generalization in In-Context Learning
by: Letey, Mary I., et al.
Published: (2025)

Learning to Flow from Generative Pretext Tasks for Neural Architecture Encoding
by: Kim, Sunwoo, et al.
Published: (2025)

IM-Context: In-Context Learning for Imbalanced Regression Tasks
by: Nejjar, Ismail, et al.
Published: (2024)

Context-Scaling versus Task-Scaling in In-Context Learning
by: Abedsoltan, Amirhesam, et al.
Published: (2024)

Mimic In-Context Learning for Multimodal Tasks
by: Jiang, Yuchu, et al.
Published: (2025)

Task Vectors in In-Context Learning: Emergence, Formation, and Benefit
by: Yang, Liu, et al.
Published: (2025)

MOBODY: Model Based Off-Dynamics Offline Reinforcement Learning
by: Guo, Yihong, et al.
Published: (2025)

Cross-Domain Energy-Guided Diffusion Generation for Off-Dynamics Reinforcement Learning
by: Yang, Yu, et al.
Published: (2026)

GFPL: Generative Federated Prototype Learning for Resource-Constrained and Data-Imbalanced Vision Task
by: Lu, Shiwei, et al.
Published: (2026)

Understanding the Generalization of In-Context Learning in Transformers: An Empirical Study
by: Zhang, Xingxuan, et al.
Published: (2025)

Towards Zero-Shot Task-Generalizable Learning on fMRI
by: Wang, Jiyao, et al.
Published: (2025)

Learning Robust and Task-Invariant Functional Representation from fMRI through Siamese Self-Supervised Learning
by: Wang, Jiyao, et al.
Published: (2026)

Learning a High-quality Robotic Wiping Policy Using Systematic Reward Analysis and Visual-Language Model Based Curriculum
by: Liu, Yihong, et al.
Published: (2025)

Learning to Order: Task Sequencing as In-Context Optimization
by: Kobiolka, Jan, et al.
Published: (2026)

Technical Debt in In-Context Learning: Diminishing Efficiency in Long Context
by: Joo, Taejong, et al.
Published: (2025)

PyGALAX: An Open-Source Python Toolkit for Advanced Explainable Geospatial Machine Learning
by: Wang, Pingping, et al.
Published: (2026)

Fast-PGM: Fast Probabilistic Graphical Model Learning and Inference
by: Jiang, Jiantong, et al.
Published: (2024)

In-Context Language Learning: Architectures and Algorithms
by: Akyürek, Ekin, et al.
Published: (2024)

ReAugment: Model Zoo-Guided RL for Few-Shot Time Series Augmentation and Forecasting
by: Yuan, Haochen, et al.
Published: (2024)

How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression?
by: Wu, Jingfeng, et al.
Published: (2023)

Task Aware Dreamer for Task Generalization in Reinforcement Learning
by: Ying, Chengyang, et al.
Published: (2023)

In-Context Transfer Learning: Demonstration Synthesis by Transferring Similar Tasks
by: Wang, Dingzirui, et al.
Published: (2024)

Is attention required for ICL? Exploring the Relationship Between Model Architecture and In-Context Learning Ability
by: Lee, Ivan, et al.
Published: (2023)

AMOSL: Adaptive Modality-wise Structure Learning in Multi-view Graph Neural Networks For Enhanced Unified Representation
by: Liang, Peiyu, et al.
Published: (2024)

Learning Task Representations from In-Context Learning
by: Saglam, Baturay, et al.
Published: (2025)

Distributionally Robust Policy Evaluation under General Covariate Shift in Contextual Bandits
by: Guo, Yihong, et al.
Published: (2024)

DimGrow: Memory-Efficient Field-level Embedding Dimension Search
by: Huang, Yihong, et al.
Published: (2025)

RoboCoder: Robotic Learning from Basic Skills to General Tasks with Large Language Models
by: Li, Jingyao, et al.
Published: (2024)

What Do Language Models Learn in Context? The Structured Task Hypothesis
by: Li, Jiaoda, et al.
Published: (2024)

AsEP: Benchmarking Deep Learning Methods for Antibody-specific Epitope Prediction
by: Liu, Chunan, et al.
Published: (2024)

Understanding Task Vectors in In-Context Learning: Emergence, Functionality, and Limitations
by: Dong, Yuxin, et al.
Published: (2025)

Shielded Deep Reinforcement Learning for Complex Spacecraft Tasking
by: Reed, Robert, et al.
Published: (2024)

Generalized Graph Prompt: Toward a Unification of Pre-Training and Downstream Tasks on Graphs
by: Yu, Xingtong, et al.
Published: (2023)

Assessing the Impact of Sequence Length Learning on Classification Tasks for Transformer Encoder Models
by: Baillargeon, Jean-Thomas, et al.
Published: (2022)

Decomposed Inductive Procedure Learning: Learning Academic Tasks with Human-Like Data Efficiency
by: Weitekamp, Daniel, et al.
Published: (2025)

Can Mamba Learn How to Learn? A Comparative Study on In-Context Learning Tasks
by: Park, Jongho, et al.
Published: (2024)