Cover Image

From Compression to Expression: A Layerwise Analysis of In-Context Learning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Jiang, Jiachen, Dong, Yuxin, Zhou, Jinxin, Zhu, Zhihui
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Tracing Representation Progression: Analyzing and Enhancing Layer-Wise Similarity
by: Jiang, Jiachen, et al.
Published: (2024)

Analyzing Fine-Grained Alignment and Enhancing Vision Understanding in Multimodal Language Models
by: Jiang, Jiachen, et al.
Published: (2025)

From Emergence to Control: Probing and Modulating Self-Reflection in Language Models
by: Zhu, Xudong, et al.
Published: (2025)

In-Context Learning for Non-Stationary MIMO Equalization
by: Jiang, Jiachen, et al.
Published: (2025)

Layerwise Recall and the Geometry of Interwoven Knowledge in LLMs
by: Lei, Ge, et al.
Published: (2025)

Outlier-weighed Layerwise Sampling for LLM Fine-tuning
by: Li, Pengxiang, et al.
Published: (2024)

Discourse-Aware In-Context Learning for Temporal Expression Normalization
by: Gautam, Akash Kumar, et al.
Published: (2024)

Comparative Analysis of Demonstration Selection Algorithms for LLM In-Context Learning
by: Shu, Dong, et al.
Published: (2024)

Layerwise Change of Knowledge in Neural Networks
by: Cheng, Xu, et al.
Published: (2024)

OjaKV: Context-Aware Online Low-Rank KV Cache Compression
by: Zhu, Yuxuan, et al.
Published: (2025)

Accelerating Prefilling for Long-Context LLMs via Sparse Pattern Sharing
by: Peng, Dan, et al.
Published: (2025)

AbsTopK: Rethinking Sparse Autoencoders For Bidirectional Features
by: Zhu, Xudong, et al.
Published: (2025)

No Mean Feat: Simple, Strong Baselines for Context Compression
by: Feldman, Yair, et al.
Published: (2025)

Rapid Word Learning Through Meta In-Context Learning
by: Wang, Wentao, et al.
Published: (2025)

In-context Autoencoder for Context Compression in a Large Language Model
by: Ge, Tao, et al.
Published: (2023)

Distinct Computations Emerge From Compositional Curricula in In-Context Learning
by: Lee, Jin Hwa, et al.
Published: (2025)

Adapting LLMs for Efficient Context Processing through Soft Prompt Compression
by: Wang, Cangqing, et al.
Published: (2024)

Boosting In-Context Learning in LLMs Through the Lens of Classical Supervised Learning
by: Gundem, Korel, et al.
Published: (2025)

Unraveling the Mechanics of Learning-Based Demonstration Selection for In-Context Learning
by: Liu, Hui, et al.
Published: (2024)

KVSculpt: KV Cache Compression as Distillation
by: Jiang, Bo, et al.
Published: (2026)

LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning
by: Pan, Rui, et al.
Published: (2024)

Uncovering Emergent Physics Representations Learned In-Context by Large Language Models
by: Song, Yeongwoo, et al.
Published: (2025)

Is In-Context Learning Learning?
by: de Wynter, Adrian
Published: (2025)

Adaptive Large Language Models By Layerwise Attention Shortcuts
by: Verma, Prateek, et al.
Published: (2024)

Goal-Directed Search Outperforms Goal-Agnostic Memory Compression in Long-Context Memory Tasks
by: Zheng, Yicong, et al.
Published: (2025)

Confucius3-Math: A Lightweight High-Performance Reasoning LLM for Chinese K-12 Mathematics Learning
by: Wu, Lixin, et al.
Published: (2025)

Parallel Structures in Pre-training Data Yield In-Context Learning
by: Chen, Yanda, et al.
Published: (2024)

Memorization in In-Context Learning
by: Golchin, Shahriar, et al.
Published: (2024)

Revisiting In-Context Learning with Long Context Language Models
by: Baek, Jinheon, et al.
Published: (2024)

DeltaEvolve: Accelerating Scientific Discovery through Momentum-Driven Evolution
by: Jiang, Jiachen, et al.
Published: (2026)

The Impossibility Triangle of Long-Context Modeling
by: Zhou, Yan
Published: (2026)

Batch Calibration: Rethinking Calibration for In-Context Learning and Prompt Engineering
by: Zhou, Han, et al.
Published: (2023)

Latent Context Compilation: Distilling Long Context into Compact Portable Memory
by: Li, Zeju, et al.
Published: (2026)

MoBA: Mixture of Block Attention for Long-Context LLMs
by: Lu, Enzhe, et al.
Published: (2025)

Learning to Compress Prompt in Natural Language Formats
by: Chuang, Yu-Neng, et al.
Published: (2024)

ConCISE: Confidence-guided Compression in Step-by-step Efficient Reasoning
by: Qiao, Ziqing, et al.
Published: (2025)

ICLR: In-Context Learning of Representations
by: Park, Core Francisco, et al.
Published: (2024)

Many-Shot In-Context Learning
by: Agarwal, Rishabh, et al.
Published: (2024)

In-Context Learning Learns Label Relationships but Is Not Conventional Learning
by: Kossen, Jannik, et al.
Published: (2023)

Teaching Language Models to Critique via Reinforcement Learning
by: Xie, Zhihui, et al.
Published: (2025)