Cover Image

Mechanistic Fine-tuning for In-context Learning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Cho, Hakaze, Luo, Peng, Kato, Mariko, Kaenbyou, Rin, Inoue, Naoya
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Revisiting In-context Learning Inference Circuit in Large Language Models
by: Cho, Hakaze, et al.
Published: (2024)

Token-based Decision Criteria Are Suboptimal in In-context Learning
by: Cho, Hakaze, et al.
Published: (2024)

Affinity and Diversity: A Unified Metric for Demonstration Selection via Internal Representations
by: Kato, Mariko, et al.
Published: (2025)

Understanding Token Probability Encoding in Output Embeddings
by: Cho, Hakaze, et al.
Published: (2024)

Mechanism of Task-oriented Information Removal in In-context Learning
by: Cho, Hakaze, et al.
Published: (2025)

StaICC: Standardized Evaluation for Classification Task in In-context Learning
by: Cho, Hakaze, et al.
Published: (2025)

Binary Autoencoder for Mechanistic Interpretability of Large Language Models
by: Cho, Hakaze, et al.
Published: (2025)

The Transfer Neurons Hypothesis: An Underlying Mechanism for Language Latent Space Transitions in Multilingual LLMs
by: Tezuka, Hinata, et al.
Published: (2025)

Task Vectors, Learned Not Extracted: Performance Gains and Mechanistic Insight
by: Yang, Haolin, et al.
Published: (2025)

Transformer Copilot: Learning from The Mistake Log in LLM Fine-tuning
by: Zou, Jiaru, et al.
Published: (2025)

Exploring Memorization in Fine-tuned Language Models
by: Zeng, Shenglai, et al.
Published: (2023)

Learning Rate Matters: Vanilla LoRA May Suffice for LLM Fine-tuning
by: Lee, Yu-Ang, et al.
Published: (2026)

Localizing Task Recognition and Task Learning in In-Context Learning via Attention Head Analysis
by: Yang, Haolin, et al.
Published: (2025)

Efficient Model Development through Fine-tuning Transfer
by: Lin, Pin-Jie, et al.
Published: (2025)

Information Guided Regularization for Fine-tuning Language Models
by: Sharma, Mandar, et al.
Published: (2024)

Outlier-weighed Layerwise Sampling for LLM Fine-tuning
by: Li, Pengxiang, et al.
Published: (2024)

Parameter Efficient Fine-tuning via Explained Variance Adaptation
by: Paischer, Fabian, et al.
Published: (2024)

Fine-tuning Language Models with Generative Adversarial Reward Modelling
by: Yu, Zhang Ze, et al.
Published: (2023)

Efficient End-to-end Language Model Fine-tuning on Graphs
by: Xue, Rui, et al.
Published: (2023)

A Framework to Implement 1+N Multi-task Fine-tuning Pattern in LLMs Using the CGC-LORA Algorithm
by: Song, Chao, et al.
Published: (2024)

FeDeRA:Efficient Fine-tuning of Language Models in Federated Learning Leveraging Weight Decomposition
by: Yan, Yuxuan, et al.
Published: (2024)

APE: Selective Fine-tuning with Acceptance Criteria for Language Model Adaptation
by: Marín, Javier
Published: (2025)

Multitask Mayhem: Unveiling and Mitigating Safety Gaps in LLMs Fine-tuning
by: Jan, Essa, et al.
Published: (2024)

Random Masking Finds Winning Tickets for Parameter Efficient Fine-tuning
by: Xu, Jing, et al.
Published: (2024)

Sparse is Enough in Fine-tuning Pre-trained Large Language Models
by: Song, Weixi, et al.
Published: (2023)

Fine-tuning Multi-hop Question Answering with Hierarchical Graph Network
by: Xiong, Guanming
Published: (2020)

The Importance of Online Data: Understanding Preference Fine-tuning via Coverage
by: Song, Yuda, et al.
Published: (2024)

Utility-Diversity Aware Online Batch Selection for LLM Supervised Fine-tuning
by: Zou, Heming, et al.
Published: (2025)

Fundamental Safety-Capability Trade-offs in Fine-tuning Large Language Models
by: Chen, Pin-Yu, et al.
Published: (2025)

Selecting Large Language Model to Fine-tune via Rectified Scaling Law
by: Lin, Haowei, et al.
Published: (2024)

InstructAV: Instruction Fine-tuning Large Language Models for Authorship Verification
by: Hu, Yujia, et al.
Published: (2024)

Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templates
by: Lyu, Kaifeng, et al.
Published: (2024)

Taming Sensitive Weights : Noise Perturbation Fine-tuning for Robust LLM Quantization
by: Wang, Dongwei, et al.
Published: (2024)

Entity-level Factual Adaptiveness of Fine-tuning based Abstractive Summarization Models
by: Song, Jongyoon, et al.
Published: (2024)

NoisyICL: A Little Noise in Model Parameters Calibrates In-context Learning
by: Zhao, Yufeng, et al.
Published: (2024)

Implicit In-context Learning
by: Li, Zhuowei, et al.
Published: (2024)

PFDial: A Structured Dialogue Instruction Fine-tuning Method Based on UML Flowcharts
by: Zhang, Ming, et al.
Published: (2025)

LoX: Low-Rank Extrapolation Robustifies LLM Safety Against Fine-tuning
by: Perin, Gabriel J., et al.
Published: (2025)

$\textit{New News}$: System-2 Fine-tuning for Robust Integration of New Knowledge
by: Park, Core Francisco, et al.
Published: (2025)

AMAQ: Adaptive Mixed-bit Activation Quantization for Collaborative Parameter Efficient Fine-tuning
by: Song, Yurun, et al.
Published: (2025)