Imagen de Portada

Supervised Fine-Tuning Achieve Rapid Task Adaption Via Alternating Attention Head Activation Patterns

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Zhao, Yang, Du, Li, Ding, Xiao, Xiong, Kai, Liu, Ting, Qin, Bing
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Toward Secure Tuning: Mitigating Security Risks from Instruction Fine-Tuning
por: Du, Yanrui, et al.
Publicado: (2024)

Examining Inter-Consistency of Large Language Models Collaboration: An In-depth Analysis via Debate
por: Xiong, Kai, et al.
Publicado: (2023)

Deciphering the Impact of Pretraining Data on Large Language Models through Machine Unlearning
por: Zhao, Yang, et al.
Publicado: (2024)

Beyond Similarity: A Gradient-based Graph Method for Instruction Tuning Data Selection
por: Zhao, Yang, et al.
Publicado: (2025)

Diagnosing and Remedying Knowledge Deficiencies in LLMs via Label-free Curricular Meaningful Learning
por: Xiong, Kai, et al.
Publicado: (2024)

CrossICL: Cross-Task In-Context Learning via Unsupervised Demonstration Transfer
por: Gao, Jinglong, et al.
Publicado: (2025)

AdaZeta: Adaptive Zeroth-Order Tensor-Train Adaption for Memory-Efficient Large Language Models Fine-Tuning
por: Yang, Yifan, et al.
Publicado: (2024)

Text Difficulty Study: Do machines behave the same as humans regarding text difficulty?
por: Chen, Bowen, et al.
Publicado: (2022)

IA2: Alignment with ICL Activations Improves Supervised Fine-Tuning
por: Mishra, Aayush, et al.
Publicado: (2025)

CC-Tuning: A Cross-Lingual Connection Mechanism for Improving Joint Multilingual Supervised Fine-Tuning
por: Ye, Yangfan, et al.
Publicado: (2025)

Meaningful Learning: Enhancing Abstract Reasoning in Large Language Models via Generic Fact Guidance
por: Xiong, Kai, et al.
Publicado: (2024)

Anchoring Refusal Direction: Mitigating Safety Risks in Tuning via Projection Constraint
por: Du, Yanrui, et al.
Publicado: (2025)

iTool: Reinforced Fine-Tuning with Dynamic Deficiency Calibration for Advanced Tool Use
por: Zeng, Yirong, et al.
Publicado: (2025)

Anchored Supervised Fine-Tuning
por: Zhu, He, et al.
Publicado: (2025)

Mind the Gap: Data Rewriting for Stable Off-Policy Supervised Fine-Tuning
por: Zhao, Shiwan, et al.
Publicado: (2025)

Beware of Your Po! Measuring and Mitigating AI Safety Risks in Role-Play Fine-Tuning of LLMs
por: Zhao, Weixiang, et al.
Publicado: (2025)

Self-Route: Automatic Mode Switching via Capability Estimation for Efficient Reasoning
por: He, Yang, et al.
Publicado: (2025)

On the Role of Reasoning Patterns in the Generalization Discrepancy of Long Chain-of-Thought Supervised Fine-Tuning
por: Li, Zhaoyi, et al.
Publicado: (2026)

MAESTRO: Meta-learning Adaptive Estimation of Scalarization Trade-offs for Reward Optimization
por: Zhao, Yang, et al.
Publicado: (2026)

Proximal Supervised Fine-Tuning
por: Zhu, Wenhong, et al.
Publicado: (2025)

Separate the Wheat from the Chaff: A Post-Hoc Approach to Safety Re-Alignment for Fine-Tuned Language Models
por: Wu, Di, et al.
Publicado: (2024)

Com$^2$: A Causal-Guided Benchmark for Exploring Complex Commonsense Reasoning in Large Language Models
por: Xiong, Kai, et al.
Publicado: (2025)

ReCo: Reliable Causal Chain Reasoning via Structural Causal Recurrent Neural Networks
por: Xiong, Kai, et al.
Publicado: (2022)

Improved Supervised Fine-Tuning for Large Language Models to Mitigate Catastrophic Forgetting
por: Ding, Fei, et al.
Publicado: (2025)

UFT: Unifying Supervised and Reinforcement Fine-Tuning
por: Liu, Mingyang, et al.
Publicado: (2025)

UFO-RL: Uncertainty-Focused Optimization for Efficient Reinforcement Learning Data Selection
por: Zhao, Yang, et al.
Publicado: (2025)

Localizing Task Recognition and Task Learning in In-Context Learning via Attention Head Analysis
por: Yang, Haolin, et al.
Publicado: (2025)

Large Language Models Are Still Misled by Simple Bias Ensembles
por: Sun, Zhouhao, et al.
Publicado: (2025)

Causal-Guided Active Learning for Debiasing Large Language Models
por: Du, Li, et al.
Publicado: (2024)

Information Gain-Guided Causal Intervention for Autonomous Debiasing Large Language Models
por: Sun, Zhouhao, et al.
Publicado: (2025)

Token Cleaning: Fine-Grained Data Selection for LLM Supervised Fine-Tuning
por: Pang, Jinlong, et al.
Publicado: (2025)

Fine-Grained Activation Steering: Steering Less, Achieving More
por: Feng, Zijian, et al.
Publicado: (2026)

SED-SFT: Selectively Encouraging Diversity in Supervised Fine-Tuning
por: Chen, Yijie, et al.
Publicado: (2026)

Rethinking Data Selection for Supervised Fine-Tuning
por: Shen, Ming
Publicado: (2024)

Task-KV: Task-aware KV Cache Optimization via Semantic Differentiation of Attention Heads
por: He, Xingyang, et al.
Publicado: (2025)

Achieving Sparse Activation in Small Language Models
por: Song, Jifeng, et al.
Publicado: (2024)

Enhancing Non-English Capabilities of English-Centric Large Language Models through Deep Supervision Fine-Tuning
por: Huo, Wenshuai, et al.
Publicado: (2025)

Knowledge Graph Error Detection with Contrastive Confidence Adaption
por: Liu, Xiangyu, et al.
Publicado: (2023)

Expert Pyramid Tuning: Efficient Parameter Fine-Tuning for Expertise-Driven Task Allocation
por: Zhang, Jia-Chen, et al.
Publicado: (2026)

Uncovering the Role of Initial Saliency in U-Shaped Attention Bias: Scaling Initial Token Weight for Enhanced Long-Text Processing
por: Qiang, Zewen, et al.
Publicado: (2025)