Imagen de Portada

Surgery: Mitigating Harmful Fine-Tuning for Large Language Models via Attention Sink

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Liu, Guozhi, Lin, Weiwei, Huang, Tiansheng, Mo, Ruichao, Mu, Qi, Wang, Xiumin, Shen, Li
Formato:	Preprint
Publicado:	2026
Materias:	Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Targeted Vaccine: Safety Alignment for Large Language Models against Harmful Fine-Tuning via Layer-wise Perturbation
por: Liu, Guozhi, et al.
Publicado: (2024)

Pharmacist: Safety Alignment Data Curation for Large Language Models against Harmful Fine-tuning
por: Liu, Guozhi, et al.
Publicado: (2025)

Panacea: Mitigating Harmful Fine-tuning for Large Language Models via Post-fine-tuning Perturbation
por: Wang, Yibo, et al.
Publicado: (2025)

Booster: Tackling Harmful Fine-tuning for Large Language Models via Attenuating Harmful Perturbation
por: Huang, Tiansheng, et al.
Publicado: (2024)

Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Survey
por: Huang, Tiansheng, et al.
Publicado: (2024)

Antidote: Post-fine-tuning Safety Alignment for Large Language Models against Harmful Fine-tuning
por: Huang, Tiansheng, et al.
Publicado: (2024)

Virus: Harmful Fine-tuning Attack for Large Language Models Bypassing Guardrail Moderation
por: Huang, Tiansheng, et al.
Publicado: (2025)

Adaptive Defense against Harmful Fine-Tuning for Large Language Models via Bayesian Data Scheduler
por: Hu, Zixuan, et al.
Publicado: (2025)

Towards Explainable Harmful Meme Detection through Multimodal Debate between Large Language Models
por: Lin, Hongzhan, et al.
Publicado: (2024)

Mitigating Safety Tax via Distribution-Grounded Refinement in Large Reasoning Models
por: Xie, Yingsha, et al.
Publicado: (2026)

Attention Sinks in Diffusion Language Models
por: Rulli, Maximo Eduardo, et al.
Publicado: (2025)

PokeLLMon: A Human-Parity Agent for Pokemon Battles with Large Language Models
por: Hu, Sihao, et al.
Publicado: (2024)

SinkRouter: Sink-Aware Routing for Efficient Long-Context Decoding in Large Language and Multimodal Models
por: Liu, Junnan, et al.
Publicado: (2026)

Noise Augmented Fine Tuning for Mitigating Hallucinations in Large Language Models
por: Khadangi, Afshin, et al.
Publicado: (2025)

LFTF: Locating First and Then Fine-Tuning for Mitigating Gender Bias in Large Language Models
por: Qin, Zhanyue, et al.
Publicado: (2025)

Efficient Streaming Language Models with Attention Sinks
por: Xiao, Guangxuan, et al.
Publicado: (2023)

When Attention Sink Emerges in Language Models: An Empirical View
por: Gu, Xiangming, et al.
Publicado: (2024)

Fine-Tuning Code Language Models to Detect Cross-Language Bugs
por: Li, Zengyang, et al.
Publicado: (2025)

Assessing and Mitigating Data Memorization Risks in Fine-Tuned Large Language Models
por: Ramakrishnan, Badrinath, et al.
Publicado: (2025)

Improved Supervised Fine-Tuning for Large Language Models to Mitigate Catastrophic Forgetting
por: Ding, Fei, et al.
Publicado: (2025)

Hierarchical Alignment: Surgical Fine-Tuning via Functional Layer Specialization in Large Language Models
por: Zhang, Yukun, et al.
Publicado: (2025)

Attention Sinks Induce Gradient Sinks: Massive Activations as Gradient Regulators in Transformers
por: Chen, Yihong, et al.
Publicado: (2026)

Attention Sinks and Outliers in Attention Residuals
por: Luo, Haozheng, et al.
Publicado: (2026)

Beyond Fine-Tuning: Effective Strategies for Mitigating Hallucinations in Large Language Models for Data Analytics
por: Rumiantsau, Mikhail, et al.
Publicado: (2024)

Data Mixing Optimization for Supervised Fine-Tuning of Large Language Models
por: Li, Yuan, et al.
Publicado: (2025)

OneNet: A Fine-Tuning Free Framework for Few-Shot Entity Linking via Large Language Model Prompting
por: Liu, Xukai, et al.
Publicado: (2024)

Data Efficient Adaptation in Large Language Models via Continuous Low-Rank Fine-Tuning
por: Han, Xiao, et al.
Publicado: (2025)

Restoring Calibration for Aligned Large Language Models: A Calibration-Aware Fine-Tuning Approach
por: Xiao, Jiancong, et al.
Publicado: (2025)

CAT: Causal Attention Tuning For Injecting Fine-grained Causal Knowledge into Large Language Models
por: Han, Kairong, et al.
Publicado: (2025)

Memorization in Fine-Tuned Large Language Models
por: Savine, Danil
Publicado: (2025)

CTRAP: Embedding Collapse Trap to Safeguard Large Language Models from Harmful Fine-Tuning
por: Yi, Biao, et al.
Publicado: (2025)

The Spike, the Sparse and the Sink: Anatomy of Massive Activations and Attention Sinks
por: Sun, Shangwen, et al.
Publicado: (2026)

To Sink or Not to Sink: Visual Information Pathways in Large Vision-Language Models
por: Luo, Jiayun, et al.
Publicado: (2025)

Legal Documents Drafting with Fine-Tuned Pre-Trained Large Language Model
por: Lin, Chun-Hsien, et al.
Publicado: (2024)

A Self-Supervised Reinforcement Learning Approach for Fine-Tuning Large Language Models Using Cross-Attention Signals
por: Kiruluta, Andrew, et al.
Publicado: (2025)

SLASH the Sink: Sharpening Structural Attention Inside LLMs
por: Liu, Yiming, et al.
Publicado: (2026)

Improving Large Language Models with Concept-Aware Fine-Tuning
por: Chen, Michael K., et al.
Publicado: (2025)

Parameter-Efficient Fine-Tuning of Large Language Models via Deconvolution in Subspace
por: Zhang, Jia-Chen, et al.
Publicado: (2025)

Safety-Aware Fine-Tuning of Large Language Models
por: Choi, Hyeong Kyu, et al.
Publicado: (2024)

Phased Instruction Fine-Tuning for Large Language Models
por: Pang, Wei, et al.
Publicado: (2024)