Gespeichert in:
| Hauptverfasser: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2503.19041 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915685137907712 |
|---|---|
| author | Liu, Kangwei Wang, Mengru Luo, Yujie Yuan, Lin Sun, Mengshu Liang, Lei Zhang, Zhiqiang Zhou, Jun Hooi, Bryan Deng, Shumin |
| author_facet | Liu, Kangwei Wang, Mengru Luo, Yujie Yuan, Lin Sun, Mengshu Liang, Lei Zhang, Zhiqiang Zhou, Jun Hooi, Bryan Deng, Shumin |
| contents | Fine-tuning enables large language models (LLMs) to adapt to specific domains, but often compromises their previously established safety alignment. To mitigate the degradation of model safety during fine-tuning, we introduce LookAhead Tuning, a lightweight and effective data-driven approach that preserves safety during fine-tuning. The method introduces two simple strategies that modify training data by previewing partial answer prefixes, thereby minimizing perturbations to the model's initial token distributions and maintaining its built-in safety mechanisms. Comprehensive experiments demonstrate that LookAhead Tuning effectively maintains model safety without sacrificing robust performance on downstream tasks. Our findings position LookAhead Tuning as a reliable and efficient solution for the safe and effective adaptation of LLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_19041 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LookAhead Tuning: Safer Language Models via Partial Answer Previews Liu, Kangwei Wang, Mengru Luo, Yujie Yuan, Lin Sun, Mengshu Liang, Lei Zhang, Zhiqiang Zhou, Jun Hooi, Bryan Deng, Shumin Computation and Language Artificial Intelligence Computer Vision and Pattern Recognition Machine Learning Multimedia Fine-tuning enables large language models (LLMs) to adapt to specific domains, but often compromises their previously established safety alignment. To mitigate the degradation of model safety during fine-tuning, we introduce LookAhead Tuning, a lightweight and effective data-driven approach that preserves safety during fine-tuning. The method introduces two simple strategies that modify training data by previewing partial answer prefixes, thereby minimizing perturbations to the model's initial token distributions and maintaining its built-in safety mechanisms. Comprehensive experiments demonstrate that LookAhead Tuning effectively maintains model safety without sacrificing robust performance on downstream tasks. Our findings position LookAhead Tuning as a reliable and efficient solution for the safe and effective adaptation of LLMs. |
| title | LookAhead Tuning: Safer Language Models via Partial Answer Previews |
| topic | Computation and Language Artificial Intelligence Computer Vision and Pattern Recognition Machine Learning Multimedia |
| url | https://arxiv.org/abs/2503.19041 |