Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2507.19672 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908468341899264 |
|---|---|
| author | Lu, Haoran Fang, Luyang Zhang, Ruidong Li, Xinliang Cai, Jiazhang Cheng, Huimin Tang, Lin Liu, Ziyu Sun, Zeliang Wang, Tao Zhang, Yingchuan Zidan, Arif Hassan Xu, Jinwen Yu, Jincheng Yu, Meizhi Jiang, Hanqi Gong, Xilin Luo, Weidi Sun, Bolun Chen, Yongkai Ma, Terry Wu, Shushan Zhou, Yifan Chen, Junhao Xiang, Haotian Zhang, Jing Jahin, Afrar Ruan, Wei Deng, Ke Pan, Yi Wang, Peilong Li, Jiahui Liu, Zhengliang Zhang, Lu Zhao, Lin Liu, Wei Zhu, Dajiang Xing, Xin Dou, Fei Zhang, Wei Huang, Chao Liu, Rongjie Zhang, Mengrui Liu, Yiwen Sun, Xiaoxiao Lu, Qin Xiang, Zhen Zhong, Wenxuan Liu, Tianming Ma, Ping |
| author_facet | Lu, Haoran Fang, Luyang Zhang, Ruidong Li, Xinliang Cai, Jiazhang Cheng, Huimin Tang, Lin Liu, Ziyu Sun, Zeliang Wang, Tao Zhang, Yingchuan Zidan, Arif Hassan Xu, Jinwen Yu, Jincheng Yu, Meizhi Jiang, Hanqi Gong, Xilin Luo, Weidi Sun, Bolun Chen, Yongkai Ma, Terry Wu, Shushan Zhou, Yifan Chen, Junhao Xiang, Haotian Zhang, Jing Jahin, Afrar Ruan, Wei Deng, Ke Pan, Yi Wang, Peilong Li, Jiahui Liu, Zhengliang Zhang, Lu Zhao, Lin Liu, Wei Zhu, Dajiang Xing, Xin Dou, Fei Zhang, Wei Huang, Chao Liu, Rongjie Zhang, Mengrui Liu, Yiwen Sun, Xiaoxiao Lu, Qin Xiang, Zhen Zhong, Wenxuan Liu, Tianming Ma, Ping |
| contents | Due to the remarkable capabilities and growing impact of large language models (LLMs), they have been deeply integrated into many aspects of society. Thus, ensuring their alignment with human values and intentions has emerged as a critical challenge. This survey provides a comprehensive overview of practical alignment techniques, training protocols, and empirical findings in LLM alignment. We analyze the development of alignment methods across diverse paradigms, characterizing the fundamental trade-offs between core alignment objectives. Our analysis shows that while supervised fine-tuning enables basic instruction-following, preference-based methods offer more flexibility for aligning with nuanced human intent. We discuss state-of-the-art techniques, including Direct Preference Optimization (DPO), Constitutional AI, brain-inspired methods, and alignment uncertainty quantification (AUQ), highlighting their approaches to balancing quality and efficiency. We review existing evaluation frameworks and benchmarking datasets, emphasizing limitations such as reward misspecification, distributional robustness, and scalable oversight. We summarize strategies adopted by leading AI labs to illustrate the current state of practice. We conclude by outlining open problems in oversight, value pluralism, robustness, and continuous alignment. This survey aims to inform both researchers and practitioners navigating the evolving landscape of LLM alignment. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_19672 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challenges Lu, Haoran Fang, Luyang Zhang, Ruidong Li, Xinliang Cai, Jiazhang Cheng, Huimin Tang, Lin Liu, Ziyu Sun, Zeliang Wang, Tao Zhang, Yingchuan Zidan, Arif Hassan Xu, Jinwen Yu, Jincheng Yu, Meizhi Jiang, Hanqi Gong, Xilin Luo, Weidi Sun, Bolun Chen, Yongkai Ma, Terry Wu, Shushan Zhou, Yifan Chen, Junhao Xiang, Haotian Zhang, Jing Jahin, Afrar Ruan, Wei Deng, Ke Pan, Yi Wang, Peilong Li, Jiahui Liu, Zhengliang Zhang, Lu Zhao, Lin Liu, Wei Zhu, Dajiang Xing, Xin Dou, Fei Zhang, Wei Huang, Chao Liu, Rongjie Zhang, Mengrui Liu, Yiwen Sun, Xiaoxiao Lu, Qin Xiang, Zhen Zhong, Wenxuan Liu, Tianming Ma, Ping Artificial Intelligence Machine Learning Due to the remarkable capabilities and growing impact of large language models (LLMs), they have been deeply integrated into many aspects of society. Thus, ensuring their alignment with human values and intentions has emerged as a critical challenge. This survey provides a comprehensive overview of practical alignment techniques, training protocols, and empirical findings in LLM alignment. We analyze the development of alignment methods across diverse paradigms, characterizing the fundamental trade-offs between core alignment objectives. Our analysis shows that while supervised fine-tuning enables basic instruction-following, preference-based methods offer more flexibility for aligning with nuanced human intent. We discuss state-of-the-art techniques, including Direct Preference Optimization (DPO), Constitutional AI, brain-inspired methods, and alignment uncertainty quantification (AUQ), highlighting their approaches to balancing quality and efficiency. We review existing evaluation frameworks and benchmarking datasets, emphasizing limitations such as reward misspecification, distributional robustness, and scalable oversight. We summarize strategies adopted by leading AI labs to illustrate the current state of practice. We conclude by outlining open problems in oversight, value pluralism, robustness, and continuous alignment. This survey aims to inform both researchers and practitioners navigating the evolving landscape of LLM alignment. |
| title | Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challenges |
| topic | Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2507.19672 |