Alignment at Pre-training! Towards Native Alignment for Arabic LLMs
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913597244833792 |
|---|---|
| author | Liang, Juhao Cai, Zhenyang Zhu, Jianqing Huang, Huang Zong, Kewei An, Bang Alharthi, Mosen He, Juncai Zhang, Lian Li, Haizhou Wang, Benyou Xu, Jinchao |
| author_facet | Liang, Juhao Cai, Zhenyang Zhu, Jianqing Huang, Huang Zong, Kewei An, Bang Alharthi, Mosen He, Juncai Zhang, Lian Li, Haizhou Wang, Benyou Xu, Jinchao |
| contents | The alignment of large language models (LLMs) is critical for developing effective and safe language models. Traditional approaches focus on aligning models during the instruction tuning or reinforcement learning stages, referred to in this paper as `post alignment'. We argue that alignment during the pre-training phase, which we term `native alignment', warrants investigation. Native alignment aims to prevent unaligned content from the beginning, rather than relying on post-hoc processing. This approach leverages extensively aligned pre-training data to enhance the effectiveness and usability of pre-trained models. Our study specifically explores the application of native alignment in the context of Arabic LLMs. We conduct comprehensive experiments and ablation studies to evaluate the impact of native alignment on model performance and alignment stability. Additionally, we release open-source Arabic LLMs that demonstrate state-of-the-art performance on various benchmarks, providing significant benefits to the Arabic LLM community. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_03253 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Alignment at Pre-training! Towards Native Alignment for Arabic LLMs Liang, Juhao Cai, Zhenyang Zhu, Jianqing Huang, Huang Zong, Kewei An, Bang Alharthi, Mosen He, Juncai Zhang, Lian Li, Haizhou Wang, Benyou Xu, Jinchao Computation and Language The alignment of large language models (LLMs) is critical for developing effective and safe language models. Traditional approaches focus on aligning models during the instruction tuning or reinforcement learning stages, referred to in this paper as `post alignment'. We argue that alignment during the pre-training phase, which we term `native alignment', warrants investigation. Native alignment aims to prevent unaligned content from the beginning, rather than relying on post-hoc processing. This approach leverages extensively aligned pre-training data to enhance the effectiveness and usability of pre-trained models. Our study specifically explores the application of native alignment in the context of Arabic LLMs. We conduct comprehensive experiments and ablation studies to evaluate the impact of native alignment on model performance and alignment stability. Additionally, we release open-source Arabic LLMs that demonstrate state-of-the-art performance on various benchmarks, providing significant benefits to the Arabic LLM community. |
| title | Alignment at Pre-training! Towards Native Alignment for Arabic LLMs |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2412.03253 |