ABC Align: Large Language Model Alignment for Safety & Accuracy
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917738548559872 |
|---|---|
| author | Seneque, Gareth Ho, Lap-Hang Kuperman, Ariel Saeedi, Nafise Erfanian Molendijk, Jeffrey |
| author_facet | Seneque, Gareth Ho, Lap-Hang Kuperman, Ariel Saeedi, Nafise Erfanian Molendijk, Jeffrey |
| contents | Alignment of Large Language Models (LLMs) remains an unsolved problem. Human preferences are highly distributed and can be captured at multiple levels of abstraction, from the individual to diverse populations. Organisational preferences, represented by standards and principles, are defined to mitigate reputational risk or meet legislative obligations. In this paper, we present ABC Align, a novel alignment methodology for LLMs that enables integration of the standards and preferences of a large media organisation into the LLM itself. We combine a set of data and methods that build on recent breakthroughs in synthetic data generation, preference optimisation, and post-training model quantisation. Our unified approach mitigates bias and improves accuracy, while preserving reasoning capability, as measured against standard benchmarks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_00307 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | ABC Align: Large Language Model Alignment for Safety & Accuracy Seneque, Gareth Ho, Lap-Hang Kuperman, Ariel Saeedi, Nafise Erfanian Molendijk, Jeffrey Machine Learning Artificial Intelligence Computation and Language 68T50 I.2.7 Alignment of Large Language Models (LLMs) remains an unsolved problem. Human preferences are highly distributed and can be captured at multiple levels of abstraction, from the individual to diverse populations. Organisational preferences, represented by standards and principles, are defined to mitigate reputational risk or meet legislative obligations. In this paper, we present ABC Align, a novel alignment methodology for LLMs that enables integration of the standards and preferences of a large media organisation into the LLM itself. We combine a set of data and methods that build on recent breakthroughs in synthetic data generation, preference optimisation, and post-training model quantisation. Our unified approach mitigates bias and improves accuracy, while preserving reasoning capability, as measured against standard benchmarks. |
| title | ABC Align: Large Language Model Alignment for Safety & Accuracy |
| topic | Machine Learning Artificial Intelligence Computation and Language 68T50 I.2.7 |
| url | https://arxiv.org/abs/2408.00307 |