From Tarzan to Tolkien: Controlling the Language Proficiency Level of LLMs for Content Generation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917685141438464 |
|---|---|
| author | Malik, Ali Mayhew, Stephen Piech, Chris Bicknell, Klinton |
| author_facet | Malik, Ali Mayhew, Stephen Piech, Chris Bicknell, Klinton |
| contents | We study the problem of controlling the difficulty level of text generated by Large Language Models (LLMs) for contexts where end-users are not fully proficient, such as language learners. Using a novel framework, we evaluate the effectiveness of several key approaches for this task, including few-shot prompting, supervised finetuning, and reinforcement learning (RL), utilising both GPT-4 and open source alternatives like LLama2-7B and Mistral-7B.
Our findings reveal a large performance gap between GPT-4 and the open source models when using prompt-based strategies. However, we show how to bridge this gap with a careful combination of finetuning and RL alignment. Our best model, CALM (CEFR-Aligned Language Model), surpasses the performance of GPT-4 and other strategies, at only a fraction of the cost. We further validate the quality of our results through a small-scale human study. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_03030 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | From Tarzan to Tolkien: Controlling the Language Proficiency Level of LLMs for Content Generation Malik, Ali Mayhew, Stephen Piech, Chris Bicknell, Klinton Computation and Language Machine Learning We study the problem of controlling the difficulty level of text generated by Large Language Models (LLMs) for contexts where end-users are not fully proficient, such as language learners. Using a novel framework, we evaluate the effectiveness of several key approaches for this task, including few-shot prompting, supervised finetuning, and reinforcement learning (RL), utilising both GPT-4 and open source alternatives like LLama2-7B and Mistral-7B. Our findings reveal a large performance gap between GPT-4 and the open source models when using prompt-based strategies. However, we show how to bridge this gap with a careful combination of finetuning and RL alignment. Our best model, CALM (CEFR-Aligned Language Model), surpasses the performance of GPT-4 and other strategies, at only a fraction of the cost. We further validate the quality of our results through a small-scale human study. |
| title | From Tarzan to Tolkien: Controlling the Language Proficiency Level of LLMs for Content Generation |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2406.03030 |