Understanding and Alleviating Memory Consumption in RLHF for LLMs
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910658069528576 |
|---|---|
| author | Zhou, Jin Yang, Hanmei Steven Tang Xiang, Mingcan Guan, Hui Liu, Tongping |
| author_facet | Zhou, Jin Yang, Hanmei Steven Tang Xiang, Mingcan Guan, Hui Liu, Tongping |
| contents | Fine-tuning with Reinforcement Learning with Human Feedback (RLHF) is essential for aligning large language models (LLMs). However, RLHF often encounters significant memory challenges. This study is the first to examine memory usage in the RLHF context, exploring various memory management strategies and unveiling the reasons behind excessive memory consumption. Additionally, we introduce a simple yet effective approach that substantially reduces the memory required for RLHF fine-tuning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_15651 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Understanding and Alleviating Memory Consumption in RLHF for LLMs Zhou, Jin Yang, Hanmei Steven Tang Xiang, Mingcan Guan, Hui Liu, Tongping Machine Learning Fine-tuning with Reinforcement Learning with Human Feedback (RLHF) is essential for aligning large language models (LLMs). However, RLHF often encounters significant memory challenges. This study is the first to examine memory usage in the RLHF context, exploring various memory management strategies and unveiling the reasons behind excessive memory consumption. Additionally, we introduce a simple yet effective approach that substantially reduces the memory required for RLHF fine-tuning. |
| title | Understanding and Alleviating Memory Consumption in RLHF for LLMs |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2410.15651 |