Toward Sustainable GenAI using Generation Directives for Carbon-Friendly Large Language Model Inference
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913272400183296 |
|---|---|
| author | Li, Baolin Jiang, Yankai Gadepally, Vijay Tiwari, Devesh |
| author_facet | Li, Baolin Jiang, Yankai Gadepally, Vijay Tiwari, Devesh |
| contents | The rapid advancement of Generative Artificial Intelligence (GenAI) across diverse sectors raises significant environmental concerns, notably the carbon emissions from their cloud and high performance computing (HPC) infrastructure. This paper presents Sprout, an innovative framework designed to address these concerns by reducing the carbon footprint of generative Large Language Model (LLM) inference services. Sprout leverages the innovative concept of "generation directives" to guide the autoregressive generation process, thereby enhancing carbon efficiency. Our proposed method meticulously balances the need for ecological sustainability with the demand for high-quality generation outcomes. Employing a directive optimizer for the strategic assignment of generation directives to user prompts and an original offline quality evaluator, Sprout demonstrates a significant reduction in carbon emissions by over 40% in real-world evaluations using the Llama2 LLM and global electricity grid data. This research marks a critical step toward aligning AI technology with sustainable practices, highlighting the potential for mitigating environmental impacts in the rapidly expanding domain of generative artificial intelligence. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_12900 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Toward Sustainable GenAI using Generation Directives for Carbon-Friendly Large Language Model Inference Li, Baolin Jiang, Yankai Gadepally, Vijay Tiwari, Devesh Distributed, Parallel, and Cluster Computing Artificial Intelligence Computation and Language Machine Learning The rapid advancement of Generative Artificial Intelligence (GenAI) across diverse sectors raises significant environmental concerns, notably the carbon emissions from their cloud and high performance computing (HPC) infrastructure. This paper presents Sprout, an innovative framework designed to address these concerns by reducing the carbon footprint of generative Large Language Model (LLM) inference services. Sprout leverages the innovative concept of "generation directives" to guide the autoregressive generation process, thereby enhancing carbon efficiency. Our proposed method meticulously balances the need for ecological sustainability with the demand for high-quality generation outcomes. Employing a directive optimizer for the strategic assignment of generation directives to user prompts and an original offline quality evaluator, Sprout demonstrates a significant reduction in carbon emissions by over 40% in real-world evaluations using the Llama2 LLM and global electricity grid data. This research marks a critical step toward aligning AI technology with sustainable practices, highlighting the potential for mitigating environmental impacts in the rapidly expanding domain of generative artificial intelligence. |
| title | Toward Sustainable GenAI using Generation Directives for Carbon-Friendly Large Language Model Inference |
| topic | Distributed, Parallel, and Cluster Computing Artificial Intelligence Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2403.12900 |