Unveiling Environmental Impacts of Large Language Model Serving: A Functional Unit View
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908383224791040 |
|---|---|
| author | Wu, Yanran Hua, Inez Ding, Yi |
| author_facet | Wu, Yanran Hua, Inez Ding, Yi |
| contents | Large language models (LLMs) offer powerful capabilities but come with significant environmental impact, particularly in carbon emissions. Existing studies benchmark carbon emissions but lack a standardized basis for comparison across different model configurations. To address this, we introduce the concept of functional unit (FU) as a standardized basis and develop FUEL, the first FU-based framework for evaluating LLM serving's environmental impact. Through three case studies, we uncover key insights and trade-offs in reducing carbon emissions by optimizing model size, quantization strategy, and hardware choice, paving the way for more sustainable LLM serving. The code is available at https://github.com/jojacola/FUEL. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_11256 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Unveiling Environmental Impacts of Large Language Model Serving: A Functional Unit View Wu, Yanran Hua, Inez Ding, Yi Machine Learning Hardware Architecture Computation and Language Large language models (LLMs) offer powerful capabilities but come with significant environmental impact, particularly in carbon emissions. Existing studies benchmark carbon emissions but lack a standardized basis for comparison across different model configurations. To address this, we introduce the concept of functional unit (FU) as a standardized basis and develop FUEL, the first FU-based framework for evaluating LLM serving's environmental impact. Through three case studies, we uncover key insights and trade-offs in reducing carbon emissions by optimizing model size, quantization strategy, and hardware choice, paving the way for more sustainable LLM serving. The code is available at https://github.com/jojacola/FUEL. |
| title | Unveiling Environmental Impacts of Large Language Model Serving: A Functional Unit View |
| topic | Machine Learning Hardware Architecture Computation and Language |
| url | https://arxiv.org/abs/2502.11256 |