Low-resource domain adaptation while minimizing energy and hardware resource consumption
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912424210202624 |
|---|---|
| author | Maina, Hernán Wolovick, Nicolás Benotti, Luciana |
| author_facet | Maina, Hernán Wolovick, Nicolás Benotti, Luciana |
| contents | Training Large Language Models (LLMs) is costly in terms of energy, hardware, and annotated data, often resulting in a positionality rooted in predominant cultures and values (Santy et al., 2023). Domain adaptation has emerged as a promising strategy to better align models with diverse cultural and value contexts (Hershcovich et al., 2022), but its computational cost remains a significant barrier, particularly for research groups lacking access to large-scale infrastructure. In this paper, we evaluate how the use of different numerical precision formats and data parallelization strategies impacts both training speed (as a proxy to energy and hardware consumption) and model accuracy, with the goal of facilitating domain adaptation in low-resource environments. Our findings are relevant to any setting where energy efficiency, accessibility, or limited hardware availability are key concerns. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_08433 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Low-resource domain adaptation while minimizing energy and hardware resource consumption Maina, Hernán Wolovick, Nicolás Benotti, Luciana Computation and Language Distributed, Parallel, and Cluster Computing Machine Learning Training Large Language Models (LLMs) is costly in terms of energy, hardware, and annotated data, often resulting in a positionality rooted in predominant cultures and values (Santy et al., 2023). Domain adaptation has emerged as a promising strategy to better align models with diverse cultural and value contexts (Hershcovich et al., 2022), but its computational cost remains a significant barrier, particularly for research groups lacking access to large-scale infrastructure. In this paper, we evaluate how the use of different numerical precision formats and data parallelization strategies impacts both training speed (as a proxy to energy and hardware consumption) and model accuracy, with the goal of facilitating domain adaptation in low-resource environments. Our findings are relevant to any setting where energy efficiency, accessibility, or limited hardware availability are key concerns. |
| title | Low-resource domain adaptation while minimizing energy and hardware resource consumption |
| topic | Computation and Language Distributed, Parallel, and Cluster Computing Machine Learning |
| url | https://arxiv.org/abs/2506.08433 |