Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2511.08644 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917475999809536 |
|---|---|
| author | Kumar, Punit Imran, Asif Kosar, Tevfik |
| author_facet | Kumar, Punit Imran, Asif Kosar, Tevfik |
| contents | This paper presents a detailed comparative analysis of the performance of three major Python data manipulation libraries - Pandas, Polars, and Dask - specifically when embedded within complete deep learning (DL) training and inference pipelines. The research bridges a gap in existing literature by studying how these libraries interact with substantial GPU workloads during critical phases like data loading, preprocessing, and batch feeding. The authors measured key performance indicators including runtime, memory usage, disk usage, and energy consumption (both CPU and GPU) across various machine learning models and datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_08644 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Energy Consumption of Dataframe Libraries for End-to-End Deep Learning Pipelines:A Comparative Analysis Kumar, Punit Imran, Asif Kosar, Tevfik Software Engineering Artificial Intelligence Performance This paper presents a detailed comparative analysis of the performance of three major Python data manipulation libraries - Pandas, Polars, and Dask - specifically when embedded within complete deep learning (DL) training and inference pipelines. The research bridges a gap in existing literature by studying how these libraries interact with substantial GPU workloads during critical phases like data loading, preprocessing, and batch feeding. The authors measured key performance indicators including runtime, memory usage, disk usage, and energy consumption (both CPU and GPU) across various machine learning models and datasets. |
| title | Energy Consumption of Dataframe Libraries for End-to-End Deep Learning Pipelines:A Comparative Analysis |
| topic | Software Engineering Artificial Intelligence Performance |
| url | https://arxiv.org/abs/2511.08644 |