Energy Consumption of Dataframe Libraries for End-to-End Deep Learning Pipelines:A Comparative Analysis
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917475999809536 |
|---|---|
| author | Kumar, Punit Imran, Asif Kosar, Tevfik |
| author_facet | Kumar, Punit Imran, Asif Kosar, Tevfik |
| contents | This paper presents a detailed comparative analysis of the performance of three major Python data manipulation libraries - Pandas, Polars, and Dask - specifically when embedded within complete deep learning (DL) training and inference pipelines. The research bridges a gap in existing literature by studying how these libraries interact with substantial GPU workloads during critical phases like data loading, preprocessing, and batch feeding. The authors measured key performance indicators including runtime, memory usage, disk usage, and energy consumption (both CPU and GPU) across various machine learning models and datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_08644 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Energy Consumption of Dataframe Libraries for End-to-End Deep Learning Pipelines:A Comparative Analysis Kumar, Punit Imran, Asif Kosar, Tevfik Software Engineering Artificial Intelligence Performance This paper presents a detailed comparative analysis of the performance of three major Python data manipulation libraries - Pandas, Polars, and Dask - specifically when embedded within complete deep learning (DL) training and inference pipelines. The research bridges a gap in existing literature by studying how these libraries interact with substantial GPU workloads during critical phases like data loading, preprocessing, and batch feeding. The authors measured key performance indicators including runtime, memory usage, disk usage, and energy consumption (both CPU and GPU) across various machine learning models and datasets. |
| title | Energy Consumption of Dataframe Libraries for End-to-End Deep Learning Pipelines:A Comparative Analysis |
| topic | Software Engineering Artificial Intelligence Performance |
| url | https://arxiv.org/abs/2511.08644 |