Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2411.14759 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917844934983680 |
|---|---|
| author | Lu, Kai Zhao, Siqi Wan, Jiguang |
| author_facet | Lu, Kai Zhao, Siqi Wan, Jiguang |
| contents | Efficient management of storage resources in big data and cloud computing environments requires accurate identification of data's "cold" and "hot" states. Traditional methods, such as rule-based algorithms and early AI techniques, often struggle with dynamic workloads, leading to low accuracy, poor adaptability, and high operational overhead. To address these issues, we propose a novel solution based on online learning strategies. Our approach dynamically adapts to changing data access patterns, achieving higher accuracy and lower operational costs. Rigorous testing with both synthetic and real-world datasets demonstrates a significant improvement, achieving a 90% accuracy rate in hot-cold classification. Additionally, the computational and storage overheads are considerably reduced. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_14759 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Hammer: Towards Efficient Hot-Cold Data Identification via Online Learning Lu, Kai Zhao, Siqi Wan, Jiguang Machine Learning Artificial Intelligence Efficient management of storage resources in big data and cloud computing environments requires accurate identification of data's "cold" and "hot" states. Traditional methods, such as rule-based algorithms and early AI techniques, often struggle with dynamic workloads, leading to low accuracy, poor adaptability, and high operational overhead. To address these issues, we propose a novel solution based on online learning strategies. Our approach dynamically adapts to changing data access patterns, achieving higher accuracy and lower operational costs. Rigorous testing with both synthetic and real-world datasets demonstrates a significant improvement, achieving a 90% accuracy rate in hot-cold classification. Additionally, the computational and storage overheads are considerably reduced. |
| title | Hammer: Towards Efficient Hot-Cold Data Identification via Online Learning |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2411.14759 |