Unleashing the Power of Continual Learning on Non-Centralized Devices: A Survey
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909601924907008 |
|---|---|
| author | Li, Yichen Wang, Haozhao Xu, Wenchao Xiao, Tianzhe Liu, Hong Tu, Minzhu Wang, Yuying Yang, Xin Zhang, Rui Yu, Shui Guo, Song Li, Ruixuan |
| author_facet | Li, Yichen Wang, Haozhao Xu, Wenchao Xiao, Tianzhe Liu, Hong Tu, Minzhu Wang, Yuying Yang, Xin Zhang, Rui Yu, Shui Guo, Song Li, Ruixuan |
| contents | Non-Centralized Continual Learning (NCCL) has become an emerging paradigm for enabling distributed devices such as vehicles and servers to handle streaming data from a joint non-stationary environment. To achieve high reliability and scalability in deploying this paradigm in distributed systems, it is essential to conquer challenges stemming from both spatial and temporal dimensions, manifesting as distribution shifts, catastrophic forgetting, heterogeneity, and privacy issues. This survey focuses on a comprehensive examination of the development of the non-centralized continual learning algorithms and the real-world deployment across distributed devices. We begin with an introduction to the background and fundamentals of non-centralized learning and continual learning. Then, we review existing solutions from three levels to represent how existing techniques alleviate the catastrophic forgetting and distribution shift. Additionally, we delve into the various types of heterogeneity issues, security, and privacy attributes, as well as real-world applications across three prevalent scenarios. Furthermore, we establish a large-scale benchmark to revisit this problem and analyze the performance of the state-of-the-art NCCL approaches. Finally, we discuss the important challenges and future research directions in NCCL. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_13840 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Unleashing the Power of Continual Learning on Non-Centralized Devices: A Survey Li, Yichen Wang, Haozhao Xu, Wenchao Xiao, Tianzhe Liu, Hong Tu, Minzhu Wang, Yuying Yang, Xin Zhang, Rui Yu, Shui Guo, Song Li, Ruixuan Machine Learning Distributed, Parallel, and Cluster Computing Non-Centralized Continual Learning (NCCL) has become an emerging paradigm for enabling distributed devices such as vehicles and servers to handle streaming data from a joint non-stationary environment. To achieve high reliability and scalability in deploying this paradigm in distributed systems, it is essential to conquer challenges stemming from both spatial and temporal dimensions, manifesting as distribution shifts, catastrophic forgetting, heterogeneity, and privacy issues. This survey focuses on a comprehensive examination of the development of the non-centralized continual learning algorithms and the real-world deployment across distributed devices. We begin with an introduction to the background and fundamentals of non-centralized learning and continual learning. Then, we review existing solutions from three levels to represent how existing techniques alleviate the catastrophic forgetting and distribution shift. Additionally, we delve into the various types of heterogeneity issues, security, and privacy attributes, as well as real-world applications across three prevalent scenarios. Furthermore, we establish a large-scale benchmark to revisit this problem and analyze the performance of the state-of-the-art NCCL approaches. Finally, we discuss the important challenges and future research directions in NCCL. |
| title | Unleashing the Power of Continual Learning on Non-Centralized Devices: A Survey |
| topic | Machine Learning Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2412.13840 |