Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2501.17164 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912208333570048 |
|---|---|
| author | Li, Zuguang Wu, Wen Wu, Shaohua Lin, Qiaohua Sun, Yaping Wang, Hui |
| author_facet | Li, Zuguang Wu, Wen Wu, Shaohua Lin, Qiaohua Sun, Yaping Wang, Hui |
| contents | Large models (LMs) have immense potential in Internet of Things (IoT) systems, enabling applications such as intelligent voice assistants, predictive maintenance, and healthcare monitoring. However, training LMs on edge servers raises data privacy concerns, while deploying them directly on IoT devices is constrained by limited computational and memory resources. We analyze the key challenges of training LMs in IoT systems, including energy constraints, latency requirements, and device heterogeneity, and propose potential solutions such as dynamic resource management, adaptive model partitioning, and clustered collaborative training. Furthermore, we propose a split knowledge distillation framework to efficiently distill LMs into smaller, deployable versions for IoT devices while ensuring raw data remains local. This framework integrates knowledge distillation and split learning to minimize energy consumption and meet low model training delay requirements. A case study is presented to evaluate the feasibility and performance of the proposed framework. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_17164 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Split Knowledge Distillation for Large Models in IoT: Architecture, Challenges, and Solutions Li, Zuguang Wu, Wen Wu, Shaohua Lin, Qiaohua Sun, Yaping Wang, Hui Machine Learning Artificial Intelligence Large models (LMs) have immense potential in Internet of Things (IoT) systems, enabling applications such as intelligent voice assistants, predictive maintenance, and healthcare monitoring. However, training LMs on edge servers raises data privacy concerns, while deploying them directly on IoT devices is constrained by limited computational and memory resources. We analyze the key challenges of training LMs in IoT systems, including energy constraints, latency requirements, and device heterogeneity, and propose potential solutions such as dynamic resource management, adaptive model partitioning, and clustered collaborative training. Furthermore, we propose a split knowledge distillation framework to efficiently distill LMs into smaller, deployable versions for IoT devices while ensuring raw data remains local. This framework integrates knowledge distillation and split learning to minimize energy consumption and meet low model training delay requirements. A case study is presented to evaluate the feasibility and performance of the proposed framework. |
| title | Split Knowledge Distillation for Large Models in IoT: Architecture, Challenges, and Solutions |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2501.17164 |