FedProphet: Memory-Efficient Federated Adversarial Training via Robust and Consistent Cascade Learning
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910911364595712 |
|---|---|
| author | Tang, Minxue Wang, Yitu Zhang, Jingyang DiValentin, Louis Ding, Aolin Hass, Amin Chen, Yiran Li, Hai "Helen" |
| author_facet | Tang, Minxue Wang, Yitu Zhang, Jingyang DiValentin, Louis Ding, Aolin Hass, Amin Chen, Yiran Li, Hai "Helen" |
| contents | Federated Adversarial Training (FAT) can supplement robustness against adversarial examples to Federated Learning (FL), promoting a meaningful step toward trustworthy AI. However, FAT requires large models to preserve high accuracy while achieving strong robustness, incurring high memory-swapping latency when training on memory-constrained edge devices. Existing memory-efficient FL methods suffer from poor accuracy and weak robustness due to inconsistent local and global models. In this paper, we propose FedProphet, a novel FAT framework that can achieve memory efficiency, robustness, and consistency simultaneously. FedProphget reduces the memory requirement in local training while guaranteeing adversarial robustness by adversarial cascade learning with strong convexity regularization, and we show that the strong robustness also implies low inconsistency in FedProphet. We also develop a training coordinator on the server of FL, with Adaptive Perturbation Adjustment for utility-robustness balance and Differentiated Module Assignment for objective inconsistency mitigation. FedPeophet significantly outperforms other baselines under different experimental settings, maintaining the accuracy and robustness of end-to-end FAT with 80% memory reduction and up to 10.8x speedup in training time. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_08372 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | FedProphet: Memory-Efficient Federated Adversarial Training via Robust and Consistent Cascade Learning Tang, Minxue Wang, Yitu Zhang, Jingyang DiValentin, Louis Ding, Aolin Hass, Amin Chen, Yiran Li, Hai "Helen" Machine Learning Artificial Intelligence Federated Adversarial Training (FAT) can supplement robustness against adversarial examples to Federated Learning (FL), promoting a meaningful step toward trustworthy AI. However, FAT requires large models to preserve high accuracy while achieving strong robustness, incurring high memory-swapping latency when training on memory-constrained edge devices. Existing memory-efficient FL methods suffer from poor accuracy and weak robustness due to inconsistent local and global models. In this paper, we propose FedProphet, a novel FAT framework that can achieve memory efficiency, robustness, and consistency simultaneously. FedProphget reduces the memory requirement in local training while guaranteeing adversarial robustness by adversarial cascade learning with strong convexity regularization, and we show that the strong robustness also implies low inconsistency in FedProphet. We also develop a training coordinator on the server of FL, with Adaptive Perturbation Adjustment for utility-robustness balance and Differentiated Module Assignment for objective inconsistency mitigation. FedPeophet significantly outperforms other baselines under different experimental settings, maintaining the accuracy and robustness of end-to-end FAT with 80% memory reduction and up to 10.8x speedup in training time. |
| title | FedProphet: Memory-Efficient Federated Adversarial Training via Robust and Consistent Cascade Learning |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2409.08372 |