Robustness in Wireless Distributed Learning: An Information-Theoretic Analysis
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911043096150016 |
|---|---|
| author | He, Yangshuo Yu, Guanding Dai, Huaiyu |
| author_facet | He, Yangshuo Yu, Guanding Dai, Huaiyu |
| contents | In recent years, the application of artificial intelligence (AI) in wireless communications has demonstrated inherent robustness against wireless channel distortions. Most existing works empirically leverage this robustness to yield considerable performance gains through AI architectural designs. However, there is a lack of direct theoretical analysis of this robustness and its potential to enhance communication efficiency, which restricts the full exploitation of these advantages. In this paper, we adopt an information-theoretic approach to evaluate the robustness in wireless distributed learning by deriving an upper bound on the task performance loss due to imperfect wireless channels. Utilizing this insight, we define task outage probability and characterize the maximum transmission rate under task accuracy guarantees, referred to as the task-aware $ε$-capacity resulting from the robustness. To achieve the utility of the theoretical results in practical settings, we present an efficient algorithm for the approximation of the upper bound. Subsequently, we devise a robust training framework that optimizes the trade-off between robustness and task accuracy, enhancing the robustness against channel distortions. Extensive experiments validate the effectiveness of the proposed upper bound and task-aware $ε$-capacity and demonstrate that the proposed robust training framework achieves high robustness, thus ensuring a high transmission rate while maintaining inference performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_17577 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Robustness in Wireless Distributed Learning: An Information-Theoretic Analysis He, Yangshuo Yu, Guanding Dai, Huaiyu Information Theory Signal Processing In recent years, the application of artificial intelligence (AI) in wireless communications has demonstrated inherent robustness against wireless channel distortions. Most existing works empirically leverage this robustness to yield considerable performance gains through AI architectural designs. However, there is a lack of direct theoretical analysis of this robustness and its potential to enhance communication efficiency, which restricts the full exploitation of these advantages. In this paper, we adopt an information-theoretic approach to evaluate the robustness in wireless distributed learning by deriving an upper bound on the task performance loss due to imperfect wireless channels. Utilizing this insight, we define task outage probability and characterize the maximum transmission rate under task accuracy guarantees, referred to as the task-aware $ε$-capacity resulting from the robustness. To achieve the utility of the theoretical results in practical settings, we present an efficient algorithm for the approximation of the upper bound. Subsequently, we devise a robust training framework that optimizes the trade-off between robustness and task accuracy, enhancing the robustness against channel distortions. Extensive experiments validate the effectiveness of the proposed upper bound and task-aware $ε$-capacity and demonstrate that the proposed robust training framework achieves high robustness, thus ensuring a high transmission rate while maintaining inference performance. |
| title | Robustness in Wireless Distributed Learning: An Information-Theoretic Analysis |
| topic | Information Theory Signal Processing |
| url | https://arxiv.org/abs/2401.17577 |