CORE: Compensable Reward as a Catalyst for Improving Offline RL in Wireless Networks
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915691334991872 |
|---|---|
| author | Zu, Lipeng Zhou, Hansong Qian, Yu Chakraborty, Shayok Yuan, Yukun Guo, Linke Zhang, Xiaonan |
| author_facet | Zu, Lipeng Zhou, Hansong Qian, Yu Chakraborty, Shayok Yuan, Yukun Guo, Linke Zhang, Xiaonan |
| contents | Real-world wireless data are expensive to collect and often lack sufficient expert demonstrations, causing existing offline RL methods to overfit suboptimal behaviors and exhibit unstable performance. To address this issue, we propose CORE, an offline RL framework specifically designed for wireless environments. CORE identifies latent expert trajectories from noisy datasets via behavior embedding clustering, and trains a conditional variational autoencoder with a contrastive objective to separate expert and non-expert behaviors in latent space. Based on the learned representations, CORE constructs compensable rewards that reflect expert-likelihood, effectively guiding policy learning under limited or imperfect supervision. More broadly, this work represents one of the early systematic explorations of offline RL in wireless networking, where prior adoption remains limited. Beyond introducing offline RL techniques to this domain, we further examine intrinsic wireless data characteristics and develop a domain-aligned algorithm that explicitly accounts for their structural properties. While offline RL has not yet been fully established as a standard methodology in the wireless community, our study aims to provide foundational insights and empirical evidence to support its broader acceptance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_19671 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | CORE: Compensable Reward as a Catalyst for Improving Offline RL in Wireless Networks Zu, Lipeng Zhou, Hansong Qian, Yu Chakraborty, Shayok Yuan, Yukun Guo, Linke Zhang, Xiaonan Networking and Internet Architecture Real-world wireless data are expensive to collect and often lack sufficient expert demonstrations, causing existing offline RL methods to overfit suboptimal behaviors and exhibit unstable performance. To address this issue, we propose CORE, an offline RL framework specifically designed for wireless environments. CORE identifies latent expert trajectories from noisy datasets via behavior embedding clustering, and trains a conditional variational autoencoder with a contrastive objective to separate expert and non-expert behaviors in latent space. Based on the learned representations, CORE constructs compensable rewards that reflect expert-likelihood, effectively guiding policy learning under limited or imperfect supervision. More broadly, this work represents one of the early systematic explorations of offline RL in wireless networking, where prior adoption remains limited. Beyond introducing offline RL techniques to this domain, we further examine intrinsic wireless data characteristics and develop a domain-aligned algorithm that explicitly accounts for their structural properties. While offline RL has not yet been fully established as a standard methodology in the wireless community, our study aims to provide foundational insights and empirical evidence to support its broader acceptance. |
| title | CORE: Compensable Reward as a Catalyst for Improving Offline RL in Wireless Networks |
| topic | Networking and Internet Architecture |
| url | https://arxiv.org/abs/2512.19671 |