Debiased Model-based Representations for Sample-efficient Continuous Control
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909035377197056 |
|---|---|
| author | Lyu, Jiafei Lin, Zichuan Fujimoto, Scott Yang, Kai Chen, Yangkun Yang, Saiyong Lu, Zongqing Ye, Deheng |
| author_facet | Lyu, Jiafei Lin, Zichuan Fujimoto, Scott Yang, Kai Chen, Yangkun Yang, Saiyong Lu, Zongqing Ye, Deheng |
| contents | Model-based representations recently stand out as a promising framework that embeds latent dynamics information into the representations for downstream off-policy actor-critic learning. It implicitly combines the advantages of both model-free and model-based approaches while avoiding the training costs associated with model-based methods. Nevertheless, existing model-based representation methods can fail to capture sufficient information about relevant variables and can overfit to early experiences in the replay buffer. These incur biases in representation and actor-critic learning, leading to inferior performance. To address this, we propose Debiased model-based Representations for Q-learning, tagged DR.Q algorithm. DR.Q explicitly maximizes the mutual information between the representations of the current state-action pair and the next state besides minimizing their deviations, and samples transitions with faded prioritized experience replay. We evaluate DR.Q on numerous continuous control benchmarks with a single set of hyperparameters, and the results demonstrate that DR.Q can match or surpass recent strong baselines, sometimes outperforming them by a large margin. Our code is available at https://github.com/dmksjfl/DR.Q. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_11711 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Debiased Model-based Representations for Sample-efficient Continuous Control Lyu, Jiafei Lin, Zichuan Fujimoto, Scott Yang, Kai Chen, Yangkun Yang, Saiyong Lu, Zongqing Ye, Deheng Machine Learning Artificial Intelligence Model-based representations recently stand out as a promising framework that embeds latent dynamics information into the representations for downstream off-policy actor-critic learning. It implicitly combines the advantages of both model-free and model-based approaches while avoiding the training costs associated with model-based methods. Nevertheless, existing model-based representation methods can fail to capture sufficient information about relevant variables and can overfit to early experiences in the replay buffer. These incur biases in representation and actor-critic learning, leading to inferior performance. To address this, we propose Debiased model-based Representations for Q-learning, tagged DR.Q algorithm. DR.Q explicitly maximizes the mutual information between the representations of the current state-action pair and the next state besides minimizing their deviations, and samples transitions with faded prioritized experience replay. We evaluate DR.Q on numerous continuous control benchmarks with a single set of hyperparameters, and the results demonstrate that DR.Q can match or surpass recent strong baselines, sometimes outperforming them by a large margin. Our code is available at https://github.com/dmksjfl/DR.Q. |
| title | Debiased Model-based Representations for Sample-efficient Continuous Control |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2605.11711 |