Boosting deep Reinforcement Learning using pretraining with Logical Options
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917319053148160 |
|---|---|
| author | Ye, Zihan Chau, Phil Emunds, Raban Blüml, Jannis Derstroff, Cedric Delfosse, Quentin Arenz, Oleg Kersting, Kristian |
| author_facet | Ye, Zihan Chau, Phil Emunds, Raban Blüml, Jannis Derstroff, Cedric Delfosse, Quentin Arenz, Oleg Kersting, Kristian |
| contents | Deep reinforcement learning agents are often misaligned, as they over-exploit early reward signals. Recently, several symbolic approaches have addressed these challenges by encoding sparse objectives along with aligned plans. However, purely symbolic architectures are complex to scale and difficult to apply to continuous settings. Hence, we propose a hybrid approach, inspired by humans' ability to acquire new skills. We use a two-stage framework that injects symbolic structure into neural-based reinforcement learning agents without sacrificing the expressivity of deep policies. Our method, called Hybrid Hierarchical RL (H^2RL), introduces a logical option-based pretraining strategy to steer the learning policy away from short-term reward loops and toward goal-directed behavior while allowing the final policy to be refined via standard environment interaction. Empirically, we show that this approach consistently improves long-horizon decision-making and yields agents that outperform strong neural, symbolic, and neuro-symbolic baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_06565 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Boosting deep Reinforcement Learning using pretraining with Logical Options Ye, Zihan Chau, Phil Emunds, Raban Blüml, Jannis Derstroff, Cedric Delfosse, Quentin Arenz, Oleg Kersting, Kristian Artificial Intelligence Machine Learning Deep reinforcement learning agents are often misaligned, as they over-exploit early reward signals. Recently, several symbolic approaches have addressed these challenges by encoding sparse objectives along with aligned plans. However, purely symbolic architectures are complex to scale and difficult to apply to continuous settings. Hence, we propose a hybrid approach, inspired by humans' ability to acquire new skills. We use a two-stage framework that injects symbolic structure into neural-based reinforcement learning agents without sacrificing the expressivity of deep policies. Our method, called Hybrid Hierarchical RL (H^2RL), introduces a logical option-based pretraining strategy to steer the learning policy away from short-term reward loops and toward goal-directed behavior while allowing the final policy to be refined via standard environment interaction. Empirically, we show that this approach consistently improves long-horizon decision-making and yields agents that outperform strong neural, symbolic, and neuro-symbolic baselines. |
| title | Boosting deep Reinforcement Learning using pretraining with Logical Options |
| topic | Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2603.06565 |