SkillTree: Explainable Skill-Based Deep Reinforcement Learning for Long-Horizon Control Tasks
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913153484324864 |
|---|---|
| author | Wen, Yongyan Li, Siyuan Zuo, Rongchang Yuan, Lei Mao, Hangyu Liu, Peng |
| author_facet | Wen, Yongyan Li, Siyuan Zuo, Rongchang Yuan, Lei Mao, Hangyu Liu, Peng |
| contents | Deep reinforcement learning (DRL) has achieved remarkable success in various research domains. However, its reliance on neural networks results in a lack of transparency, which limits its practical applications. To achieve explainability, decision trees have emerged as a popular and promising alternative to neural networks. Nonetheless, due to their limited expressiveness, traditional decision trees struggle with high-dimensional long-horizon continuous control tasks. In this paper, we proposes SkillTree, a novel framework that reduces complex continuous action spaces into discrete skill spaces. Our hierarchical approach integrates a differentiable decision tree within the high-level policy to generate skill embeddings, which subsequently guide the low-level policy in executing skills. By making skill decisions explainable, we achieve skill-level explainability, enhancing the understanding of the decision-making process in complex tasks. Experimental results demonstrate that our method achieves performance comparable to skill-based neural networks in complex robotic arm control domains. Furthermore, SkillTree offers explanations at the skill level, thereby increasing the transparency of the decision-making process. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_12173 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | SkillTree: Explainable Skill-Based Deep Reinforcement Learning for Long-Horizon Control Tasks Wen, Yongyan Li, Siyuan Zuo, Rongchang Yuan, Lei Mao, Hangyu Liu, Peng Machine Learning Artificial Intelligence Deep reinforcement learning (DRL) has achieved remarkable success in various research domains. However, its reliance on neural networks results in a lack of transparency, which limits its practical applications. To achieve explainability, decision trees have emerged as a popular and promising alternative to neural networks. Nonetheless, due to their limited expressiveness, traditional decision trees struggle with high-dimensional long-horizon continuous control tasks. In this paper, we proposes SkillTree, a novel framework that reduces complex continuous action spaces into discrete skill spaces. Our hierarchical approach integrates a differentiable decision tree within the high-level policy to generate skill embeddings, which subsequently guide the low-level policy in executing skills. By making skill decisions explainable, we achieve skill-level explainability, enhancing the understanding of the decision-making process in complex tasks. Experimental results demonstrate that our method achieves performance comparable to skill-based neural networks in complex robotic arm control domains. Furthermore, SkillTree offers explanations at the skill level, thereby increasing the transparency of the decision-making process. |
| title | SkillTree: Explainable Skill-Based Deep Reinforcement Learning for Long-Horizon Control Tasks |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2411.12173 |