Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2401.11666 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908515816177664 |
|---|---|
| author | Wang, Zhiyuan Qu, Xiaoyang Xiao, Jing Chen, Bokui Wang, Jianzong |
| author_facet | Wang, Zhiyuan Qu, Xiaoyang Xiao, Jing Chen, Bokui Wang, Jianzong |
| contents | Catastrophic forgetting poses a substantial challenge for managing intelligent agents controlled by a large model, causing performance degradation when these agents face new tasks. In our work, we propose a novel solution - the Progressive Prompt Decision Transformer (P2DT). This method enhances a transformer-based model by dynamically appending decision tokens during new task training, thus fostering task-specific policies. Our approach mitigates forgetting in continual and offline reinforcement learning scenarios. Moreover, P2DT leverages trajectories collected via traditional reinforcement learning from all tasks and generates new task-specific tokens during training, thereby retaining knowledge from previous studies. Preliminary results demonstrate that our model effectively alleviates catastrophic forgetting and scales well with increasing task environments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_11666 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | P2DT: Mitigating Forgetting in task-incremental Learning with progressive prompt Decision Transformer Wang, Zhiyuan Qu, Xiaoyang Xiao, Jing Chen, Bokui Wang, Jianzong Machine Learning Artificial Intelligence Catastrophic forgetting poses a substantial challenge for managing intelligent agents controlled by a large model, causing performance degradation when these agents face new tasks. In our work, we propose a novel solution - the Progressive Prompt Decision Transformer (P2DT). This method enhances a transformer-based model by dynamically appending decision tokens during new task training, thus fostering task-specific policies. Our approach mitigates forgetting in continual and offline reinforcement learning scenarios. Moreover, P2DT leverages trajectories collected via traditional reinforcement learning from all tasks and generates new task-specific tokens during training, thereby retaining knowledge from previous studies. Preliminary results demonstrate that our model effectively alleviates catastrophic forgetting and scales well with increasing task environments. |
| title | P2DT: Mitigating Forgetting in task-incremental Learning with progressive prompt Decision Transformer |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2401.11666 |