Multi-Objective Deep Reinforcement Learning for Optimisation in Autonomous Systems
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910624819183616 |
|---|---|
| author | Rosero, Juan C. Dusparic, Ivana Cardozo, Nicolás |
| author_facet | Rosero, Juan C. Dusparic, Ivana Cardozo, Nicolás |
| contents | Reinforcement Learning (RL) is used extensively in Autonomous Systems (AS) as it enables learning at runtime without the need for a model of the environment or predefined actions. However, most applications of RL in AS, such as those based on Q-learning, can only optimize one objective, making it necessary in multi-objective systems to combine multiple objectives in a single objective function with predefined weights. A number of Multi-Objective Reinforcement Learning (MORL) techniques exist but they have mostly been applied in RL benchmarks rather than real-world AS systems. In this work, we use a MORL technique called Deep W-Learning (DWN) and apply it to the Emergent Web Servers exemplar, a self-adaptive server, to find the optimal configuration for runtime performance optimization. We compare DWN to two single-objective optimization implementations: ε-greedy algorithm and Deep Q-Networks. Our initial evaluation shows that DWN optimizes multiple objectives simultaneously with similar results than DQN and ε-greedy approaches, having a better performance for some metrics, and avoids issues associated with combining multiple objectives into a single utility function. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_01188 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Multi-Objective Deep Reinforcement Learning for Optimisation in Autonomous Systems Rosero, Juan C. Dusparic, Ivana Cardozo, Nicolás Artificial Intelligence Reinforcement Learning (RL) is used extensively in Autonomous Systems (AS) as it enables learning at runtime without the need for a model of the environment or predefined actions. However, most applications of RL in AS, such as those based on Q-learning, can only optimize one objective, making it necessary in multi-objective systems to combine multiple objectives in a single objective function with predefined weights. A number of Multi-Objective Reinforcement Learning (MORL) techniques exist but they have mostly been applied in RL benchmarks rather than real-world AS systems. In this work, we use a MORL technique called Deep W-Learning (DWN) and apply it to the Emergent Web Servers exemplar, a self-adaptive server, to find the optimal configuration for runtime performance optimization. We compare DWN to two single-objective optimization implementations: ε-greedy algorithm and Deep Q-Networks. Our initial evaluation shows that DWN optimizes multiple objectives simultaneously with similar results than DQN and ε-greedy approaches, having a better performance for some metrics, and avoids issues associated with combining multiple objectives into a single utility function. |
| title | Multi-Objective Deep Reinforcement Learning for Optimisation in Autonomous Systems |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2408.01188 |