Algorithms for learning value-aligned policies considering admissibility relaxation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917687494443008 |
|---|---|
| author | Holgado-Sánchez, Andrés Arias, Joaquín Billhardt, Holger Ossowski, Sascha |
| author_facet | Holgado-Sánchez, Andrés Arias, Joaquín Billhardt, Holger Ossowski, Sascha |
| contents | The emerging field of \emph{value awareness engineering} claims that software agents and systems should be value-aware, i.e. they must make decisions in accordance with human values. In this context, such agents must be capable of explicitly reasoning as to how far different courses of action are aligned with these values. For this purpose, values are often modelled as preferences over states or actions, which are then aggregated to determine the sequences of actions that are maximally aligned with a certain value. Recently, additional value admissibility constraints at this level have been considered as well.
However, often relaxed versions of these constraints are needed, and this increases considerably the complexity of computing value-aligned policies. To obtain efficient algorithms that make value-aligned decisions considering admissibility relaxation, we propose the use of learning techniques, in particular, we have used constrained reinforcement learning algorithms. In this paper, we present two algorithms, $ε\text{-}ADQL$ for strategies based on local alignment and its extension $ε\text{-}CADQL$ for a sequence of decisions. We have validated their efficiency in a water distribution problem in a drought scenario. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_04838 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Algorithms for learning value-aligned policies considering admissibility relaxation Holgado-Sánchez, Andrés Arias, Joaquín Billhardt, Holger Ossowski, Sascha Artificial Intelligence Computers and Society Machine Learning The emerging field of \emph{value awareness engineering} claims that software agents and systems should be value-aware, i.e. they must make decisions in accordance with human values. In this context, such agents must be capable of explicitly reasoning as to how far different courses of action are aligned with these values. For this purpose, values are often modelled as preferences over states or actions, which are then aggregated to determine the sequences of actions that are maximally aligned with a certain value. Recently, additional value admissibility constraints at this level have been considered as well. However, often relaxed versions of these constraints are needed, and this increases considerably the complexity of computing value-aligned policies. To obtain efficient algorithms that make value-aligned decisions considering admissibility relaxation, we propose the use of learning techniques, in particular, we have used constrained reinforcement learning algorithms. In this paper, we present two algorithms, $ε\text{-}ADQL$ for strategies based on local alignment and its extension $ε\text{-}CADQL$ for a sequence of decisions. We have validated their efficiency in a water distribution problem in a drought scenario. |
| title | Algorithms for learning value-aligned policies considering admissibility relaxation |
| topic | Artificial Intelligence Computers and Society Machine Learning |
| url | https://arxiv.org/abs/2406.04838 |