Optimizing Sensor Redundancy in Sequential Decision-Making Problems
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916517057134592 |
|---|---|
| author | Nüßlein, Jonas Zorn, Maximilian Ritz, Fabian Stein, Jonas Stenzel, Gerhard Schönberger, Julian Gabor, Thomas Linnhoff-Popien, Claudia |
| author_facet | Nüßlein, Jonas Zorn, Maximilian Ritz, Fabian Stein, Jonas Stenzel, Gerhard Schönberger, Julian Gabor, Thomas Linnhoff-Popien, Claudia |
| contents | Reinforcement Learning (RL) policies are designed to predict actions based on current observations to maximize cumulative future rewards. In real-world applications (i.e., non-simulated environments), sensors are essential for measuring the current state and providing the observations on which RL policies rely to make decisions. A significant challenge in deploying RL policies in real-world scenarios is handling sensor dropouts, which can result from hardware malfunctions, physical damage, or environmental factors like dust on a camera lens. A common strategy to mitigate this issue is the use of backup sensors, though this comes with added costs. This paper explores the optimization of backup sensor configurations to maximize expected returns while keeping costs below a specified threshold, C. Our approach uses a second-order approximation of expected returns and includes penalties for exceeding cost constraints. We then optimize this quadratic program using Tabu Search, a meta-heuristic algorithm. The approach is evaluated across eight OpenAI Gym environments and a custom Unity-based robotic environment (RobotArmGrasping). Empirical results demonstrate that our quadratic program effectively approximates real expected returns, facilitating the identification of optimal sensor configurations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_07686 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Optimizing Sensor Redundancy in Sequential Decision-Making Problems Nüßlein, Jonas Zorn, Maximilian Ritz, Fabian Stein, Jonas Stenzel, Gerhard Schönberger, Julian Gabor, Thomas Linnhoff-Popien, Claudia Robotics Artificial Intelligence Machine Learning Reinforcement Learning (RL) policies are designed to predict actions based on current observations to maximize cumulative future rewards. In real-world applications (i.e., non-simulated environments), sensors are essential for measuring the current state and providing the observations on which RL policies rely to make decisions. A significant challenge in deploying RL policies in real-world scenarios is handling sensor dropouts, which can result from hardware malfunctions, physical damage, or environmental factors like dust on a camera lens. A common strategy to mitigate this issue is the use of backup sensors, though this comes with added costs. This paper explores the optimization of backup sensor configurations to maximize expected returns while keeping costs below a specified threshold, C. Our approach uses a second-order approximation of expected returns and includes penalties for exceeding cost constraints. We then optimize this quadratic program using Tabu Search, a meta-heuristic algorithm. The approach is evaluated across eight OpenAI Gym environments and a custom Unity-based robotic environment (RobotArmGrasping). Empirical results demonstrate that our quadratic program effectively approximates real expected returns, facilitating the identification of optimal sensor configurations. |
| title | Optimizing Sensor Redundancy in Sequential Decision-Making Problems |
| topic | Robotics Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2412.07686 |