Offline Imitation Learning from Multiple Baselines with Applications to Compiler Optimization
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909154234335232 |
|---|---|
| author | Marinov, Teodor V. Agarwal, Alekh Trofin, Mircea |
| author_facet | Marinov, Teodor V. Agarwal, Alekh Trofin, Mircea |
| contents | This work studies a Reinforcement Learning (RL) problem in which we are given a set of trajectories collected with K baseline policies. Each of these policies can be quite suboptimal in isolation, and have strong performance in complementary parts of the state space. The goal is to learn a policy which performs as well as the best combination of baselines on the entire state space. We propose a simple imitation learning based algorithm, show a sample complexity bound on its accuracy and prove that the the algorithm is minimax optimal by showing a matching lower bound. Further, we apply the algorithm in the setting of machine learning guided compiler optimization to learn policies for inlining programs with the objective of creating a small binary. We demonstrate that we can learn a policy that outperforms an initial policy learned via standard RL through a few iterations of our approach. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_19462 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Offline Imitation Learning from Multiple Baselines with Applications to Compiler Optimization Marinov, Teodor V. Agarwal, Alekh Trofin, Mircea Machine Learning Programming Languages This work studies a Reinforcement Learning (RL) problem in which we are given a set of trajectories collected with K baseline policies. Each of these policies can be quite suboptimal in isolation, and have strong performance in complementary parts of the state space. The goal is to learn a policy which performs as well as the best combination of baselines on the entire state space. We propose a simple imitation learning based algorithm, show a sample complexity bound on its accuracy and prove that the the algorithm is minimax optimal by showing a matching lower bound. Further, we apply the algorithm in the setting of machine learning guided compiler optimization to learn policies for inlining programs with the objective of creating a small binary. We demonstrate that we can learn a policy that outperforms an initial policy learned via standard RL through a few iterations of our approach. |
| title | Offline Imitation Learning from Multiple Baselines with Applications to Compiler Optimization |
| topic | Machine Learning Programming Languages |
| url | https://arxiv.org/abs/2403.19462 |