LangProp: A code optimization framework using Large Language Models applied to driving
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929334462185472 |
|---|---|
| author | Ishida, Shu Corrado, Gianluca Fedoseev, George Yeo, Hudson Russell, Lloyd Shotton, Jamie Henriques, João F. Hu, Anthony |
| author_facet | Ishida, Shu Corrado, Gianluca Fedoseev, George Yeo, Hudson Russell, Lloyd Shotton, Jamie Henriques, João F. Hu, Anthony |
| contents | We propose LangProp, a framework for iteratively optimizing code generated by large language models (LLMs), in both supervised and reinforcement learning settings. While LLMs can generate sensible coding solutions zero-shot, they are often sub-optimal. Especially for code generation tasks, it is likely that the initial code will fail on certain edge cases. LangProp automatically evaluates the code performance on a dataset of input-output pairs, catches any exceptions, and feeds the results back to the LLM in the training loop, so that the LLM can iteratively improve the code it generates. By adopting a metric- and data-driven training paradigm for this code optimization procedure, one could easily adapt findings from traditional machine learning techniques such as imitation learning, DAgger, and reinforcement learning. We show LangProp's applicability to general domains such as Sudoku and CartPole, as well as demonstrate the first proof of concept of automated code optimization for autonomous driving in CARLA. We show that LangProp can generate interpretable and transparent policies that can be verified and improved in a metric- and data-driven way. Our code is available at https://github.com/shuishida/LangProp. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_10314 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | LangProp: A code optimization framework using Large Language Models applied to driving Ishida, Shu Corrado, Gianluca Fedoseev, George Yeo, Hudson Russell, Lloyd Shotton, Jamie Henriques, João F. Hu, Anthony Software Engineering Artificial Intelligence Machine Learning Robotics We propose LangProp, a framework for iteratively optimizing code generated by large language models (LLMs), in both supervised and reinforcement learning settings. While LLMs can generate sensible coding solutions zero-shot, they are often sub-optimal. Especially for code generation tasks, it is likely that the initial code will fail on certain edge cases. LangProp automatically evaluates the code performance on a dataset of input-output pairs, catches any exceptions, and feeds the results back to the LLM in the training loop, so that the LLM can iteratively improve the code it generates. By adopting a metric- and data-driven training paradigm for this code optimization procedure, one could easily adapt findings from traditional machine learning techniques such as imitation learning, DAgger, and reinforcement learning. We show LangProp's applicability to general domains such as Sudoku and CartPole, as well as demonstrate the first proof of concept of automated code optimization for autonomous driving in CARLA. We show that LangProp can generate interpretable and transparent policies that can be verified and improved in a metric- and data-driven way. Our code is available at https://github.com/shuishida/LangProp. |
| title | LangProp: A code optimization framework using Large Language Models applied to driving |
| topic | Software Engineering Artificial Intelligence Machine Learning Robotics |
| url | https://arxiv.org/abs/2401.10314 |