MathViz-E: A Case-study in Domain-Specialized Tool-Using Agents
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909267235176448 |
|---|---|
| author | Bulusu, Arya Man, Brandon Jagmohan, Ashish Vempaty, Aditya Mari-Wyka, Jennifer Akkil, Deepak |
| author_facet | Bulusu, Arya Man, Brandon Jagmohan, Ashish Vempaty, Aditya Mari-Wyka, Jennifer Akkil, Deepak |
| contents | There has been significant recent interest in harnessing LLMs to control software systems through multi-step reasoning, planning and tool-usage. While some promising results have been obtained, application to specific domains raises several general issues including the control of specialized domain tools, the lack of existing datasets for training and evaluation, and the non-triviality of automated system evaluation and improvement. In this paper, we present a case-study where we examine these issues in the context of a specific domain. Specifically, we present an automated math visualizer and solver system for mathematical pedagogy. The system orchestrates mathematical solvers and math graphing tools to produce accurate visualizations from simple natural language commands. We describe the creation of specialized data-sets, and also develop an auto-evaluator to easily evaluate the outputs of our system by comparing them to ground-truth expressions. We have open sourced the data-sets and code for the proposed system. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_17544 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MathViz-E: A Case-study in Domain-Specialized Tool-Using Agents Bulusu, Arya Man, Brandon Jagmohan, Ashish Vempaty, Aditya Mari-Wyka, Jennifer Akkil, Deepak Software Engineering Artificial Intelligence There has been significant recent interest in harnessing LLMs to control software systems through multi-step reasoning, planning and tool-usage. While some promising results have been obtained, application to specific domains raises several general issues including the control of specialized domain tools, the lack of existing datasets for training and evaluation, and the non-triviality of automated system evaluation and improvement. In this paper, we present a case-study where we examine these issues in the context of a specific domain. Specifically, we present an automated math visualizer and solver system for mathematical pedagogy. The system orchestrates mathematical solvers and math graphing tools to produce accurate visualizations from simple natural language commands. We describe the creation of specialized data-sets, and also develop an auto-evaluator to easily evaluate the outputs of our system by comparing them to ground-truth expressions. We have open sourced the data-sets and code for the proposed system. |
| title | MathViz-E: A Case-study in Domain-Specialized Tool-Using Agents |
| topic | Software Engineering Artificial Intelligence |
| url | https://arxiv.org/abs/2407.17544 |