AURA: Agent for Understanding, Reasoning, and Automated Tool Use in Voice-Driven Tasks
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909665306083328 |
|---|---|
| author | Maben, Leander Melroy Lakshmy, Gayathri Ganesh Radhakrishnan, Srijith Arora, Siddhant Watanabe, Shinji |
| author_facet | Maben, Leander Melroy Lakshmy, Gayathri Ganesh Radhakrishnan, Srijith Arora, Siddhant Watanabe, Shinji |
| contents | Despite advances in language and speech technologies, no open-source system enables full speech-to-speech, multi-turn dialogue with integrated tool use and agentic reasoning. We introduce AURA (Agent for Understanding, Reasoning, and Automated Tool Use), the first open-source, speech-native assistant capable of completing complex, goal-driven tasks through dynamic tool invocation and multi-turn conversation. AURA combines open-weight ASR, TTS, and LLMs in a cascaded pipeline and supports tools such as calendar booking, contact lookup, web search, and email. Its modular design allows easy integration of new tools using natural language prompts and action classes. On VoiceBench, AURA scores 92.75% on OpenBookQA-outperforming all open-weight systems and nearing GPT-4o-and 4.39 on AlpacaEval, competitive with other open-weight systems. Human evaluation shows 90% task success on complex, multi-turn speech tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_23049 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | AURA: Agent for Understanding, Reasoning, and Automated Tool Use in Voice-Driven Tasks Maben, Leander Melroy Lakshmy, Gayathri Ganesh Radhakrishnan, Srijith Arora, Siddhant Watanabe, Shinji Artificial Intelligence Computation and Language Sound Audio and Speech Processing 68T42, 68T50, I.2.7; I.2.11; H.5.5 Despite advances in language and speech technologies, no open-source system enables full speech-to-speech, multi-turn dialogue with integrated tool use and agentic reasoning. We introduce AURA (Agent for Understanding, Reasoning, and Automated Tool Use), the first open-source, speech-native assistant capable of completing complex, goal-driven tasks through dynamic tool invocation and multi-turn conversation. AURA combines open-weight ASR, TTS, and LLMs in a cascaded pipeline and supports tools such as calendar booking, contact lookup, web search, and email. Its modular design allows easy integration of new tools using natural language prompts and action classes. On VoiceBench, AURA scores 92.75% on OpenBookQA-outperforming all open-weight systems and nearing GPT-4o-and 4.39 on AlpacaEval, competitive with other open-weight systems. Human evaluation shows 90% task success on complex, multi-turn speech tasks. |
| title | AURA: Agent for Understanding, Reasoning, and Automated Tool Use in Voice-Driven Tasks |
| topic | Artificial Intelligence Computation and Language Sound Audio and Speech Processing 68T42, 68T50, I.2.7; I.2.11; H.5.5 |
| url | https://arxiv.org/abs/2506.23049 |