Speculative Actions: A Lossless Framework for Faster Agentic Systems
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915951303196672 |
|---|---|
| author | Ye, Naimeng Ahuja, Arnav Liargkovas, Georgios Lu, Yunan Kaffes, Kostis Peng, Tianyi |
| author_facet | Ye, Naimeng Ahuja, Arnav Liargkovas, Georgios Lu, Yunan Kaffes, Kostis Peng, Tianyi |
| contents | AI agents are increasingly deployed in complex, interactive environments, yet their runtime remains a major bottleneck for training, evaluation, and real-world use. Typical agent behavior unfolds sequentially, with each action requiring an API call that can incur substantial latency. For example, a game of chess between two state-of-the-art agents can take hours. We introduce Speculative Actions, a lossless acceleration framework for general agentic systems. Inspired by speculative execution in microprocessors and speculative decoding in LLM inference, our method uses faster models to predict likely future actions and execute them in parallel, committing only when predictions match. We evaluate speculative actions across gaming, e-commerce, and web search environments, and additionally study a lossy extension in an operating systems setting. Across domains, we achieve up to 55% next-action prediction accuracy, translating into up to 20% latency reductions. Finally, we present a cost-latency analysis that formalizes the tradeoff between speculative breadth and time savings. This analysis enables principled tuning and selective branch launching to ensure that multi-branch speculation delivers practical speedups without prohibitive cost growth. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_04371 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Speculative Actions: A Lossless Framework for Faster Agentic Systems Ye, Naimeng Ahuja, Arnav Liargkovas, Georgios Lu, Yunan Kaffes, Kostis Peng, Tianyi Artificial Intelligence Distributed, Parallel, and Cluster Computing Multiagent Systems AI agents are increasingly deployed in complex, interactive environments, yet their runtime remains a major bottleneck for training, evaluation, and real-world use. Typical agent behavior unfolds sequentially, with each action requiring an API call that can incur substantial latency. For example, a game of chess between two state-of-the-art agents can take hours. We introduce Speculative Actions, a lossless acceleration framework for general agentic systems. Inspired by speculative execution in microprocessors and speculative decoding in LLM inference, our method uses faster models to predict likely future actions and execute them in parallel, committing only when predictions match. We evaluate speculative actions across gaming, e-commerce, and web search environments, and additionally study a lossy extension in an operating systems setting. Across domains, we achieve up to 55% next-action prediction accuracy, translating into up to 20% latency reductions. Finally, we present a cost-latency analysis that formalizes the tradeoff between speculative breadth and time savings. This analysis enables principled tuning and selective branch launching to ensure that multi-branch speculation delivers practical speedups without prohibitive cost growth. |
| title | Speculative Actions: A Lossless Framework for Faster Agentic Systems |
| topic | Artificial Intelligence Distributed, Parallel, and Cluster Computing Multiagent Systems |
| url | https://arxiv.org/abs/2510.04371 |