TVCACHE: A Stateful Tool-Value Cache for Post-Training LLM Agents
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912897777532928 |
|---|---|
| author | Kumar, Abhishek Vijaya Kataria, Bhaskar Oh, Byungsoo Manzoor, Emaad Singh, Rachee |
| author_facet | Kumar, Abhishek Vijaya Kataria, Bhaskar Oh, Byungsoo Manzoor, Emaad Singh, Rachee |
| contents | In RL post-training of LLM agents, calls to external tools take several seconds or even minutes, leaving allocated GPUs idle and inflating post-training time and cost. While many tool invocations repeat across parallel rollouts and could in principle be cached, naively caching their outputs for reuse is incorrect since tool outputs depend on the environment state induced by prior agent interactions. We present TVCACHE, a stateful tool-value cache for LLM agent post-training. TVCACHE maintains a tree of observed tool-call sequences and performs longest-prefix matching for cache lookups: a hit occurs only when the agent's full tool history matches a previously executed sequence, guaranteeing identical environment state. On three diverse workloads-terminal-based tasks, SQL generation, and video understanding. TVCACHE achieves cache hit rates of up to 70% and reduces median tool call execution time by up to 6.9X, with no degradation in post-training reward accumulation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_10986 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | TVCACHE: A Stateful Tool-Value Cache for Post-Training LLM Agents Kumar, Abhishek Vijaya Kataria, Bhaskar Oh, Byungsoo Manzoor, Emaad Singh, Rachee Machine Learning In RL post-training of LLM agents, calls to external tools take several seconds or even minutes, leaving allocated GPUs idle and inflating post-training time and cost. While many tool invocations repeat across parallel rollouts and could in principle be cached, naively caching their outputs for reuse is incorrect since tool outputs depend on the environment state induced by prior agent interactions. We present TVCACHE, a stateful tool-value cache for LLM agent post-training. TVCACHE maintains a tree of observed tool-call sequences and performs longest-prefix matching for cache lookups: a hit occurs only when the agent's full tool history matches a previously executed sequence, guaranteeing identical environment state. On three diverse workloads-terminal-based tasks, SQL generation, and video understanding. TVCACHE achieves cache hit rates of up to 70% and reduces median tool call execution time by up to 6.9X, with no degradation in post-training reward accumulation. |
| title | TVCACHE: A Stateful Tool-Value Cache for Post-Training LLM Agents |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2602.10986 |