SemanticALLI: Caching Reasoning, Not Just Responses, in Agentic Systems
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866914297795313664 |
|---|---|
| author | Chillara, Varun Kline, Dylan Alvares, Christopher Wooten, Evan Yang, Huan Khetan, Shlok Bauer, Cade Guillory, Tré Shah, Tanishka Dhariwal, Yashodhara Pavlov, Volodymyr Popstefanov, George |
| author_facet | Chillara, Varun Kline, Dylan Alvares, Christopher Wooten, Evan Yang, Huan Khetan, Shlok Bauer, Cade Guillory, Tré Shah, Tanishka Dhariwal, Yashodhara Pavlov, Volodymyr Popstefanov, George |
| contents | Agentic AI pipelines suffer from a hidden inefficiency: they frequently reconstruct identical intermediate logic, such as metric normalization or chart scaffolding, even when the user's natural language phrasing is entirely novel. Conventional boundary caching fails to capture this inefficiency because it treats inference as a monolithic black box.
We introduce SemanticALLI, a pipeline-aware architecture within Alli (PMG's marketing intelligence platform), designed to operationalize redundant reasoning. By decomposing generation into Analytic Intent Resolution (AIR) and Visualization Synthesis (VS), SemanticALLI elevates structured intermediate representations (IRs) to first-class, cacheable artifacts.
The impact of caching within the agentic loop is substantial. In our evaluation, baseline monolithic caching caps at a 38.7% hit rate due to linguistic variance. In contrast, our structured approach allows for an additional stage, the Visualization Synthesis stage, to achieve an 83.10% hit rate, bypassing 4,023 LLM calls with a median latency of just 2.66 ms. This internal reuse reduces total token consumption, offering a practical lesson for AI system design: even when users rarely repeat themselves, the pipeline often does, at stable, structured checkpoints where caching is most reliable. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_16286 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | SemanticALLI: Caching Reasoning, Not Just Responses, in Agentic Systems Chillara, Varun Kline, Dylan Alvares, Christopher Wooten, Evan Yang, Huan Khetan, Shlok Bauer, Cade Guillory, Tré Shah, Tanishka Dhariwal, Yashodhara Pavlov, Volodymyr Popstefanov, George Artificial Intelligence Multiagent Systems Agentic AI pipelines suffer from a hidden inefficiency: they frequently reconstruct identical intermediate logic, such as metric normalization or chart scaffolding, even when the user's natural language phrasing is entirely novel. Conventional boundary caching fails to capture this inefficiency because it treats inference as a monolithic black box. We introduce SemanticALLI, a pipeline-aware architecture within Alli (PMG's marketing intelligence platform), designed to operationalize redundant reasoning. By decomposing generation into Analytic Intent Resolution (AIR) and Visualization Synthesis (VS), SemanticALLI elevates structured intermediate representations (IRs) to first-class, cacheable artifacts. The impact of caching within the agentic loop is substantial. In our evaluation, baseline monolithic caching caps at a 38.7% hit rate due to linguistic variance. In contrast, our structured approach allows for an additional stage, the Visualization Synthesis stage, to achieve an 83.10% hit rate, bypassing 4,023 LLM calls with a median latency of just 2.66 ms. This internal reuse reduces total token consumption, offering a practical lesson for AI system design: even when users rarely repeat themselves, the pipeline often does, at stable, structured checkpoints where caching is most reliable. |
| title | SemanticALLI: Caching Reasoning, Not Just Responses, in Agentic Systems |
| topic | Artificial Intelligence Multiagent Systems |
| url | https://arxiv.org/abs/2601.16286 |