Cover Image

Stateful Inference for Low-Latency Multi-Agent Tool Calling

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Norgren, Victor
Format:	Preprint
Published:	2026
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Attention Once Is All You Need: Efficient Streaming Inference with Stateful Transformers
by: Norgren, Victor
Published: (2026)

Reinforced Agent: Inference-Time Feedback for Tool-Calling Agents
by: Ta, Anh, et al.
Published: (2026)

Latency-Quality Routing for Functionally Equivalent Tools in LLM Agents
by: Chu, Kexin, et al.
Published: (2026)

Low Latency Transformer Inference on FPGAs for Physics Applications with hls4ml
by: Jiang, Zhixing, et al.
Published: (2024)

Speculative Interaction Agents: Building Real-Time Agents with Asynchronous I/O and Speculative Tool Calling
by: Hooper, Coleman, et al.
Published: (2026)

ML Inference Scheduling with Predictable Latency
by: Zhao, Haidong, et al.
Published: (2025)

Outcome-Aware Tool Selection for Semantic Routers: Latency-Constrained Learning Without LLM Inference
by: Chen, Huamin, et al.
Published: (2026)

Reinforcement Learning for Tool-Calling Agents in Fast Healthcare Interoperability Resources (FHIR)
by: Knorr, Marius S., et al.
Published: (2026)

ServerlessLLM: Low-Latency Serverless Inference for Large Language Models
by: Fu, Yao, et al.
Published: (2024)

Architectural Implications of Neural Network Inference for High Data-Rate, Low-Latency Scientific Applications
by: Weng, Olivia, et al.
Published: (2024)

PolyLUT: Learning Piecewise Polynomials for Ultra-Low Latency FPGA LUT-based Inference
by: Andronic, Marta, et al.
Published: (2023)

Dynamic Tool Dependency Retrieval for Lightweight Function Calling
by: Patel, Bhrij, et al.
Published: (2025)

To Call or Not to Call: Diagnosing Intrinsic Over-Calling Bias in LLM Agents
by: Shi, Wei, et al.
Published: (2026)

Hybrid JIT-CUDA Graph Optimization for Low-Latency Large Language Model Inference
by: Yadav, Divakar Kumar, et al.
Published: (2026)

LiteVLM: A Low-Latency Vision-Language Model Inference Pipeline for Resource-Constrained Environments
by: Huang, Jin, et al.
Published: (2025)

Low-Latency Neural Inference on an Edge Device for Real-Time Handwriting Recognition from EEG Signals
by: Sen, Ovishake, et al.
Published: (2025)

ChipChat: Low-Latency Cascaded Conversational Agent in MLX
by: Likhomanenko, Tatiana, et al.
Published: (2025)

SaFeR-ToolKit: Structured Reasoning via Virtual Tool Calling for Multimodal Safety
by: Xu, Zixuan, et al.
Published: (2026)

SNAP: Low-Latency Test-Time Adaptation with Sparse Updates
by: Cha, Hyeongheon, et al.
Published: (2025)

SynAE: A Framework for Measuring the Quality of Synthetic Data for Tool-Calling Agent Evaluations
by: Wang, Shuaiqi, et al.
Published: (2026)

lm-Meter: Unveiling Runtime Inference Latency for On-Device Language Models
by: Wang, Haoxin, et al.
Published: (2025)

TinyAgent: Function Calling at the Edge
by: Erdogan, Lutfi Eren, et al.
Published: (2024)

Agent JIT Compilation for Latency-Optimizing Web Agent Planning and Scheduling
by: Winston, Caleb, et al.
Published: (2026)

TVCACHE: A Stateful Tool-Value Cache for Post-Training LLM Agents
by: Kumar, Abhishek Vijaya, et al.
Published: (2026)

Low Rank Field-Weighted Factorization Machines for Low Latency Item Recommendation
by: Shtoff, Alex, et al.
Published: (2024)

Greening AI Inference with Accuracy and Latency-aware User Incentives
by: Siris, Vasilios A., et al.
Published: (2026)

On-device Semantic Selection Made Low Latency and Memory Efficient with Monolithic Forwarding
by: Zhou, Jiahao, et al.
Published: (2025)

LoopTool: Closing the Data-Training Loop for Robust LLM Tool Calls
by: Zhang, Kangning, et al.
Published: (2025)

LatencyPrism: Online Non-intrusive Latency Sculpting for SLO-Guaranteed LLM Inference
by: Du, Yin, et al.
Published: (2026)

InstMeter: An Instruction-Level Method to Predict Energy and Latency of DL Model Inference on MCUs
by: Liu, Hao, et al.
Published: (2026)

An LLM-Tool Compiler for Fused Parallel Function Calling
by: Singh, Simranjit, et al.
Published: (2024)

ToolACE: Winning the Points of LLM Function Calling
by: Liu, Weiwen, et al.
Published: (2024)

Research on Low-Latency Inference and Training Efficiency Optimization for Graph Neural Network and Large Language Model-Based Recommendation Systems
by: Zhao, Yushang, et al.
Published: (2025)

Towards Deep Encrypted Training: Low-Latency, Memory-Efficient, and High-Throughput Inference for Privacy-Preserving Neural Networks
by: Njungle, Nges Brian, et al.
Published: (2026)

Truncated Neural Likelihood Estimation for Simulation-Based Inference in State-Space Models
by: Tsampourakis, Kostas, et al.
Published: (2026)

FlowDA: Accurate, Low-Latency Weather Data Assimilation via Flow Matching
by: Cheng, Ran, et al.
Published: (2026)

3D Optimization for AI Inference Scaling: Balancing Accuracy, Cost, and Latency
by: Jung, Minseok, et al.
Published: (2025)

Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve
by: Agrawal, Amey, et al.
Published: (2024)

DVFS-Aware DNN Inference on GPUs: Latency Modeling and Performance Analysis
by: Han, Yunchu, et al.
Published: (2025)

A Multi-Agent Framework for Stateful Inference-Time Search
by: Lalan, Arshika, et al.
Published: (2025)