Less is More: Optimizing Function Calling for LLM Execution on Edge Devices
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866929603582361600 |
|---|---|
| author | Paramanayakam, Varatheepan Karatzas, Andreas Anagnostopoulos, Iraklis Stamoulis, Dimitrios |
| author_facet | Paramanayakam, Varatheepan Karatzas, Andreas Anagnostopoulos, Iraklis Stamoulis, Dimitrios |
| contents | The advanced function-calling capabilities of foundation models open up new possibilities for deploying agents to perform complex API tasks. However, managing large amounts of data and interacting with numerous APIs makes function calling hardware-intensive and costly, especially on edge devices. Current Large Language Models (LLMs) struggle with function calling at the edge because they cannot handle complex inputs or manage multiple tools effectively. This results in low task-completion accuracy, increased delays, and higher power consumption. In this work, we introduce Less-is-More, a novel fine-tuning-free function-calling scheme for dynamic tool selection. Our approach is based on the key insight that selectively reducing the number of tools available to LLMs significantly improves their function-calling performance, execution time, and power efficiency on edge devices. Experimental results with state-of-the-art LLMs on edge hardware show agentic success rate improvements, with execution time reduced by up to 70% and power consumption by up to 40%. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_15399 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Less is More: Optimizing Function Calling for LLM Execution on Edge Devices Paramanayakam, Varatheepan Karatzas, Andreas Anagnostopoulos, Iraklis Stamoulis, Dimitrios Performance Distributed, Parallel, and Cluster Computing Machine Learning The advanced function-calling capabilities of foundation models open up new possibilities for deploying agents to perform complex API tasks. However, managing large amounts of data and interacting with numerous APIs makes function calling hardware-intensive and costly, especially on edge devices. Current Large Language Models (LLMs) struggle with function calling at the edge because they cannot handle complex inputs or manage multiple tools effectively. This results in low task-completion accuracy, increased delays, and higher power consumption. In this work, we introduce Less-is-More, a novel fine-tuning-free function-calling scheme for dynamic tool selection. Our approach is based on the key insight that selectively reducing the number of tools available to LLMs significantly improves their function-calling performance, execution time, and power efficiency on edge devices. Experimental results with state-of-the-art LLMs on edge hardware show agentic success rate improvements, with execution time reduced by up to 70% and power consumption by up to 40%. |
| title | Less is More: Optimizing Function Calling for LLM Execution on Edge Devices |
| topic | Performance Distributed, Parallel, and Cluster Computing Machine Learning |
| url | https://arxiv.org/abs/2411.15399 |