GeckOpt: LLM System Efficiency via Intent-Based Tool Selection
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911851726503936 |
|---|---|
| author | Fore, Michael Singh, Simranjit Stamoulis, Dimitrios |
| author_facet | Fore, Michael Singh, Simranjit Stamoulis, Dimitrios |
| contents | In this preliminary study, we investigate a GPT-driven intent-based reasoning approach to streamline tool selection for large language models (LLMs) aimed at system efficiency. By identifying the intent behind user prompts at runtime, we narrow down the API toolset required for task execution, reducing token consumption by up to 24.6\%. Early results on a real-world, massively parallel Copilot platform with over 100 GPT-4-Turbo nodes show cost reductions and potential towards improving LLM-based system efficiency. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_15804 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | GeckOpt: LLM System Efficiency via Intent-Based Tool Selection Fore, Michael Singh, Simranjit Stamoulis, Dimitrios Machine Learning Artificial Intelligence In this preliminary study, we investigate a GPT-driven intent-based reasoning approach to streamline tool selection for large language models (LLMs) aimed at system efficiency. By identifying the intent behind user prompts at runtime, we narrow down the API toolset required for task execution, reducing token consumption by up to 24.6\%. Early results on a real-world, massively parallel Copilot platform with over 100 GPT-4-Turbo nodes show cost reductions and potential towards improving LLM-based system efficiency. |
| title | GeckOpt: LLM System Efficiency via Intent-Based Tool Selection |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2404.15804 |