Hammer: Robust Function-Calling for On-Device Language Models via Function Masking
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916431068659712 |
|---|---|
| author | Lin, Qiqiang Wen, Muning Peng, Qiuying Nie, Guanyu Liao, Junwei Wang, Jun Mo, Xiaoyun Zhou, Jiamu Cheng, Cheng Zhao, Yin Wang, Jun Zhang, Weinan |
| author_facet | Lin, Qiqiang Wen, Muning Peng, Qiuying Nie, Guanyu Liao, Junwei Wang, Jun Mo, Xiaoyun Zhou, Jiamu Cheng, Cheng Zhao, Yin Wang, Jun Zhang, Weinan |
| contents | Large language models have demonstrated impressive value in performing as autonomous agents when equipped with external tools and API calls. Nonetheless, effectively harnessing their potential for executing complex tasks crucially relies on enhancements in their function calling capabilities. This paper identifies a critical gap in existing function calling models, where performance varies significantly across benchmarks, often due to being misled by specific naming conventions. To address such an issue, we introduce Hammer, a novel family of foundation models specifically engineered for on-device function calling. Hammer employs an augmented dataset that enhances models' sensitivity to irrelevant functions and incorporates function masking techniques to minimize misleading. Our empirical evaluations reveal that Hammer not only outperforms larger models but also demonstrates robust generalization across diverse benchmarks, achieving sota results. Our open source contributions include a specialized dataset for irrelevance detection, a tuning framework for enhanced generalization, and the Hammer models, establishing a new standard for function calling performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_04587 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Hammer: Robust Function-Calling for On-Device Language Models via Function Masking Lin, Qiqiang Wen, Muning Peng, Qiuying Nie, Guanyu Liao, Junwei Wang, Jun Mo, Xiaoyun Zhou, Jiamu Cheng, Cheng Zhao, Yin Wang, Jun Zhang, Weinan Machine Learning Artificial Intelligence Software Engineering Large language models have demonstrated impressive value in performing as autonomous agents when equipped with external tools and API calls. Nonetheless, effectively harnessing their potential for executing complex tasks crucially relies on enhancements in their function calling capabilities. This paper identifies a critical gap in existing function calling models, where performance varies significantly across benchmarks, often due to being misled by specific naming conventions. To address such an issue, we introduce Hammer, a novel family of foundation models specifically engineered for on-device function calling. Hammer employs an augmented dataset that enhances models' sensitivity to irrelevant functions and incorporates function masking techniques to minimize misleading. Our empirical evaluations reveal that Hammer not only outperforms larger models but also demonstrates robust generalization across diverse benchmarks, achieving sota results. Our open source contributions include a specialized dataset for irrelevance detection, a tuning framework for enhanced generalization, and the Hammer models, establishing a new standard for function calling performance. |
| title | Hammer: Robust Function-Calling for On-Device Language Models via Function Masking |
| topic | Machine Learning Artificial Intelligence Software Engineering |
| url | https://arxiv.org/abs/2410.04587 |