Task-Specific Efficiency Analysis: When Small Language Models Outperform Large Language Models
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911535642705920 |
|---|---|
| author | Cao, Jinghan Ma, Yu Li, Xinjin Ren, Qingyang Chen, Xiangyun |
| author_facet | Cao, Jinghan Ma, Yu Li, Xinjin Ren, Qingyang Chen, Xiangyun |
| contents | Large Language Models achieve remarkable performance but incur substantial computational costs unsuitable for resource-constrained deployments. This paper presents the first comprehensive task-specific efficiency analysis comparing 16 language models across five diverse NLP tasks. We introduce the Performance-Efficiency Ratio (PER), a novel metric integrating accuracy, throughput, memory, and latency through geometric mean normalization. Our systematic evaluation reveals that small models (0.5--3B parameters) achieve superior PER scores across all given tasks. These findings establish quantitative foundations for deploying small models in production environments prioritizing inference efficiency over marginal accuracy gains. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_21389 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Task-Specific Efficiency Analysis: When Small Language Models Outperform Large Language Models Cao, Jinghan Ma, Yu Li, Xinjin Ren, Qingyang Chen, Xiangyun Computation and Language Machine Learning Large Language Models achieve remarkable performance but incur substantial computational costs unsuitable for resource-constrained deployments. This paper presents the first comprehensive task-specific efficiency analysis comparing 16 language models across five diverse NLP tasks. We introduce the Performance-Efficiency Ratio (PER), a novel metric integrating accuracy, throughput, memory, and latency through geometric mean normalization. Our systematic evaluation reveals that small models (0.5--3B parameters) achieve superior PER scores across all given tasks. These findings establish quantitative foundations for deploying small models in production environments prioritizing inference efficiency over marginal accuracy gains. |
| title | Task-Specific Efficiency Analysis: When Small Language Models Outperform Large Language Models |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2603.21389 |