PaDeLLM-NER: Parallel Decoding in Large Language Models for Named Entity Recognition
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916489735438336 |
|---|---|
| author | Lu, Jinghui Yang, Ziwei Wang, Yanjie Liu, Xuejing Mac Namee, Brian Huang, Can |
| author_facet | Lu, Jinghui Yang, Ziwei Wang, Yanjie Liu, Xuejing Mac Namee, Brian Huang, Can |
| contents | In this study, we aim to reduce generation latency for Named Entity Recognition (NER) with Large Language Models (LLMs). The main cause of high latency in LLMs is the sequential decoding process, which autoregressively generates all labels and mentions for NER, significantly increase the sequence length. To this end, we introduce Parallel Decoding in LLM for NE} (PaDeLLM-NER), a approach that integrates seamlessly into existing generative model frameworks without necessitating additional modules or architectural modifications. PaDeLLM-NER allows for the simultaneous decoding of all mentions, thereby reducing generation latency. Experiments reveal that PaDeLLM-NER significantly increases inference speed that is 1.76 to 10.22 times faster than the autoregressive approach for both English and Chinese. Simultaneously it maintains the quality of predictions as evidenced by the performance that is on par with the state-of-the-art across various datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_04838 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | PaDeLLM-NER: Parallel Decoding in Large Language Models for Named Entity Recognition Lu, Jinghui Yang, Ziwei Wang, Yanjie Liu, Xuejing Mac Namee, Brian Huang, Can Computation and Language Artificial Intelligence In this study, we aim to reduce generation latency for Named Entity Recognition (NER) with Large Language Models (LLMs). The main cause of high latency in LLMs is the sequential decoding process, which autoregressively generates all labels and mentions for NER, significantly increase the sequence length. To this end, we introduce Parallel Decoding in LLM for NE} (PaDeLLM-NER), a approach that integrates seamlessly into existing generative model frameworks without necessitating additional modules or architectural modifications. PaDeLLM-NER allows for the simultaneous decoding of all mentions, thereby reducing generation latency. Experiments reveal that PaDeLLM-NER significantly increases inference speed that is 1.76 to 10.22 times faster than the autoregressive approach for both English and Chinese. Simultaneously it maintains the quality of predictions as evidenced by the performance that is on par with the state-of-the-art across various datasets. |
| title | PaDeLLM-NER: Parallel Decoding in Large Language Models for Named Entity Recognition |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2402.04838 |