Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2408.11845 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913476512841728 |
|---|---|
| author | Pang, Yutong Paul, Debjyoti Jiang, Kevin Zhang, Xuedong Lei, Xin |
| author_facet | Pang, Yutong Paul, Debjyoti Jiang, Kevin Zhang, Xuedong Lei, Xin |
| contents | This paper introduces two advancements in the field of Large Language Model Annotation with a focus on punctuation restoration tasks. Our first contribution is the application of LLaMA for punctuation restoration, which demonstrates superior performance compared to the established benchmark.
Despite its impressive quality, LLaMA faces challenges regarding inference speed and hallucinations. To address this, our second contribution presents Forward Pass Only Decoding (FPOD), a novel decoding approach for annotation tasks. This innovative method results in a substantial 19.8x improvement in inference speed, effectively addressing a critical bottleneck and enhancing the practical utility of LLaMA for large-scale data annotation tasks without hallucinations.
The combination of these contributions not only solidifies LLaMA as a powerful tool for punctuation restoration but also highlights FPOD as a crucial strategy for overcoming speed constraints. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_11845 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | LLaMA based Punctuation Restoration With Forward Pass Only Decoding Pang, Yutong Paul, Debjyoti Jiang, Kevin Zhang, Xuedong Lei, Xin Computation and Language This paper introduces two advancements in the field of Large Language Model Annotation with a focus on punctuation restoration tasks. Our first contribution is the application of LLaMA for punctuation restoration, which demonstrates superior performance compared to the established benchmark. Despite its impressive quality, LLaMA faces challenges regarding inference speed and hallucinations. To address this, our second contribution presents Forward Pass Only Decoding (FPOD), a novel decoding approach for annotation tasks. This innovative method results in a substantial 19.8x improvement in inference speed, effectively addressing a critical bottleneck and enhancing the practical utility of LLaMA for large-scale data annotation tasks without hallucinations. The combination of these contributions not only solidifies LLaMA as a powerful tool for punctuation restoration but also highlights FPOD as a crucial strategy for overcoming speed constraints. |
| title | LLaMA based Punctuation Restoration With Forward Pass Only Decoding |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2408.11845 |