PositionID: LLMs can Control Lengths, Copy and Paste with Explicit Positional Awareness
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912065289977856 |
|---|---|
| author | Wang, Zekun Duan, Feiyu Zhang, Yibo Zhou, Wangchunshu Xu, Ke Huang, Wenhao Fu, Jie |
| author_facet | Wang, Zekun Duan, Feiyu Zhang, Yibo Zhou, Wangchunshu Xu, Ke Huang, Wenhao Fu, Jie |
| contents | Large Language Models (LLMs) demonstrate impressive capabilities across various domains, including role-playing, creative writing, mathematical reasoning, and coding. Despite these advancements, LLMs still encounter challenges with length control, frequently failing to adhere to specific length constraints due to their token-level operations and insufficient training on data with strict length limitations. We identify this issue as stemming from a lack of positional awareness and propose novel approaches--PositionID Prompting and PositionID Fine-Tuning--to address it. These methods enhance the model's ability to continuously monitor and manage text length during generation. Additionally, we introduce PositionID CP Prompting to enable LLMs to perform copy and paste operations accurately. Furthermore, we develop two benchmarks for evaluating length control and copy-paste abilities. Our experiments demonstrate that our methods significantly improve the model's adherence to length constraints and copy-paste accuracy without compromising response quality. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_07035 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | PositionID: LLMs can Control Lengths, Copy and Paste with Explicit Positional Awareness Wang, Zekun Duan, Feiyu Zhang, Yibo Zhou, Wangchunshu Xu, Ke Huang, Wenhao Fu, Jie Computation and Language Artificial Intelligence Large Language Models (LLMs) demonstrate impressive capabilities across various domains, including role-playing, creative writing, mathematical reasoning, and coding. Despite these advancements, LLMs still encounter challenges with length control, frequently failing to adhere to specific length constraints due to their token-level operations and insufficient training on data with strict length limitations. We identify this issue as stemming from a lack of positional awareness and propose novel approaches--PositionID Prompting and PositionID Fine-Tuning--to address it. These methods enhance the model's ability to continuously monitor and manage text length during generation. Additionally, we introduce PositionID CP Prompting to enable LLMs to perform copy and paste operations accurately. Furthermore, we develop two benchmarks for evaluating length control and copy-paste abilities. Our experiments demonstrate that our methods significantly improve the model's adherence to length constraints and copy-paste accuracy without compromising response quality. |
| title | PositionID: LLMs can Control Lengths, Copy and Paste with Explicit Positional Awareness |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2410.07035 |