Concise Thoughts: Impact of Output Length on LLM Reasoning and Cost
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912200314060800 |
|---|---|
| author | Nayab, Sania Rossolini, Giulio Simoni, Marco Saracino, Andrea Buttazzo, Giorgio Manes, Nicolamaria Giacomelli, Fabrizio |
| author_facet | Nayab, Sania Rossolini, Giulio Simoni, Marco Saracino, Andrea Buttazzo, Giorgio Manes, Nicolamaria Giacomelli, Fabrizio |
| contents | Today's large language models (LLMs) can solve challenging question-answering tasks, and prompt engineering techniques, such as chain-of-thought (CoT), have gained attention for enhancing the explanation and correctness of outputs. However, many models and techniques tend to produce excessively verbose and lengthy answers, leading to issues with both conciseness and generation time. To address this, this paper analyzes the impact of output lengths on LLM inference pipelines by introducing and proposing novel metrics to evaluate the \textit{correct conciseness} of a model and related prompting techniques. Then, we examine the impact of controlling output length through a refined prompt engineering strategy, Constrained-CoT (CCoT), which encourages the model to produce more concise outputs. To better understand the effects of such a prompt, we also introduce two additional scores for analyzing the conciseness, measured in terms of redundancy and information flow in generated answers. Experiments on pretrained LLMs and multiple datasets demonstrate the benefits of the proposed metrics and the effectiveness of CCoT across different models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_19825 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Concise Thoughts: Impact of Output Length on LLM Reasoning and Cost Nayab, Sania Rossolini, Giulio Simoni, Marco Saracino, Andrea Buttazzo, Giorgio Manes, Nicolamaria Giacomelli, Fabrizio Computation and Language Artificial Intelligence Today's large language models (LLMs) can solve challenging question-answering tasks, and prompt engineering techniques, such as chain-of-thought (CoT), have gained attention for enhancing the explanation and correctness of outputs. However, many models and techniques tend to produce excessively verbose and lengthy answers, leading to issues with both conciseness and generation time. To address this, this paper analyzes the impact of output lengths on LLM inference pipelines by introducing and proposing novel metrics to evaluate the \textit{correct conciseness} of a model and related prompting techniques. Then, we examine the impact of controlling output length through a refined prompt engineering strategy, Constrained-CoT (CCoT), which encourages the model to produce more concise outputs. To better understand the effects of such a prompt, we also introduce two additional scores for analyzing the conciseness, measured in terms of redundancy and information flow in generated answers. Experiments on pretrained LLMs and multiple datasets demonstrate the benefits of the proposed metrics and the effectiveness of CCoT across different models. |
| title | Concise Thoughts: Impact of Output Length on LLM Reasoning and Cost |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2407.19825 |