ERGO: Entropy-guided Resetting for Generation Optimization in Multi-turn Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909881570689024 |
|---|---|
| author | Khalid, Haziq Mohammad Jeyaganthan, Athikash Do, Timothy Fu, Yicheng O'Brien, Sean Sharma, Vasu Zhu, Kevin |
| author_facet | Khalid, Haziq Mohammad Jeyaganthan, Athikash Do, Timothy Fu, Yicheng O'Brien, Sean Sharma, Vasu Zhu, Kevin |
| contents | Large Language Models (LLMs) suffer significant performance degradation in multi-turn conversations when information is presented incrementally. Given that multi-turn conversations characterize everyday interactions with LLMs, this degradation poses a severe challenge to real world usability. We hypothesize that abrupt increases in model uncertainty signal misalignment in multi-turn LLM interactions, and we exploit this insight to dynamically realign conversational context. We introduce ERGO (Entropy-guided Resetting for Generation Optimization), which continuously quantifies internal uncertainty via Shannon entropy over next token distributions and triggers adaptive prompt consolidation when a sharp spike in entropy is detected. By treating uncertainty as a first class signal rather than a nuisance to eliminate, ERGO embraces variability in language and modeling, representing and responding to uncertainty. In multi-turn tasks with incrementally revealed instructions, ERGO yields a 56.6% average performance gain over standard baselines, increases aptitude (peak performance capability) by 24.7%, and decreases unreliability (variability in performance) by 35.3%, demonstrating that uncertainty aware interventions can improve both accuracy and reliability in conversational AI. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_14077 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ERGO: Entropy-guided Resetting for Generation Optimization in Multi-turn Language Models Khalid, Haziq Mohammad Jeyaganthan, Athikash Do, Timothy Fu, Yicheng O'Brien, Sean Sharma, Vasu Zhu, Kevin Computation and Language Large Language Models (LLMs) suffer significant performance degradation in multi-turn conversations when information is presented incrementally. Given that multi-turn conversations characterize everyday interactions with LLMs, this degradation poses a severe challenge to real world usability. We hypothesize that abrupt increases in model uncertainty signal misalignment in multi-turn LLM interactions, and we exploit this insight to dynamically realign conversational context. We introduce ERGO (Entropy-guided Resetting for Generation Optimization), which continuously quantifies internal uncertainty via Shannon entropy over next token distributions and triggers adaptive prompt consolidation when a sharp spike in entropy is detected. By treating uncertainty as a first class signal rather than a nuisance to eliminate, ERGO embraces variability in language and modeling, representing and responding to uncertainty. In multi-turn tasks with incrementally revealed instructions, ERGO yields a 56.6% average performance gain over standard baselines, increases aptitude (peak performance capability) by 24.7%, and decreases unreliability (variability in performance) by 35.3%, demonstrating that uncertainty aware interventions can improve both accuracy and reliability in conversational AI. |
| title | ERGO: Entropy-guided Resetting for Generation Optimization in Multi-turn Language Models |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2510.14077 |