From Long to Lean: Performance-aware and Adaptive Chain-of-Thought Compression via Multi-round Refinement
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911178232430592 |
|---|---|
| author | Yan, Jianzhi Liu, Le Pan, Youcheng Chen, Shiwei Yuan, Zike Xiang, Yang Tang, Buzhou |
| author_facet | Yan, Jianzhi Liu, Le Pan, Youcheng Chen, Shiwei Yuan, Zike Xiang, Yang Tang, Buzhou |
| contents | Chain-of-Thought (CoT) reasoning improves performance on complex tasks but introduces significant inference latency due to verbosity. We propose Multiround Adaptive Chain-of-Thought Compression (MACC), a framework that leverages the token elasticity phenomenon--where overly small token budgets can paradoxically increase output length--to progressively compress CoTs via multiround refinement. This adaptive strategy allows MACC to determine the optimal compression depth for each input. Our method achieves an average accuracy improvement of 5.6 percent over state-of-the-art baselines, while also reducing CoT length by an average of 47 tokens and significantly lowering latency. Furthermore, we show that test-time performance--accuracy and token length--can be reliably predicted using interpretable features like perplexity and compression rate on the training set. Evaluated across different models, our method enables efficient model selection and forecasting without repeated fine-tuning, demonstrating that CoT compression is both effective and predictable. Our code will be released in https://github.com/Leon221220/MACC. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_22144 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | From Long to Lean: Performance-aware and Adaptive Chain-of-Thought Compression via Multi-round Refinement Yan, Jianzhi Liu, Le Pan, Youcheng Chen, Shiwei Yuan, Zike Xiang, Yang Tang, Buzhou Computation and Language Artificial Intelligence Chain-of-Thought (CoT) reasoning improves performance on complex tasks but introduces significant inference latency due to verbosity. We propose Multiround Adaptive Chain-of-Thought Compression (MACC), a framework that leverages the token elasticity phenomenon--where overly small token budgets can paradoxically increase output length--to progressively compress CoTs via multiround refinement. This adaptive strategy allows MACC to determine the optimal compression depth for each input. Our method achieves an average accuracy improvement of 5.6 percent over state-of-the-art baselines, while also reducing CoT length by an average of 47 tokens and significantly lowering latency. Furthermore, we show that test-time performance--accuracy and token length--can be reliably predicted using interpretable features like perplexity and compression rate on the training set. Evaluated across different models, our method enables efficient model selection and forecasting without repeated fine-tuning, demonstrating that CoT compression is both effective and predictable. Our code will be released in https://github.com/Leon221220/MACC. |
| title | From Long to Lean: Performance-aware and Adaptive Chain-of-Thought Compression via Multi-round Refinement |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2509.22144 |