The End of Manual Decoding: Towards Truly End-to-End Language Models
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866918180231839744 |
|---|---|
| author | Wang, Zhichao Ma, Dongyang Huang, Xinting Cai, Deng Lan, Tian Xu, Jiahao Mi, Haitao Tang, Xiaoying Wang, Yan |
| author_facet | Wang, Zhichao Ma, Dongyang Huang, Xinting Cai, Deng Lan, Tian Xu, Jiahao Mi, Haitao Tang, Xiaoying Wang, Yan |
| contents | The "end-to-end" label for LLMs is a misnomer. In practice, they depend on a non-differentiable decoding process that requires laborious, hand-tuning of hyperparameters like temperature and top-p. This paper introduces AutoDeco, a novel architecture that enables truly "end-to-end" generation by learning to control its own decoding strategy. We augment the standard transformer with lightweight heads that, at each step, dynamically predict context-specific temperature and top-p values alongside the next-token logits. This approach transforms decoding into a parametric, token-level process, allowing the model to self-regulate its sampling strategy within a single forward pass.
Through extensive experiments on eight benchmarks, we demonstrate that AutoDeco not only significantly outperforms default decoding strategies but also achieves performance comparable to an oracle-tuned baseline derived from "hacking the test set"-a practical upper bound for any static method. Crucially, we uncover an emergent capability for instruction-based decoding control: the model learns to interpret natural language commands (e.g., "generate with low randomness") and adjusts its predicted temperature and top-p on a token-by-token basis, opening a new paradigm for steerable and interactive LLM decoding. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_26697 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | The End of Manual Decoding: Towards Truly End-to-End Language Models Wang, Zhichao Ma, Dongyang Huang, Xinting Cai, Deng Lan, Tian Xu, Jiahao Mi, Haitao Tang, Xiaoying Wang, Yan Computation and Language Artificial Intelligence The "end-to-end" label for LLMs is a misnomer. In practice, they depend on a non-differentiable decoding process that requires laborious, hand-tuning of hyperparameters like temperature and top-p. This paper introduces AutoDeco, a novel architecture that enables truly "end-to-end" generation by learning to control its own decoding strategy. We augment the standard transformer with lightweight heads that, at each step, dynamically predict context-specific temperature and top-p values alongside the next-token logits. This approach transforms decoding into a parametric, token-level process, allowing the model to self-regulate its sampling strategy within a single forward pass. Through extensive experiments on eight benchmarks, we demonstrate that AutoDeco not only significantly outperforms default decoding strategies but also achieves performance comparable to an oracle-tuned baseline derived from "hacking the test set"-a practical upper bound for any static method. Crucially, we uncover an emergent capability for instruction-based decoding control: the model learns to interpret natural language commands (e.g., "generate with low randomness") and adjusts its predicted temperature and top-p on a token-by-token basis, opening a new paradigm for steerable and interactive LLM decoding. |
| title | The End of Manual Decoding: Towards Truly End-to-End Language Models |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2510.26697 |