Decomposition-Enhanced Training for Post-Hoc Attributions In Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911250869387264 |
|---|---|
| author | Balasubramanian, Sriram Basu, Samyadeep Goswami, Koustava Rossi, Ryan Manjunatha, Varun Santhosh, Roshan Zhang, Ruiyi Feizi, Soheil Lipka, Nedim |
| author_facet | Balasubramanian, Sriram Basu, Samyadeep Goswami, Koustava Rossi, Ryan Manjunatha, Varun Santhosh, Roshan Zhang, Ruiyi Feizi, Soheil Lipka, Nedim |
| contents | Large language models (LLMs) are increasingly used for long-document question answering, where reliable attribution to sources is critical for trust. Existing post-hoc attribution methods work well for extractive QA but struggle in multi-hop, abstractive, and semi-extractive settings, where answers synthesize information across passages. To address these challenges, we argue that post-hoc attribution can be reframed as a reasoning problem, where answers are decomposed into constituent units, each tied to specific context. We first show that prompting models to generate such decompositions alongside attributions improves performance. Building on this, we introduce DecompTune, a post-training method that teaches models to produce answer decompositions as intermediate reasoning steps. We curate a diverse dataset of complex QA tasks, annotated with decompositions by a strong LLM, and post-train Qwen-2.5 (7B and 14B) using a two-stage SFT + GRPO pipeline with task-specific curated rewards. Across extensive experiments and ablations, DecompTune substantially improves attribution quality, outperforming prior methods and matching or exceeding state-of-the-art frontier models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_25766 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Decomposition-Enhanced Training for Post-Hoc Attributions In Language Models Balasubramanian, Sriram Basu, Samyadeep Goswami, Koustava Rossi, Ryan Manjunatha, Varun Santhosh, Roshan Zhang, Ruiyi Feizi, Soheil Lipka, Nedim Computation and Language Large language models (LLMs) are increasingly used for long-document question answering, where reliable attribution to sources is critical for trust. Existing post-hoc attribution methods work well for extractive QA but struggle in multi-hop, abstractive, and semi-extractive settings, where answers synthesize information across passages. To address these challenges, we argue that post-hoc attribution can be reframed as a reasoning problem, where answers are decomposed into constituent units, each tied to specific context. We first show that prompting models to generate such decompositions alongside attributions improves performance. Building on this, we introduce DecompTune, a post-training method that teaches models to produce answer decompositions as intermediate reasoning steps. We curate a diverse dataset of complex QA tasks, annotated with decompositions by a strong LLM, and post-train Qwen-2.5 (7B and 14B) using a two-stage SFT + GRPO pipeline with task-specific curated rewards. Across extensive experiments and ablations, DecompTune substantially improves attribution quality, outperforming prior methods and matching or exceeding state-of-the-art frontier models. |
| title | Decomposition-Enhanced Training for Post-Hoc Attributions In Language Models |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2510.25766 |