Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2605.30251 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918529945567232 |
|---|---|
| author | Lin, Zizhuo Liu, Quanling Quan, Jinsheng Zhang, Chao Zhu, Yifan Shi, Xing Xu, Jingtao Li, Zhihui Luo, Yawei |
| author_facet | Lin, Zizhuo Liu, Quanling Quan, Jinsheng Zhang, Chao Zhu, Yifan Shi, Xing Xu, Jingtao Li, Zhihui Luo, Yawei |
| contents | Large language models (LLMs) often solve a task when all instructions are given in a single prompt, but fail when the same information is revealed gradually across turns. When a clean FULL prompt and a RAW-SHARDED conversation contain the same complete user evidence, the model should still arrive at the same answer. We argue that a key reason for this gap is self-anchored drift: responses produced under partial information introduce unsupported assumptions, and those assumptions later distort the final answer. To reduce this effect, we propose Canonical-Context On-Policy Distillation (CCOPD). During training, the same base model is used in two roles: a frozen teacher conditioned on the clean FULL prompt and a trainable student that receives the same evidence incrementally through a multi-turn conversation; CCOPD aligns the student's behavior on its own trajectories with the teacher's canonical full-context behavior. Trained only on math problem conversations, CCOPD yields a 32\% average relative improvement in RAW-SHARDED performance over the original base model across math and five zero-shot out-of-domain task families, while largely preserving full-context performance. Further analyses suggest that CCOPD strengthens grounding in user evidence and reduces sensitivity to contamination from earlier assistant turns. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_30251 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Same Evidence, Different Answers: Canonical-Context On-Policy Distillation for Multi-Turn Language Models Lin, Zizhuo Liu, Quanling Quan, Jinsheng Zhang, Chao Zhu, Yifan Shi, Xing Xu, Jingtao Li, Zhihui Luo, Yawei Computation and Language Artificial Intelligence Large language models (LLMs) often solve a task when all instructions are given in a single prompt, but fail when the same information is revealed gradually across turns. When a clean FULL prompt and a RAW-SHARDED conversation contain the same complete user evidence, the model should still arrive at the same answer. We argue that a key reason for this gap is self-anchored drift: responses produced under partial information introduce unsupported assumptions, and those assumptions later distort the final answer. To reduce this effect, we propose Canonical-Context On-Policy Distillation (CCOPD). During training, the same base model is used in two roles: a frozen teacher conditioned on the clean FULL prompt and a trainable student that receives the same evidence incrementally through a multi-turn conversation; CCOPD aligns the student's behavior on its own trajectories with the teacher's canonical full-context behavior. Trained only on math problem conversations, CCOPD yields a 32\% average relative improvement in RAW-SHARDED performance over the original base model across math and five zero-shot out-of-domain task families, while largely preserving full-context performance. Further analyses suggest that CCOPD strengthens grounding in user evidence and reduces sensitivity to contamination from earlier assistant turns. |
| title | Same Evidence, Different Answers: Canonical-Context On-Policy Distillation for Multi-Turn Language Models |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2605.30251 |