Context-Fidelity Boosting: Enhancing Faithful Generation through Watermark-Inspired Decoding
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910162297552896 |
|---|---|
| author | Zhang, Weixu Ye, Fanghua Gao, Qiang Li, Jian Wu, Haolun Tian, Yuxing Duan, Sijing Du, Nan Li, Xiaolong Liu, Xue |
| author_facet | Zhang, Weixu Ye, Fanghua Gao, Qiang Li, Jian Wu, Haolun Tian, Yuxing Duan, Sijing Du, Nan Li, Xiaolong Liu, Xue |
| contents | Large language models (LLMs) often produce content that contradicts or overlooks information provided in the input context, a phenomenon known as faithfulness hallucination. In this paper, we propose Context-Fidelity Boosting (CFB), a lightweight and general decoding-time framework that reduces such hallucinations by increasing the generation probability of source-supported tokens. Motivated by logit-shaping principles from watermarking techniques, CFB applies additive token-level logit adjustments based on a token's degree of support from the input context. Specifically, we develop three boosting strategies: static boosting, which applies a fixed bias to source-supported tokens; context-aware boosting, which scales this bias using the divergence between next-token distributions with and without context; and token-aware boosting, which further redistributes the adaptive bias according to local relevance estimated from source-position attention and source-scoped semantic similarity. CFB requires no retraining or architectural changes, making it compatible with a wide range of LLMs. Experiments on summarization and question answering tasks across multiple open-source LLMs show that CFB consistently improves faithfulness metrics with minimal generation overhead. Our implementation is fully open-sourced. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_22335 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Context-Fidelity Boosting: Enhancing Faithful Generation through Watermark-Inspired Decoding Zhang, Weixu Ye, Fanghua Gao, Qiang Li, Jian Wu, Haolun Tian, Yuxing Duan, Sijing Du, Nan Li, Xiaolong Liu, Xue Computation and Language Large language models (LLMs) often produce content that contradicts or overlooks information provided in the input context, a phenomenon known as faithfulness hallucination. In this paper, we propose Context-Fidelity Boosting (CFB), a lightweight and general decoding-time framework that reduces such hallucinations by increasing the generation probability of source-supported tokens. Motivated by logit-shaping principles from watermarking techniques, CFB applies additive token-level logit adjustments based on a token's degree of support from the input context. Specifically, we develop three boosting strategies: static boosting, which applies a fixed bias to source-supported tokens; context-aware boosting, which scales this bias using the divergence between next-token distributions with and without context; and token-aware boosting, which further redistributes the adaptive bias according to local relevance estimated from source-position attention and source-scoped semantic similarity. CFB requires no retraining or architectural changes, making it compatible with a wide range of LLMs. Experiments on summarization and question answering tasks across multiple open-source LLMs show that CFB consistently improves faithfulness metrics with minimal generation overhead. Our implementation is fully open-sourced. |
| title | Context-Fidelity Boosting: Enhancing Faithful Generation through Watermark-Inspired Decoding |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2604.22335 |