Aligning with Your Own Voice: Self-Corrected Preference Learning for Hallucination Mitigation in LVLMs
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910169553698816 |
|---|---|
| author | Lim, Byeonggeuk Yun, JungMin Kwon, Junehyoung Kim, Kyeonghyun Kim, YoungBin |
| author_facet | Lim, Byeonggeuk Yun, JungMin Kwon, Junehyoung Kim, Kyeonghyun Kim, YoungBin |
| contents | Large Vision-Language Models (LVLMs) frequently suffer from hallucinations. Existing preference learning-based approaches largely rely on proprietary models to construct preference datasets. We identify that this reliance introduces a distributional mismatch between the proprietary and target models that hinders efficient alignment. To address this, we propose Alignment via VErified Self-correction DPO (AVES-DPO), a framework that aligns LVLMs using in-distribution data derived from the model's intrinsic knowledge. Our approach employs a consensus-based verification mechanism to diagnose diverse hallucinations and guides the model to self-correct, thereby generating preference pairs strictly compatible with its internal distribution. Extensive experiments demonstrate that AVES-DPO surpasses existing baselines in hallucination mitigation while requiring only 5.2k samples. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_24395 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Aligning with Your Own Voice: Self-Corrected Preference Learning for Hallucination Mitigation in LVLMs Lim, Byeonggeuk Yun, JungMin Kwon, Junehyoung Kim, Kyeonghyun Kim, YoungBin Artificial Intelligence Large Vision-Language Models (LVLMs) frequently suffer from hallucinations. Existing preference learning-based approaches largely rely on proprietary models to construct preference datasets. We identify that this reliance introduces a distributional mismatch between the proprietary and target models that hinders efficient alignment. To address this, we propose Alignment via VErified Self-correction DPO (AVES-DPO), a framework that aligns LVLMs using in-distribution data derived from the model's intrinsic knowledge. Our approach employs a consensus-based verification mechanism to diagnose diverse hallucinations and guides the model to self-correct, thereby generating preference pairs strictly compatible with its internal distribution. Extensive experiments demonstrate that AVES-DPO surpasses existing baselines in hallucination mitigation while requiring only 5.2k samples. |
| title | Aligning with Your Own Voice: Self-Corrected Preference Learning for Hallucination Mitigation in LVLMs |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2604.24395 |