Breaking the Illusion: When Positive Meets Negative in Multimodal Decoding
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915989857239040 |
|---|---|
| author | Jiang, Yubo An, Yitong Yang, Xin Wuerkaixi, Abudukelimu Cheng, Xuxin Xie, Fengying Jiang, Zhiguo Liu, Cao Zeng, Ke Zhang, Haopeng |
| author_facet | Jiang, Yubo An, Yitong Yang, Xin Wuerkaixi, Abudukelimu Cheng, Xuxin Xie, Fengying Jiang, Zhiguo Liu, Cao Zeng, Ke Zhang, Haopeng |
| contents | Vision-Language Models (VLMs) are frequently undermined by object hallucination, generating content that contradicts visual reality, due to an over-reliance on linguistic priors. We introduce Positive-and-Negative Decoding (PND), a training-free inference framework that intervenes directly in the decoding process to enforce visual fidelity. PND is motivated by our finding of an attention imbalance in VLMs, where visual features are under-weighted. Our framework introduces a dual-path contrast: a positive path that amplifies visual evidence and a negative path that constructs counterfactuals to penalize prior-dominant generation. By contrasting outputs from both paths during decoding, PND steers generation toward visually grounded results. Experiments on POPE, MME, and CHAIR demonstrate state-of-the-art performance without retraining. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_06679 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Breaking the Illusion: When Positive Meets Negative in Multimodal Decoding Jiang, Yubo An, Yitong Yang, Xin Wuerkaixi, Abudukelimu Cheng, Xuxin Xie, Fengying Jiang, Zhiguo Liu, Cao Zeng, Ke Zhang, Haopeng Machine Learning Vision-Language Models (VLMs) are frequently undermined by object hallucination, generating content that contradicts visual reality, due to an over-reliance on linguistic priors. We introduce Positive-and-Negative Decoding (PND), a training-free inference framework that intervenes directly in the decoding process to enforce visual fidelity. PND is motivated by our finding of an attention imbalance in VLMs, where visual features are under-weighted. Our framework introduces a dual-path contrast: a positive path that amplifies visual evidence and a negative path that constructs counterfactuals to penalize prior-dominant generation. By contrasting outputs from both paths during decoding, PND steers generation toward visually grounded results. Experiments on POPE, MME, and CHAIR demonstrate state-of-the-art performance without retraining. |
| title | Breaking the Illusion: When Positive Meets Negative in Multimodal Decoding |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2605.06679 |