A Picture Is Worth a Graph: A Blueprint Debate Paradigm for Multimodal Reasoning
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929450279501824 |
|---|---|
| author | Zheng, Changmeng Liang, Dayong Zhang, Wengyu Wei, Xiao-Yong Chua, Tat-Seng Li, Qing |
| author_facet | Zheng, Changmeng Liang, Dayong Zhang, Wengyu Wei, Xiao-Yong Chua, Tat-Seng Li, Qing |
| contents | This paper presents a pilot study aimed at introducing multi-agent debate into multimodal reasoning. The study addresses two key challenges: the trivialization of opinions resulting from excessive summarization and the diversion of focus caused by distractor concepts introduced from images. These challenges stem from the inductive (bottom-up) nature of existing debating schemes. To address the issue, we propose a deductive (top-down) debating approach called Blueprint Debate on Graphs (BDoG). In BDoG, debates are confined to a blueprint graph to prevent opinion trivialization through world-level summarization. Moreover, by storing evidence in branches within the graph, BDoG mitigates distractions caused by frequent but irrelevant concepts. Extensive experiments validate that BDoG is able to achieve state-of-the-art results in ScienceQA and MMBench with significant improvements over previous methods. The source code can be accessed at https://github.com/thecharm/BDoG. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_14972 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Picture Is Worth a Graph: A Blueprint Debate Paradigm for Multimodal Reasoning Zheng, Changmeng Liang, Dayong Zhang, Wengyu Wei, Xiao-Yong Chua, Tat-Seng Li, Qing Artificial Intelligence Computation and Language Multiagent Systems Multimedia This paper presents a pilot study aimed at introducing multi-agent debate into multimodal reasoning. The study addresses two key challenges: the trivialization of opinions resulting from excessive summarization and the diversion of focus caused by distractor concepts introduced from images. These challenges stem from the inductive (bottom-up) nature of existing debating schemes. To address the issue, we propose a deductive (top-down) debating approach called Blueprint Debate on Graphs (BDoG). In BDoG, debates are confined to a blueprint graph to prevent opinion trivialization through world-level summarization. Moreover, by storing evidence in branches within the graph, BDoG mitigates distractions caused by frequent but irrelevant concepts. Extensive experiments validate that BDoG is able to achieve state-of-the-art results in ScienceQA and MMBench with significant improvements over previous methods. The source code can be accessed at https://github.com/thecharm/BDoG. |
| title | A Picture Is Worth a Graph: A Blueprint Debate Paradigm for Multimodal Reasoning |
| topic | Artificial Intelligence Computation and Language Multiagent Systems Multimedia |
| url | https://arxiv.org/abs/2403.14972 |