Q-Mirror: Unlocking the Multi-Modal Potential of Scientific Text-Only QA Pairs
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908567981785088 |
|---|---|
| author | Wang, Junying Zhang, Zicheng Shen, Ye Wu, Yalun Liang, Yingji Guo, Yijin Wen, Farong Li, Wenzhe Zhao, Xuezhi Jia, Qi Zhai, Guangtao |
| author_facet | Wang, Junying Zhang, Zicheng Shen, Ye Wu, Yalun Liang, Yingji Guo, Yijin Wen, Farong Li, Wenzhe Zhao, Xuezhi Jia, Qi Zhai, Guangtao |
| contents | High-quality, multi-modal benchmarks are crucial for advancing scientific reasoning in large models yet their manual creation is costly and unscalable. To address this bottleneck, we explore the potential for transforming Text-Only QA Pairs (TQAs) into high-quality Multi-Modal QA Pairs (MMQAs), which include three parts: 1) Task Definition \& Evaluation Rubric: We develop a TQA-to-MMQA framework and establish a comprehensive, multi-dimensional MMQA quality rubric that provides principles for the transformation. 2) Benchmark Construction: Then we construct two extensive benchmarks to rigorously evaluate state-of-the-art generation \& understanding models on the distinct tasks of MMQA generation \& MMQA quality evaluation. 3) Preliminary Solution: We develop an agentic system (Q-Mirror), which operationalizes our framework by integrating MMQA generation and evaluation into a closed loop for iterative refinement. Our experiments show that while state-of-the-art models can generate MMQAs, their outputs still leave substantial gaps, underscoring the need for reliable evaluation. We further demonstrate that top-tier understanding models align closely with human judgment in MMQA quality assessment. Leveraging both insights, the Q-Mirror agent raises average scores from 78.90 to 85.22 and pass rates from 72\% to 95\%, offering a practical path to large-scale scientific benchmarks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_24297 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Q-Mirror: Unlocking the Multi-Modal Potential of Scientific Text-Only QA Pairs Wang, Junying Zhang, Zicheng Shen, Ye Wu, Yalun Liang, Yingji Guo, Yijin Wen, Farong Li, Wenzhe Zhao, Xuezhi Jia, Qi Zhai, Guangtao Computation and Language Artificial Intelligence High-quality, multi-modal benchmarks are crucial for advancing scientific reasoning in large models yet their manual creation is costly and unscalable. To address this bottleneck, we explore the potential for transforming Text-Only QA Pairs (TQAs) into high-quality Multi-Modal QA Pairs (MMQAs), which include three parts: 1) Task Definition \& Evaluation Rubric: We develop a TQA-to-MMQA framework and establish a comprehensive, multi-dimensional MMQA quality rubric that provides principles for the transformation. 2) Benchmark Construction: Then we construct two extensive benchmarks to rigorously evaluate state-of-the-art generation \& understanding models on the distinct tasks of MMQA generation \& MMQA quality evaluation. 3) Preliminary Solution: We develop an agentic system (Q-Mirror), which operationalizes our framework by integrating MMQA generation and evaluation into a closed loop for iterative refinement. Our experiments show that while state-of-the-art models can generate MMQAs, their outputs still leave substantial gaps, underscoring the need for reliable evaluation. We further demonstrate that top-tier understanding models align closely with human judgment in MMQA quality assessment. Leveraging both insights, the Q-Mirror agent raises average scores from 78.90 to 85.22 and pass rates from 72\% to 95\%, offering a practical path to large-scale scientific benchmarks. |
| title | Q-Mirror: Unlocking the Multi-Modal Potential of Scientific Text-Only QA Pairs |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2509.24297 |