Copertina

M3SciQA: A Multi-Modal Multi-Document Scientific QA Benchmark for Evaluating Foundation Models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Li, Chuhan, Shangguan, Ziyao, Zhao, Yilun, Li, Deyuan, Liu, Yixin, Cohan, Arman
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Can Multimodal Foundation Models Understand Schematic Diagrams? An Empirical Study on Information-Seeking QA over Scientific Papers
di: Zhao, Yilun, et al.
Pubblicazione: (2025)

SciVer: Evaluating Foundation Models for Multimodal Scientific Claim Verification
di: Wang, Chengye, et al.
Pubblicazione: (2025)

TOMATO: Assessing Visual Temporal Reasoning Capabilities in Multimodal Foundation Models
di: Shangguan, Ziyao, et al.
Pubblicazione: (2024)

YaleNLP @ PerAnsSumm 2025: Multi-Perspective Integration via Mixture-of-Agents for Enhanced Healthcare QA Summarization
di: Jang, Dongsuk, et al.
Pubblicazione: (2025)

SciMDR: Advancing Scientific Multimodal Document Reasoning
di: Chen, Ziyu, et al.
Pubblicazione: (2026)

SciRAG: Adaptive, Citation-Aware, and Outline-Guided Retrieval and Synthesis for Scientific Literature
di: Ding, Hang, et al.
Pubblicazione: (2025)

MSRS: Evaluating Multi-Source Retrieval-Augmented Generation
di: Phanse, Rohan, et al.
Pubblicazione: (2025)

AbGen: Evaluating Large Language Models in Ablation Study Design and Evaluation for Scientific Research
di: Zhao, Yilun, et al.
Pubblicazione: (2025)

Q-Mirror: Unlocking the Multi-Modal Potential of Scientific Text-Only QA Pairs
di: Wang, Junying, et al.
Pubblicazione: (2025)

SciDQA: A Deep Reading Comprehension Dataset over Scientific Papers
di: Singh, Shruti, et al.
Pubblicazione: (2024)

Can LLMs Identify Critical Limitations within Scientific Research? A Systematic Evaluation on AI Research Papers
di: Xu, Zhijian, et al.
Pubblicazione: (2025)

On Evaluating LLM Alignment by Evaluating LLMs as Judges
di: Liu, Yixin, et al.
Pubblicazione: (2025)

Demystifying Scientific Problem-Solving in LLMs by Probing Knowledge and Reasoning
di: Li, Alan, et al.
Pubblicazione: (2025)

Investigating Data Contamination in Modern Benchmarks for Large Language Models
di: Deng, Chunyuan, et al.
Pubblicazione: (2023)

PuzzlePlex: Benchmarking Foundation Models on Reasoning and Planning with Puzzles
di: Long, Yitao, et al.
Pubblicazione: (2025)

SciTaRC: Benchmarking QA on Scientific Tabular Data that Requires Language Reasoning and Complex Computation
di: Wang, Hexuan, et al.
Pubblicazione: (2026)

MultiChartQA: Benchmarking Vision-Language Models on Multi-Chart Problems
di: Zhu, Zifeng, et al.
Pubblicazione: (2024)

SciCoQA: Quality Assurance for Scientific Paper--Code Alignment
di: Baumgärtner, Tim, et al.
Pubblicazione: (2026)

FinTrust: A Comprehensive Benchmark of Trustworthiness Evaluation in Finance Domain
di: Hu, Tiansheng, et al.
Pubblicazione: (2025)

Automatic Inter-document Multi-hop Scientific QA Generation
di: Lee, Seungmin, et al.
Pubblicazione: (2026)

HiQA: A Hierarchical Contextual Augmentation RAG for Multi-Documents QA
di: Chen, Xinyue, et al.
Pubblicazione: (2024)

Leave No Document Behind: Benchmarking Long-Context LLMs with Extended Multi-Doc QA
di: Wang, Minzheng, et al.
Pubblicazione: (2024)

HumanEval Pro and MBPP Pro: Evaluating Large Language Models on Self-invoking Code Generation
di: Yu, Zhaojian, et al.
Pubblicazione: (2024)

SAGE: Benchmarking and Improving Retrieval for Deep Research Agents
di: Hu, Tiansheng, et al.
Pubblicazione: (2026)

MMVU: Measuring Expert-Level Multi-Discipline Video Understanding
di: Zhao, Yilun, et al.
Pubblicazione: (2025)

SUCEA: Reasoning-Intensive Retrieval for Adversarial Fact-checking through Claim Decomposition and Editing
di: Liu, Hongjun, et al.
Pubblicazione: (2025)

SciArena: An Open Evaluation Platform for Non-Verifiable Scientific Literature-Grounded Tasks
di: Zhao, Yilun, et al.
Pubblicazione: (2025)

Benchmarking Generation and Evaluation Capabilities of Large Language Models for Instruction Controllable Summarization
di: Liu, Yixin, et al.
Pubblicazione: (2023)

HYBRIDMIND: Meta Selection of Natural Language and Symbolic Language for Enhanced LLM Reasoning
di: Han, Simeng, et al.
Pubblicazione: (2024)

Understanding Reference Policies in Direct Preference Optimization
di: Liu, Yixin, et al.
Pubblicazione: (2024)

DocMath-Eval: Evaluating Math Reasoning Capabilities of LLMs in Understanding Long and Specialized Documents
di: Zhao, Yilun, et al.
Pubblicazione: (2023)

FinLFQA: Evaluating Attributed Text Generation of LLMs in Financial Long-Form Question Answering
di: Long, Yitao, et al.
Pubblicazione: (2025)

MCTS-RAG: Enhancing Retrieval-Augmented Generation with Monte Carlo Tree Search
di: Hu, Yunhai, et al.
Pubblicazione: (2025)

FanOutQA: A Multi-Hop, Multi-Document Question Answering Benchmark for Large Language Models
di: Zhu, Andrew, et al.
Pubblicazione: (2024)

Navigating Large-Scale Document Collections: MuDABench for Multi-Document Analytical QA
di: Li, Zhanli, et al.
Pubblicazione: (2026)

Table-R1: Inference-Time Scaling for Table Reasoning
di: Yang, Zheyuan, et al.
Pubblicazione: (2025)

Inter-Passage Verification for Multi-evidence Multi-answer QA
di: Chen, Bingsen, et al.
Pubblicazione: (2025)

PHYSICS: Benchmarking Foundation Models on University-Level Physics Problem Solving
di: Feng, Kaiyue, et al.
Pubblicazione: (2025)

Scientific QA System with Verifiable Answers
di: Ljajić, Adela, et al.
Pubblicazione: (2024)

SciEval: A Multi-Level Large Language Model Evaluation Benchmark for Scientific Research
di: Sun, Liangtai, et al.
Pubblicazione: (2023)