Image de couverture de livre

Can LLMs Identify Critical Limitations within Scientific Research? A Systematic Evaluation on AI Research Papers

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Xu, Zhijian, Zhao, Yilun, Patwardhan, Manasi, Vig, Lovekesh, Cohan, Arman
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

IRIS: Interactive Research Ideation System for Accelerating Scientific Discovery
par: Garikaparthi, Aniketh, et autres
Publié: (2025)

AbGen: Evaluating Large Language Models in Ablation Study Design and Evaluation for Scientific Research
par: Zhao, Yilun, et autres
Publié: (2025)

MIR: Methodology Inspiration Retrieval for Scientific Research Problems
par: Garikaparthi, Aniketh, et autres
Publié: (2025)

Acceleron: A Tool to Accelerate Research Ideation
par: Nigam, Harshit, et autres
Publié: (2024)

SciRAG: Adaptive, Citation-Aware, and Outline-Guided Retrieval and Synthesis for Scientific Literature
par: Ding, Hang, et autres
Publié: (2025)

DBRouting: Routing End User Queries to Databases for Answerability
par: Mandal, Priyangshu, et autres
Publié: (2025)

ResearchCodeAgent: An LLM Multi-Agent System for Automated Codification of Research Methodologies
par: Gandhi, Shubham, et autres
Publié: (2025)

SciMDR: Advancing Scientific Multimodal Document Reasoning
par: Chen, Ziyu, et autres
Publié: (2026)

Can AI Be a Good Peer Reviewer? A Survey of Peer Review Process, Evaluation, and the Future
par: Wu, Sihong, et autres
Publié: (2026)

BudgetMLAgent: A Cost-Effective LLM Multi-Agent system for Automating Machine Learning Tasks
par: Gandhi, Shubham, et autres
Publié: (2024)

ResearchGym: Evaluating Language Model Agents on Real-World AI Research
par: Garikaparthi, Aniketh, et autres
Publié: (2026)

Can Multimodal Foundation Models Understand Schematic Diagrams? An Empirical Study on Information-Seeking QA over Scientific Papers
par: Zhao, Yilun, et autres
Publié: (2025)

RbtAct: Rebuttal as Supervision for Actionable Review Feedback Generation
par: Wu, Sihong, et autres
Publié: (2026)

Retrieval and Augmentation of Domain Knowledge for Text-to-SQL Semantic Parsing
par: Patwardhan, Manasi, et autres
Publié: (2025)

Translating Across Cultures: LLMs for Intralingual Cultural Adaptation
par: Singh, Pushpdeep, et autres
Publié: (2024)

SciVer: Evaluating Foundation Models for Multimodal Scientific Claim Verification
par: Wang, Chengye, et autres
Publié: (2025)

REVERE: Reflective Evolving Research Engineer for Scientific Workflows
par: Gangireddi, Balaji Dinesh, et autres
Publié: (2026)

SciDQA: A Deep Reading Comprehension Dataset over Scientific Papers
par: Singh, Shruti, et autres
Publié: (2024)

FinLFQA: Evaluating Attributed Text Generation of LLMs in Financial Long-Form Question Answering
par: Long, Yitao, et autres
Publié: (2025)

SAGE: Benchmarking and Improving Retrieval for Deep Research Agents
par: Hu, Tiansheng, et autres
Publié: (2026)

M3SciQA: A Multi-Modal Multi-Document Scientific QA Benchmark for Evaluating Foundation Models
par: Li, Chuhan, et autres
Publié: (2024)

Teaching Language Models to Forecast Research Success Through Comparative Idea Evaluation
par: Mule, Srujan P, et autres
Publié: (2026)

MCTS-RAG: Enhancing Retrieval-Augmented Generation with Monte Carlo Tree Search
par: Hu, Yunhai, et autres
Publié: (2025)

AlphaResearch: Accelerating New Algorithm Discovery with Language Models
par: Yu, Zhaojian, et autres
Publié: (2025)

Table-R1: Inference-Time Scaling for Table Reasoning
par: Yang, Zheyuan, et autres
Publié: (2025)

HumanEval Pro and MBPP Pro: Evaluating Large Language Models on Self-invoking Code Generation
par: Yu, Zhaojian, et autres
Publié: (2024)

SUCEA: Reasoning-Intensive Retrieval for Adversarial Fact-checking through Claim Decomposition and Editing
par: Liu, Hongjun, et autres
Publié: (2025)

Demystifying Scientific Problem-Solving in LLMs by Probing Knowledge and Reasoning
par: Li, Alan, et autres
Publié: (2025)

On Evaluating LLM Alignment by Evaluating LLMs as Judges
par: Liu, Yixin, et autres
Publié: (2025)

FinTrust: A Comprehensive Benchmark of Trustworthiness Evaluation in Finance Domain
par: Hu, Tiansheng, et autres
Publié: (2025)

Rethinking Reasoning-Intensive Retrieval: Evaluating and Advancing Retrievers in Agentic Search Systems
par: Zhao, Yilun, et autres
Publié: (2026)

Patient-Similarity Cohort Reasoning in Clinical Text-to-SQL
par: Shen, Yifei, et autres
Publié: (2026)

LimRank: Less is More for Reasoning-Intensive Information Reranking
par: Song, Tingyu, et autres
Publié: (2025)

PaperBench: Evaluating AI's Ability to Replicate AI Research
par: Starace, Giulio, et autres
Publié: (2025)

MSRS: Evaluating Multi-Source Retrieval-Augmented Generation
par: Phanse, Rohan, et autres
Publié: (2025)

Z1: Efficient Test-time Scaling with Code
par: Yu, Zhaojian, et autres
Publié: (2025)

Investigating Data Contamination in Modern Benchmarks for Large Language Models
par: Deng, Chunyuan, et autres
Publié: (2023)

FinanceMath: Knowledge-Intensive Math Reasoning in Finance Domains
par: Zhao, Yilun, et autres
Publié: (2023)

DocMath-Eval: Evaluating Math Reasoning Capabilities of LLMs in Understanding Long and Specialized Documents
par: Zhao, Yilun, et autres
Publié: (2023)

Diffusion vs. Autoregressive Language Models: A Text Embedding Perspective
par: Zhang, Siyue, et autres
Publié: (2025)