Cover Image

FIRE-Bench: Evaluating Agents on the Rediscovery of Scientific Insights

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Zhen, Bai, Fan, Luo, Zhongyan, Su, Jinyan, Sun, Kaiser, Yu, Xinle, Liu, Jieyuan, Zhou, Kun, Cardie, Claire, Dredze, Mark, Xing, Eric P., Hu, Zhiting
Format:	Preprint
Published:	2026
Subjects:	Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Knowing but Not Showing: LLMs Recognize Ambiguity but Rarely Ask Clarifying Questions
by: Su, Jinyan, et al.
Published: (2026)

Thinking Fast and Right: Balancing Accuracy and Reasoning Length with Adaptive Rewards
by: Su, Jinyan, et al.
Published: (2025)

Multi-Hop Question Answering: When Can Humans Help, and Where do They Struggle?
by: Su, Jinyan, et al.
Published: (2025)

Adapting Fake News Detection to the Era of Large Language Models
by: Su, Jinyan, et al.
Published: (2023)

Corpus Poisoning via Approximate Greedy Gradient Descent
by: Su, Jinyan, et al.
Published: (2024)

Task Matters: Knowledge Requirements Shape LLM Responses to Context-Memory Conflict
by: Sun, Kaiser, et al.
Published: (2025)

Between Underthinking and Overthinking: An Empirical Study of Reasoning Length and correctness in LLMs
by: Su, Jinyan, et al.
Published: (2025)

Fast or Better? Balancing Accuracy and Cost in Retrieval-Augmented Generation with Flexible User Control
by: Su, Jinyan, et al.
Published: (2025)

CLIPer: Tailoring Diverse User Preference via Classifier-Guided Inference-Time Personalization
by: Su, Jinyan, et al.
Published: (2026)

Towards More Robust Retrieval-Augmented Generation: Evaluating RAG Under Adversarial Poisoning Attacks
by: Su, Jinyan, et al.
Published: (2024)

Amuro and Char: Analyzing the Relationship between Pre-Training and Fine-Tuning of Large Language Models
by: Sun, Kaiser, et al.
Published: (2024)

Reasoning Court: Combining Reasoning, Action, and Judgment for Multi-Hop Reasoning
by: Wu, Jingtian, et al.
Published: (2025)

LLMs are Better Than You Think: Label-Guided In-Context Learning for Named Entity Recognition
by: Bai, Fan, et al.
Published: (2025)

Evaluating the Evaluators: Are readability metrics good measures of readability?
by: Cachola, Isabel, et al.
Published: (2025)

Reading, Not Thinking: Understanding and Bridging the Modality Gap When Text Becomes Pixels in Multimodal LLMs
by: Sun, Kaiser, et al.
Published: (2026)

On the Failure of Latent State Persistence in Large Language Models
by: Huang, Jen-tse, et al.
Published: (2025)

Evaluating Implicit Biases in LLM Reasoning through Logic Grid Puzzles
by: Jahara, Fatima, et al.
Published: (2025)

Decentralized Arena: Towards Democratic and Scalable Automatic Evaluation of Language Models
by: Yin, Yanbin, et al.
Published: (2025)

HAPO: Training Language Models to Reason Concisely via History-Aware Policy Optimization
by: Huang, Chengyu, et al.
Published: (2025)

Transferring Fairness using Multi-Task Learning with Limited Demographic Information
by: Aguirre, Carlos, et al.
Published: (2023)

scPilot: Large Language Model Reasoning Toward Automated Single-Cell Analysis and Discovery
by: Gao, Yiming, et al.
Published: (2026)

Rediscovery
by: Banchio, Martino, et al.
Published: (2025)

MASH: Modeling Abstention via Selective Help-Seeking
by: Gul, Mustafa Omer, et al.
Published: (2025)

FairMT-Bench: Benchmarking Fairness for Multi-turn Dialogue in Conversational LLMs
by: Fan, Zhiting, et al.
Published: (2024)

Can one size fit all?: Measuring Failure in Multi-Document Summarization Domain Transfer
by: DeLucia, Alexandra, et al.
Published: (2025)

RAG LLMs are Not Safer: A Safety Analysis of Retrieval-Augmented Generation for Large Language Models
by: An, Bang, et al.
Published: (2025)

CocoaBench: Evaluating Unified Digital Agents in the Wild
by: CocoaBench Team, et al.
Published: (2026)

Give me Some Hard Questions: Synthetic Data Generation for Clinical QA
by: Bai, Fan, et al.
Published: (2024)

Schema-Driven Information Extraction from Heterogeneous Tables
by: Bai, Fan, et al.
Published: (2023)

Driving Strategy Using an Improved Ant Colony System for Energy‐Efficient Train
by: Chengda Yang, et al.
Published: (2024)

How Far Are We From True Auto-Research?
by: Zhang, Zhengxin, et al.
Published: (2026)

SciVisAgentBench: A Benchmark for Evaluating Scientific Data Analysis and Visualization Agents
by: Ai, Kuangshi, et al.
Published: (2026)

Can Optimization Trajectories Explain Multi-Task Transfer?
by: Mueller, David, et al.
Published: (2024)

MedInsightBench: Evaluating Medical Analytics Agents Through Multi-Step Insight Discovery in Multimodal Medical Data
by: Zhu, Zhenghao, et al.
Published: (2025)

AutoResearchBench: Benchmarking AI Agents on Complex Scientific Literature Discovery
by: Xiong, Lei, et al.
Published: (2026)

CellMaster: Collaborative Cell Type Annotation in Single-Cell Analysis
by: Wang, Zhen, et al.
Published: (2026)

AgentBench: Evaluating LLMs as Agents
by: Liu, Xiao, et al.
Published: (2023)

NewtonBench: Benchmarking Generalizable Scientific Law Discovery in LLM Agents
by: Zheng, Tianshi, et al.
Published: (2025)

Critiques of World Models
by: Xing, Eric, et al.
Published: (2025)

General Agentic Planning Through Simulative Reasoning with World Models
by: Deng, Mingkai, et al.
Published: (2025)