Cover Image

FlagEval Findings Report: A Preliminary Evaluation of Large Reasoning Models on Automatically Verifiable Textual and Visual Questions

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Qin, Bowen, Yue, Chen, Yin, Fang, Wang, Hui, Yao, JG, Liu, Jiakang, Zheng, Jing-Shu, Chen, Miguel Hu, Xuan, Richeng, Meng, Shibei, Zhou, Shiqi, Dai, Teng, Ren, Tong-Shuai, Cui, Wei, Yang, Xi, Du, Xialin, Xu, Xiaojing, Sun, Xue, Li, Xuejing, Liu, Yaming, Liu, Yesheng, Liu, Ying, Lin, Yonghua, Zhao, Yu, Zhang, Yunduo, Luo, Yuwen, He, Zheqi, He, Zhiyuan, Wang, Zhongyuan
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Computer Vision and Pattern Recognition Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

FlagEvalMM: A Flexible Framework for Comprehensive Multimodal Model Evaluation
by: He, Zheqi, et al.
Published: (2025)

Do Vision-Language Models Measure Up? Benchmarking Visual Measurement Reading with MeasureBench
by: Lin, Fenfen, et al.
Published: (2025)

Beyond Multiple Choice: Verifiable OpenQA for Robust Vision-Language RFT
by: Liu, Yesheng, et al.
Published: (2025)

DFlash: Block Diffusion for Flash Speculative Decoding
by: Chen, Jian, et al.
Published: (2026)

CMMU: A Benchmark for Chinese Multi-modal Multi-type Question Understanding and Reasoning
by: He, Zheqi, et al.
Published: (2024)

SEMPose: A Single End-to-end Network for Multi-object Pose Estimation
by: Liu, Xin, et al.
Published: (2024)

Nonparametric learning of heterogeneous graphical model on network-linked data
by: Wang, Yuwen, et al.
Published: (2025)

Pt Sub‐Nanoclusters on ZIF‐Derived Ultrafine Cd x Zn 1− x S Solid Solution: High‐Performance Photoreforming Lactic Acid to H 2 and Value‐Added Chemicals
by: Feng Liu, et al.
Published: (2025)

Front Cover
by: Feng Liu, et al.
Published: (2026)

Environmental Threats Increase Consumers' Purchase Intention of Cause‐Related Marketing Products Through Collective Fear and Collective Guilt: A Perspective of Group‐Based Emotions
by: Yaming Wang, et al.
Published: (2025)

Structuring GUI Elements through Vision Language Models: Towards Action Space Generation
by: Xu, Yi, et al.
Published: (2025)

Link Prediction on Textual Edge Graphs
by: Ling, Chen, et al.
Published: (2024)

From Solving to Verifying: A Unified Objective for Robust Reasoning in LLMs
by: Wang, Xiaoxuan, et al.
Published: (2025)

DrugR: Optimizing Molecular Drugs through LLM-based Explicit Reasoning
by: Liu, Haoran, et al.
Published: (2026)

GeoLoom: High-quality Geometric Diagram Generation from Textual Input
by: Wei, Xiaojing, et al.
Published: (2025)

Learning Getting-Up Policies for Real-World Humanoid Robots
by: He, Xialin, et al.
Published: (2025)

Fast Adversarial Training against Textual Adversarial Attacks
by: Yang, Yichen, et al.
Published: (2024)

On the Universal Truthfulness Hyperplane Inside LLMs
by: Liu, Junteng, et al.
Published: (2024)

OmniEval: A Benchmark for Evaluating Omni-modal Models with Visual, Auditory, and Textual Inputs
by: Zhang, Yiman, et al.
Published: (2025)

Virgo: A Preliminary Exploration on Reproducing o1-like MLLM
by: Du, Yifan, et al.
Published: (2025)

Symbol Alphabets in QCD and Flag Cluster Algebras
by: Pokraka, Andrzej, et al.
Published: (2025)

NOVER: Incentive Training for Language Models via Verifier-Free Reinforcement Learning
by: Liu, Wei, et al.
Published: (2025)

LaoBench: A Large-Scale Multidimensional Lao Benchmark for Large Language Models
by: Gao, Jian, et al.
Published: (2025)

Quantifying Multimodal Capabilities: Formal Generalization Guarantees in Pairwise Metric Learning
by: Zhou, Richeng, et al.
Published: (2026)

SciEval: A Benchmark for Automatic Evaluation of K-12 Science Instructional Materials
by: Li, Zhaohui, et al.
Published: (2026)

Trust, But Verify: A Self-Verification Approach to Reinforcement Learning with Verifiable Rewards
by: Liu, Xiaoyuan, et al.
Published: (2025)

Split-Helicity Tree Amplitudes and Flag Cluster Algebras
by: Paranjape, Shruti, et al.
Published: (2025)

ParoQuant: Pairwise Rotation Quantization for Efficient Reasoning LLM Inference
by: Liang, Yesheng, et al.
Published: (2025)

Drug Resistance Predictions Based on a Directed Flag Transformer
by: Chen, Dong, et al.
Published: (2024)

Self‐Triggered Robust Safety‐Critical Control of Multiplayer Nonlinear Systems via Adaptive Dynamic Programming
by: Shukun Liu, et al.
Published: (2026)

Neural PDE Solvers with Physics Constraints: A Comparative Study of PINNs, DRM, and WANs
by: Chen, Jiakang
Published: (2025)

Effects of Low-Dose Recombinant Human Brain Natriuretic Peptide on Anterior Myocardial Infarction Complicated by Cardiogenic Shock
by: Yesheng Pan
Published: (2017)

TEG-DB: A Comprehensive Dataset and Benchmark of Textual-Edge Graphs
by: Li, Zhuofeng, et al.
Published: (2024)

TextualVerifier: Verify TextGrad Step-by-Step
by: Situmorang, Eugenius Mario, et al.
Published: (2025)

Agentic Rubrics as Contextual Verifiers for SWE Agents
by: Raghavendra, Mohit, et al.
Published: (2026)

Distilling Textual Priors from LLM to Efficient Image Fusion
by: Zhang, Ran, et al.
Published: (2025)

S3Eval: A Synthetic, Scalable, Systematic Evaluation Suite for Large Language Models
by: Lei, Fangyu, et al.
Published: (2023)

EvalCrafter: Benchmarking and Evaluating Large Video Generation Models
by: Liu, Yaofang, et al.
Published: (2023)

ChemEval: A Comprehensive Multi-Level Chemical Evaluation for Large Language Models
by: Huang, Yuqing, et al.
Published: (2024)

Knitting Interconnected Perovskite Grains with A Π‐Conjugated Passivator for Blue Light‐Emitting Diodes
by: Lihui Liu, et al.
Published: (2024)