Cover Image

LLMEval-Fair: A Large-Scale Longitudinal Study on Robust and Fair Evaluation of Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhang, Ming, Shen, Yujiong, Deng, Jingyi, Wang, Yuhui, Sha, Huayu, Tan, Kexin, Peng, Qiyuan, Zhang, Yue, Wang, Junzhe, Liu, Shichun, Huang, Yueyuan, Tong, Jingqi, Jiang, Changhao, Wu, Yilong, Zhang, Zhihao, Wu, Mingqi, Chai, Mingxu, Xi, Zhiheng, Dou, Shihan, Gui, Tao, Zhang, Qi, Huang, Xuanjing
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

LLMEval-Med: A Real-world Clinical Benchmark for Medical LLMs with Physician Validation
by: Zhang, Ming, et al.
Published: (2025)

LLMEval-Logic: A Solver-Verified Chinese Benchmark for Logical Reasoning of LLMs with Adversarial Hardening
by: Zhang, Ming, et al.
Published: (2026)

OpenNovelty: An LLM-powered Agentic System for Verifiable Scholarly Novelty Assessment
by: Zhang, Ming, et al.
Published: (2026)

TransferTOD: A Generalizable Chinese Multi-Domain Task-Oriented Dialogue System with Transfer Capabilities
by: Zhang, Ming, et al.
Published: (2024)

PFDial: A Structured Dialogue Instruction Fine-tuning Method Based on UML Flowcharts
by: Zhang, Ming, et al.
Published: (2025)

SciAgentGym: Benchmarking Multi-Step Scientific Tool-use in LLM Agents
by: Shen, Yujiong, et al.
Published: (2026)

SpeechRole: A Large-Scale Dataset and Benchmark for Evaluating Speech Role-Playing Agents
by: Jiang, Changhao, et al.
Published: (2025)

Beyond Scaling: Measuring and Predicting the Upper Bound of Knowledge Retention in Language Model Pre-Training
by: Jiang, Changhao, et al.
Published: (2025)

Can Deep Research Agents Retrieve and Organize? Evaluating the Synthesis Gap with Expert Taxonomies
by: Zhang, Ming, et al.
Published: (2026)

Exploring the Compositional Deficiency of Large Language Models in Mathematical Reasoning
by: Zhao, Jun, et al.
Published: (2024)

Unveiling Linguistic Regions in Large Language Models
by: Zhang, Zhihao, et al.
Published: (2024)

JFTA-Bench: Evaluate LLM's Ability of Tracking and Analyzing Malfunctions Using Fault Trees
by: Wang, Yuhui, et al.
Published: (2026)

Self-Demos: Eliciting Out-of-Demonstration Generalizability in Large Language Models
by: He, Wei, et al.
Published: (2024)

What's Wrong with Your Code Generated by Large Language Models? An Extensive Study
by: Dou, Shihan, et al.
Published: (2024)

Exploring Accuracy-Fairness Trade-off in Large Language Models
by: Zhang, Qingquan, et al.
Published: (2024)

Partial Identification Approach to Counterfactual Fairness Assessment
by: Rho, Saeyoung, et al.
Published: (2025)

Equinox: Holistic Fair Scheduling in Serving Large Language Models
by: Wei, Zhixiang, et al.
Published: (2025)

Can RL Improve Generalization of LLM Agents? An Empirical Study
by: Xi, Zhiheng, et al.
Published: (2026)

DFPO: Scaling Value Modeling via Distributional Flow towards Robust and Generalizable LLM Post-Training
by: Zhu, Dingwei, et al.
Published: (2026)

ToolEyes: Fine-Grained Evaluation for Tool Learning Capabilities of Large Language Models in Real-world Scenarios
by: Ye, Junjie, et al.
Published: (2024)

CodeChameleon: Personalized Encryption Framework for Jailbreaking Large Language Models
by: Lv, Huijie, et al.
Published: (2024)

RoCoIns: Enhancing Robustness of Large Language Models through Code-Style Instructions
by: Zhang, Yuansen, et al.
Published: (2024)

Less Is More? Examining Fairness in Pruned Large Language Models for Summarising Opinions
by: Huang, Nannan, et al.
Published: (2025)

Training Large Language Models for Reasoning through Reverse Curriculum Reinforcement Learning
by: Xi, Zhiheng, et al.
Published: (2024)

Inverse-Q*: Token Level Reinforcement Learning for Aligning Large Language Models Without Preference Data
by: Xia, Han, et al.
Published: (2024)

ToolSword: Unveiling Safety Issues of Large Language Models in Tool Learning Across Three Stages
by: Ye, Junjie, et al.
Published: (2024)

A Survey on Evaluation of Multimodal Large Language Models
by: Huang, Jiaxing, et al.
Published: (2024)

FairHuman: Boosting Hand and Face Quality in Human Image Generation with Minimum Potential Delay Fairness in Diffusion Models
by: Wang, Yuxuan, et al.
Published: (2025)

MM-Doc-R1: Training Agents for Long Document Visual Question Answering through Multi-turn Reinforcement Learning
by: Lin, Jiahang, et al.
Published: (2026)

TL-Training: A Task-Feature-Based Framework for Training Large Language Models in Tool Use
by: Ye, Junjie, et al.
Published: (2024)

FairCMS: Cloud Media Sharing with Fair Copyright Protection
by: Xiao, Xiangli, et al.
Published: (2021)

Do Large Language Models Rank Fairly? An Empirical Study on the Fairness of LLMs as Rankers
by: Wang, Yuan, et al.
Published: (2024)

Feedback-Driven Tool-Use Improvements in Large Language Models via Automated Build Environments
by: Ye, Junjie, et al.
Published: (2025)

FairGT: A Fairness-aware Graph Transformer
by: Luo, Renqiang, et al.
Published: (2024)

RoTBench: A Multi-Level Benchmark for Evaluating the Robustness of Large Language Models in Tool Learning
by: Ye, Junjie, et al.
Published: (2024)

Fairness in Large Language Models: A Taxonomic Survey
by: Chu, Zhibo, et al.
Published: (2024)

HoloFair: Unified T2I Fairness Evaluation and Fair-GRPO Debiasing
by: Chen, Ruyi, et al.
Published: (2026)

Fairness in Large Language Models in Three Hours
by: Viet, Thang Doan, et al.
Published: (2024)

Towards Fairness-Aware Adversarial Learning
by: Zhang, Yanghao, et al.
Published: (2024)

Federated Unlearning in the Wild: Rethinking Fairness and Data Discrepancy
by: Huang, ZiHeng, et al.
Published: (2025)