Imagen de Portada

The FACTS Leaderboard: A Comprehensive Benchmark for Large Language Model Factuality

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Cheng, Aileen, Jacovi, Alon, Globerson, Amir, Golan, Ben, Kwong, Charles, Alberti, Chris, Tao, Connie, Ben-David, Eyal, Tomar, Gaurav Singh, Haas, Lukas, Bitton, Yonatan, Bloniarz, Adam, Bai, Aijun, Wang, Andrew, Siddiqui, Anfal, Castillo, Arturo Bajuelos, Atias, Aviel, Liu, Chang, Fry, Corey, Balle, Daniel, Ghosal, Deepanway, Kukliansky, Doron, Marcus, Dror, Gribovskaya, Elena, Ofek, Eran, Zhuang, Honglei, Laish, Itay, Ackermann, Jan, Wang, Lily, Risdal, Meg, Barnes, Megan, Fink, Michael, Amin, Mohamed, Ambar, Moran, Potikha, Natan, Gupta, Nikita, Katz, Nitzan, Velan, Noam, Roval, Ofir, Ram, Ori, Zablotskaia, Polina, Bang, Prathamesh, Agrawal, Priyanka, Ghiya, Rakesh, Ganapathy, Sanjay, Baumgartner, Simon, Erell, Sofia, Prakash, Sushant, Sellam, Thibault, Rao, Vikram, Wang, Xuanhui, Akulov, Yaroslav, Yang, Yulong, Yang, Zhen, Lai, Zhixin, Wu, Zhongru, Dragan, Anca, Hassidim, Avinatan, Pereira, Fernando, Petrov, Slav, Venkatachary, Srinivasan, Doshi, Tulsee, Matias, Yossi, Goldshtein, Sasha, Das, Dipanjan
Formato:	Preprint
Publicado:	2025
Materias:	Computation and Language Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

_version_	1866911314246369280
author	Cheng, Aileen Jacovi, Alon Globerson, Amir Golan, Ben Kwong, Charles Alberti, Chris Tao, Connie Ben-David, Eyal Tomar, Gaurav Singh Haas, Lukas Bitton, Yonatan Bloniarz, Adam Bai, Aijun Wang, Andrew Siddiqui, Anfal Castillo, Arturo Bajuelos Atias, Aviel Liu, Chang Fry, Corey Balle, Daniel Ghosal, Deepanway Kukliansky, Doron Marcus, Dror Gribovskaya, Elena Ofek, Eran Zhuang, Honglei Laish, Itay Ackermann, Jan Wang, Lily Risdal, Meg Barnes, Megan Fink, Michael Amin, Mohamed Ambar, Moran Potikha, Natan Gupta, Nikita Katz, Nitzan Velan, Noam Roval, Ofir Ram, Ori Zablotskaia, Polina Bang, Prathamesh Agrawal, Priyanka Ghiya, Rakesh Ganapathy, Sanjay Baumgartner, Simon Erell, Sofia Prakash, Sushant Sellam, Thibault Rao, Vikram Wang, Xuanhui Akulov, Yaroslav Yang, Yulong Yang, Zhen Lai, Zhixin Wu, Zhongru Dragan, Anca Hassidim, Avinatan Pereira, Fernando Petrov, Slav Venkatachary, Srinivasan Doshi, Tulsee Matias, Yossi Goldshtein, Sasha Das, Dipanjan
author_facet	Cheng, Aileen Jacovi, Alon Globerson, Amir Golan, Ben Kwong, Charles Alberti, Chris Tao, Connie Ben-David, Eyal Tomar, Gaurav Singh Haas, Lukas Bitton, Yonatan Bloniarz, Adam Bai, Aijun Wang, Andrew Siddiqui, Anfal Castillo, Arturo Bajuelos Atias, Aviel Liu, Chang Fry, Corey Balle, Daniel Ghosal, Deepanway Kukliansky, Doron Marcus, Dror Gribovskaya, Elena Ofek, Eran Zhuang, Honglei Laish, Itay Ackermann, Jan Wang, Lily Risdal, Meg Barnes, Megan Fink, Michael Amin, Mohamed Ambar, Moran Potikha, Natan Gupta, Nikita Katz, Nitzan Velan, Noam Roval, Ofir Ram, Ori Zablotskaia, Polina Bang, Prathamesh Agrawal, Priyanka Ghiya, Rakesh Ganapathy, Sanjay Baumgartner, Simon Erell, Sofia Prakash, Sushant Sellam, Thibault Rao, Vikram Wang, Xuanhui Akulov, Yaroslav Yang, Yulong Yang, Zhen Lai, Zhixin Wu, Zhongru Dragan, Anca Hassidim, Avinatan Pereira, Fernando Petrov, Slav Venkatachary, Srinivasan Doshi, Tulsee Matias, Yossi Goldshtein, Sasha Das, Dipanjan
contents	We introduce The FACTS Leaderboard, an online leaderboard suite and associated set of benchmarks that comprehensively evaluates the ability of language models to generate factually accurate text across diverse scenarios. The suite provides a holistic measure of factuality by aggregating the performance of models on four distinct sub-leaderboards: (1) FACTS Multimodal, which measures the factuality of responses to image-based questions; (2) FACTS Parametric, which assesses models' world knowledge by answering closed-book factoid questions from internal parameters; (3) FACTS Search, which evaluates factuality in information-seeking scenarios, where the model must use a search API; and (4) FACTS Grounding (v2), which evaluates whether long-form responses are grounded in provided documents, featuring significantly improved judge models. Each sub-leaderboard employs automated judge models to score model responses, and the final suite score is an average of the four components, designed to provide a robust and balanced assessment of a model's overall factuality. The FACTS Leaderboard Suite will be actively maintained, containing both public and private splits to allow for external participation while guarding its integrity. It can be found at https://www.kaggle.com/benchmarks/google/facts .
format	Preprint
id	arxiv_https___arxiv_org_abs_2512_10791
institution	arXiv
publishDate	2025
record_format	arxiv
spellingShingle	The FACTS Leaderboard: A Comprehensive Benchmark for Large Language Model Factuality Cheng, Aileen Jacovi, Alon Globerson, Amir Golan, Ben Kwong, Charles Alberti, Chris Tao, Connie Ben-David, Eyal Tomar, Gaurav Singh Haas, Lukas Bitton, Yonatan Bloniarz, Adam Bai, Aijun Wang, Andrew Siddiqui, Anfal Castillo, Arturo Bajuelos Atias, Aviel Liu, Chang Fry, Corey Balle, Daniel Ghosal, Deepanway Kukliansky, Doron Marcus, Dror Gribovskaya, Elena Ofek, Eran Zhuang, Honglei Laish, Itay Ackermann, Jan Wang, Lily Risdal, Meg Barnes, Megan Fink, Michael Amin, Mohamed Ambar, Moran Potikha, Natan Gupta, Nikita Katz, Nitzan Velan, Noam Roval, Ofir Ram, Ori Zablotskaia, Polina Bang, Prathamesh Agrawal, Priyanka Ghiya, Rakesh Ganapathy, Sanjay Baumgartner, Simon Erell, Sofia Prakash, Sushant Sellam, Thibault Rao, Vikram Wang, Xuanhui Akulov, Yaroslav Yang, Yulong Yang, Zhen Lai, Zhixin Wu, Zhongru Dragan, Anca Hassidim, Avinatan Pereira, Fernando Petrov, Slav Venkatachary, Srinivasan Doshi, Tulsee Matias, Yossi Goldshtein, Sasha Das, Dipanjan Computation and Language Artificial Intelligence We introduce The FACTS Leaderboard, an online leaderboard suite and associated set of benchmarks that comprehensively evaluates the ability of language models to generate factually accurate text across diverse scenarios. The suite provides a holistic measure of factuality by aggregating the performance of models on four distinct sub-leaderboards: (1) FACTS Multimodal, which measures the factuality of responses to image-based questions; (2) FACTS Parametric, which assesses models' world knowledge by answering closed-book factoid questions from internal parameters; (3) FACTS Search, which evaluates factuality in information-seeking scenarios, where the model must use a search API; and (4) FACTS Grounding (v2), which evaluates whether long-form responses are grounded in provided documents, featuring significantly improved judge models. Each sub-leaderboard employs automated judge models to score model responses, and the final suite score is an average of the four components, designed to provide a robust and balanced assessment of a model's overall factuality. The FACTS Leaderboard Suite will be actively maintained, containing both public and private splits to allow for external participation while guarding its integrity. It can be found at https://www.kaggle.com/benchmarks/google/facts .
title	The FACTS Leaderboard: A Comprehensive Benchmark for Large Language Model Factuality
topic	Computation and Language Artificial Intelligence
url	https://arxiv.org/abs/2512.10791