Cover Image

COLE: a Comprehensive Benchmark for French Language Understanding Evaluation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Beauchemin, David, Tremblay, Yan, Youssef, Mohamed Amine, Khoury, Richard
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Idiom Understanding as a Tool to Measure the Dialect Gap
by: Beauchemin, David, et al.
Published: (2025)

QFrCoLA: a Quebec-French Corpus of Linguistic Acceptability Judgments
by: Beauchemin, David, et al.
Published: (2025)

Benchmarking Large Language Models for Quebec Insurance: From Closed-Book to Retrieval-Augmented Generation
by: Beauchemin, David, et al.
Published: (2026)

QFrBLiMP: a Quebec-French Benchmark of Linguistic Minimal Pairs
by: Beauchemin, David, et al.
Published: (2025)

Quebec Automobile Insurance Question-Answering With Retrieval-Augmented Generation
by: Beauchemin, David, et al.
Published: (2024)

Neural Machine Translation for Coptic-French: Strategies for Low-Resource Ancient Languages
by: Chaoui, Nasma, et al.
Published: (2025)

JUDGEBERT: Assessing Legal Meaning Preservation Between Sentences
by: Beauchemin, David, et al.
Published: (2025)

DrBenchmark: A Large Language Understanding Evaluation Benchmark for French Biomedical Domain
by: Labrak, Yanis, et al.
Published: (2024)

Automated Journalistic Questions: A New Method for Extracting 5W1H in French
by: Verhaverbeke, Maxence, et al.
Published: (2025)

Low-Resource Dialect Adaptation of Large Language Models: A French Dialect Case-Study
by: Khan, Eeham, et al.
Published: (2025)

SinhalaMMLU: A Comprehensive Benchmark for Evaluating Multitask Language Understanding in Sinhala
by: Pramodya, Ashmari, et al.
Published: (2025)

HKCanto-Eval: A Benchmark for Evaluating Cantonese Language Understanding and Cultural Comprehension in LLMs
by: Cheng, Tsz Chung, et al.
Published: (2025)

New Semantic Task for the French Spoken Language Understanding MEDIA Benchmark
by: Alavoine, Nadège, et al.
Published: (2024)

FrenchToxicityPrompts: a Large Benchmark for Evaluating and Mitigating Toxicity in French Texts
by: Brun, Caroline, et al.
Published: (2024)

Evaluating Large Language Models on Time Series Feature Understanding: A Comprehensive Taxonomy and Benchmark
by: Fons, Elizabeth, et al.
Published: (2024)

ASIC-Agent: An Autonomous Multi-Agent System for ASIC Design with Benchmark Evaluation
by: Allam, Ahmed, et al.
Published: (2025)

TLUE: A Tibetan Language Understanding Evaluation Benchmark
by: Gao, Fan, et al.
Published: (2025)

A Comprehensive Analysis of Tokenization and Self-Supervised Learning in End-to-End Automatic Speech Recognition applied on French Language
by: Bañeras-Roux, Thibault, et al.
Published: (2026)

How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse
by: Seddik, Mohamed El Amine, et al.
Published: (2024)

C$^{3}$Bench: A Comprehensive Classical Chinese Understanding Benchmark for Large Language Models
by: Cao, Jiahuan, et al.
Published: (2024)

SPORTU: A Comprehensive Sports Understanding Benchmark for Multimodal Large Language Models
by: Xia, Haotian, et al.
Published: (2024)

NorEval: A Norwegian Language Understanding and Generation Evaluation Benchmark
by: Mikhailov, Vladislav, et al.
Published: (2025)

SANSKRITI: A Comprehensive Benchmark for Evaluating Language Models' Knowledge of Indian Culture
by: Maji, Arijit, et al.
Published: (2025)

Benchmarking GPT-4 against Human Translators: A Comprehensive Evaluation Across Languages, Domains, and Expertise Levels
by: Yan, Jianhao, et al.
Published: (2024)

Role-Conditioned Refusals: Evaluating Access Control Reasoning in Large Language Models
by: Klisura, Đorđe, et al.
Published: (2025)

LLMs for Generating and Evaluating Counterfactuals: A Comprehensive Study
by: Nguyen, Van Bach, et al.
Published: (2024)

ValueBench: Towards Comprehensively Evaluating Value Orientations and Understanding of Large Language Models
by: Ren, Yuanyi, et al.
Published: (2024)

LexEval: A Comprehensive Chinese Legal Benchmark for Evaluating Large Language Models
by: Li, Haitao, et al.
Published: (2024)

No Culture Left Behind: ArtELingo-28, a Benchmark of WikiArt with Captions in 28 Languages
by: Mohamed, Youssef, et al.
Published: (2024)

A Benchmark of French ASR Systems Based on Error Severity
by: Tholly, Antoine, et al.
Published: (2025)

SciCUEval: A Comprehensive Dataset for Evaluating Scientific Context Understanding in Large Language Models
by: Yu, Jing, et al.
Published: (2025)

A Benchmark Evaluation of Clinical Named Entity Recognition in French
by: Bannour, Nesrine, et al.
Published: (2024)

OmniEduBench: A Comprehensive Chinese Benchmark for Evaluating Large Language Models in Education
by: Zhang, Min, et al.
Published: (2025)

BAMBOO: A Comprehensive Benchmark for Evaluating Long Text Modeling Capacities of Large Language Models
by: Dong, Zican, et al.
Published: (2023)

CASTLE: A Comprehensive Benchmark for Evaluating Student-Tailored Personalized Safety in Large Language Models
by: Jia, Rui, et al.
Published: (2026)

Benchmarking Vision-Language Models for French PDF-to-Markdown Conversion
by: Rigal, Bruno, et al.
Published: (2026)

KoDialogBench: Evaluating Conversational Understanding of Language Models with Korean Dialogue Benchmark
by: Jang, Seongbo, et al.
Published: (2024)

Reefknot: A Comprehensive Benchmark for Relation Hallucination Evaluation, Analysis and Mitigation in Multimodal Large Language Models
by: Zheng, Kening, et al.
Published: (2024)

AgriEval: A Comprehensive Chinese Agricultural Benchmark for Large Language Models
by: Yan, Lian, et al.
Published: (2025)

BioProBench: Comprehensive Dataset and Benchmark in Biological Protocol Understanding and Reasoning
by: Liu, Yuyang, et al.
Published: (2025)