Image de couverture de livre

IRLBench: A Multi-modal, Culturally Grounded, Parallel Irish-English Benchmark for Open-Ended LLM Reasoning Evaluation

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Tran, Khanh-Tung, O'Sullivan, Barry, Nguyen, Hoang D.
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

UCCIX: Irish-eXcellence Large Language Model
par: Tran, Khanh-Tung, et autres
Publié: (2024)

Reasoning Transfer for an Extremely Low-Resource and Endangered Language: Bridging Languages Through Sample-Efficient Language Understanding
par: Tran, Khanh-Tung, et autres
Publié: (2025)

Irish-BLiMP: A Linguistic Benchmark for Evaluating Human and Language Model Performance in a Low-Resource Setting
par: McGiff, Josh, et autres
Publié: (2025)

Qomhra: A Bilingual Irish and English Large Language Model
par: McInerney, Joseph, et autres
Publié: (2025)

Multi-Agent Collaboration Mechanisms: A Survey of LLMs
par: Tran, Khanh-Tung, et autres
Publié: (2025)

AHP-Powered LLM Reasoning for Multi-Criteria Evaluation of Open-Ended Responses
par: Lu, Xiaotian, et autres
Publié: (2024)

Questionnaire meets LLM: A Benchmark and Empirical Study of Structural Skills for Understanding Questions and Responses
par: Nguyen, Duc-Hai, et autres
Publié: (2025)

Think in Parallel, Answer as One: Logit Averaging for Open-Ended Reasoning
par: Wang, Haonan, et autres
Publié: (2025)

PRISM: A Multi-Dimensional Benchmark for Evaluating LLM Peer Reviewers
par: Loc, Ngoc Phan Phuoc, et autres
Publié: (2026)

Distributional Open-Ended Evaluation of LLM Cultural Value Alignment Based on Value Codebook
par: Lee, Jaehyeok, et autres
Publié: (2026)

MMReason: An Open-Ended Multi-Modal Multi-Step Reasoning Benchmark for MLLMs Toward AGI
par: Yao, Huanjin, et autres
Publié: (2025)

OpenSIR: Open-Ended Self-Improving Reasoner
par: Kwan, Wai-Chung, et autres
Publié: (2025)

ViFactCheck: A New Benchmark Dataset and Methods for Multi-domain News Fact-Checking in Vietnamese
par: Hoa, Tran Thai, et autres
Publié: (2024)

MASIVE: Open-Ended Affective State Identification in English and Spanish
par: Deas, Nicholas, et autres
Publié: (2024)

Assessing Bias in Metric Models for LLM Open-Ended Generation Bias Benchmarks
par: Demchak, Nathaniel, et autres
Publié: (2024)

PuzzleWorld: A Benchmark for Multimodal, Open-Ended Reasoning in Puzzlehunts
par: Li, Hengzhi, et autres
Publié: (2025)

Reverse-Engineered Reasoning for Open-Ended Generation
par: Wang, Haozhe, et autres
Publié: (2025)

Scaling Open-Ended Reasoning to Predict the Future
par: Chandak, Nikhil, et autres
Publié: (2025)

Human Evaluation of English--Irish Transformer-Based NMT
par: Lankford, Séamus, et autres
Publié: (2024)

OWLViz: An Open-World Benchmark for Visual Question Answering
par: Nguyen, Thuy, et autres
Publié: (2025)

Beyond MCQ: An Open-Ended Arabic Cultural QA Benchmark with Dialect Variants
par: Bhatti, Hunzalah Hassan, et autres
Publié: (2025)

Marco-o1: Towards Open Reasoning Models for Open-Ended Solutions
par: Zhao, Yu, et autres
Publié: (2024)

BlasBench: An Open Benchmark for Irish Speech Recognition
par: Raj, Jyoutir, et autres
Publié: (2026)

Purdah and Patriarchy: Evaluating and Mitigating South Asian Biases in Open-Ended Multilingual LLM Generations
par: Rinki, Mamnuya, et autres
Publié: (2025)

Semantic Agreement Enables Efficient Open-Ended LLM Cascades
par: Soiffer, Duncan, et autres
Publié: (2025)

Bias Association Discovery Framework for Open-Ended LLM Generations
par: Pan, Jinhao, et autres
Publié: (2025)

CultureForest: Understanding and Evaluating Cultural Norm Grounded Reasoning in LLMs
par: Ye, Yangfan, et autres
Publié: (2026)

VLegal-Bench: Cognitively Grounded Benchmark for Vietnamese Legal Reasoning of Large Language Models
par: Dong, Nguyen Tien, et autres
Publié: (2025)

An Answer is just the Start: Related Insight Generation for Open-Ended Document-Grounded QA
par: Sharma, Saransh, et autres
Publié: (2026)

Towards Reproducible LLM Evaluation: Quantifying Uncertainty in LLM Benchmark Scores
par: Blackwell, Robert E., et autres
Publié: (2024)

CheckEmbed: Effective Verification of LLM Solutions to Open-Ended Tasks
par: Besta, Maciej, et autres
Publié: (2024)

MORQA: Benchmarking Evaluation Metrics for Medical Open-Ended Question Answering
par: Yim, Wen-wai, et autres
Publié: (2025)

Hard2Verify: A Step-Level Verification Benchmark for Open-Ended Frontier Math
par: Pandit, Shrey, et autres
Publié: (2025)

MATEval: A Multi-Agent Discussion Framework for Advancing Open-Ended Text Evaluation
par: Li, Yu, et autres
Publié: (2024)

End-to-End Chatbot Evaluation with Adaptive Reasoning and Uncertainty Filtering
par: Dang, Nhi, et autres
Publié: (2026)

MulCogBench: A Multi-modal Cognitive Benchmark Dataset for Evaluating Chinese and English Computational Language Models
par: Zhang, Yunhao, et autres
Publié: (2024)

Do LLMs Exhibit Human-Like Reasoning? Evaluating Theory of Mind in LLMs for Open-Ended Responses
par: Amirizaniani, Maryam, et autres
Publié: (2024)

VN-MTEB: Vietnamese Massive Text Embedding Benchmark
par: Pham, Loc, et autres
Publié: (2025)

VLUE: A New Benchmark and Multi-task Knowledge Transfer Learning for Vietnamese Natural Language Understanding
par: Do, Phong Nguyen-Thuan, et autres
Publié: (2024)

Progressive Multi-granular Alignments for Grounded Reasoning in Large Vision-Language Models
par: Le, Quang-Hung, et autres
Publié: (2024)