Copertina

IRLBench: A Multi-modal, Culturally Grounded, Parallel Irish-English Benchmark for Open-Ended LLM Reasoning Evaluation

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Tran, Khanh-Tung, O'Sullivan, Barry, Nguyen, Hoang D.
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Computation and Language Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

UCCIX: Irish-eXcellence Large Language Model
di: Tran, Khanh-Tung, et al.
Pubblicazione: (2024)

Reasoning Transfer for an Extremely Low-Resource and Endangered Language: Bridging Languages Through Sample-Efficient Language Understanding
di: Tran, Khanh-Tung, et al.
Pubblicazione: (2025)

Irish-BLiMP: A Linguistic Benchmark for Evaluating Human and Language Model Performance in a Low-Resource Setting
di: McGiff, Josh, et al.
Pubblicazione: (2025)

Qomhra: A Bilingual Irish and English Large Language Model
di: McInerney, Joseph, et al.
Pubblicazione: (2025)

Multi-Agent Collaboration Mechanisms: A Survey of LLMs
di: Tran, Khanh-Tung, et al.
Pubblicazione: (2025)

AHP-Powered LLM Reasoning for Multi-Criteria Evaluation of Open-Ended Responses
di: Lu, Xiaotian, et al.
Pubblicazione: (2024)

Questionnaire meets LLM: A Benchmark and Empirical Study of Structural Skills for Understanding Questions and Responses
di: Nguyen, Duc-Hai, et al.
Pubblicazione: (2025)

Think in Parallel, Answer as One: Logit Averaging for Open-Ended Reasoning
di: Wang, Haonan, et al.
Pubblicazione: (2025)

PRISM: A Multi-Dimensional Benchmark for Evaluating LLM Peer Reviewers
di: Loc, Ngoc Phan Phuoc, et al.
Pubblicazione: (2026)

Distributional Open-Ended Evaluation of LLM Cultural Value Alignment Based on Value Codebook
di: Lee, Jaehyeok, et al.
Pubblicazione: (2026)

MMReason: An Open-Ended Multi-Modal Multi-Step Reasoning Benchmark for MLLMs Toward AGI
di: Yao, Huanjin, et al.
Pubblicazione: (2025)

OpenSIR: Open-Ended Self-Improving Reasoner
di: Kwan, Wai-Chung, et al.
Pubblicazione: (2025)

ViFactCheck: A New Benchmark Dataset and Methods for Multi-domain News Fact-Checking in Vietnamese
di: Hoa, Tran Thai, et al.
Pubblicazione: (2024)

MASIVE: Open-Ended Affective State Identification in English and Spanish
di: Deas, Nicholas, et al.
Pubblicazione: (2024)

Assessing Bias in Metric Models for LLM Open-Ended Generation Bias Benchmarks
di: Demchak, Nathaniel, et al.
Pubblicazione: (2024)

PuzzleWorld: A Benchmark for Multimodal, Open-Ended Reasoning in Puzzlehunts
di: Li, Hengzhi, et al.
Pubblicazione: (2025)

Reverse-Engineered Reasoning for Open-Ended Generation
di: Wang, Haozhe, et al.
Pubblicazione: (2025)

Scaling Open-Ended Reasoning to Predict the Future
di: Chandak, Nikhil, et al.
Pubblicazione: (2025)

Human Evaluation of English--Irish Transformer-Based NMT
di: Lankford, Séamus, et al.
Pubblicazione: (2024)

OWLViz: An Open-World Benchmark for Visual Question Answering
di: Nguyen, Thuy, et al.
Pubblicazione: (2025)

Beyond MCQ: An Open-Ended Arabic Cultural QA Benchmark with Dialect Variants
di: Bhatti, Hunzalah Hassan, et al.
Pubblicazione: (2025)

Marco-o1: Towards Open Reasoning Models for Open-Ended Solutions
di: Zhao, Yu, et al.
Pubblicazione: (2024)

BlasBench: An Open Benchmark for Irish Speech Recognition
di: Raj, Jyoutir, et al.
Pubblicazione: (2026)

Purdah and Patriarchy: Evaluating and Mitigating South Asian Biases in Open-Ended Multilingual LLM Generations
di: Rinki, Mamnuya, et al.
Pubblicazione: (2025)

Semantic Agreement Enables Efficient Open-Ended LLM Cascades
di: Soiffer, Duncan, et al.
Pubblicazione: (2025)

Bias Association Discovery Framework for Open-Ended LLM Generations
di: Pan, Jinhao, et al.
Pubblicazione: (2025)

CultureForest: Understanding and Evaluating Cultural Norm Grounded Reasoning in LLMs
di: Ye, Yangfan, et al.
Pubblicazione: (2026)

VLegal-Bench: Cognitively Grounded Benchmark for Vietnamese Legal Reasoning of Large Language Models
di: Dong, Nguyen Tien, et al.
Pubblicazione: (2025)

An Answer is just the Start: Related Insight Generation for Open-Ended Document-Grounded QA
di: Sharma, Saransh, et al.
Pubblicazione: (2026)

Towards Reproducible LLM Evaluation: Quantifying Uncertainty in LLM Benchmark Scores
di: Blackwell, Robert E., et al.
Pubblicazione: (2024)

CheckEmbed: Effective Verification of LLM Solutions to Open-Ended Tasks
di: Besta, Maciej, et al.
Pubblicazione: (2024)

MORQA: Benchmarking Evaluation Metrics for Medical Open-Ended Question Answering
di: Yim, Wen-wai, et al.
Pubblicazione: (2025)

Hard2Verify: A Step-Level Verification Benchmark for Open-Ended Frontier Math
di: Pandit, Shrey, et al.
Pubblicazione: (2025)

MATEval: A Multi-Agent Discussion Framework for Advancing Open-Ended Text Evaluation
di: Li, Yu, et al.
Pubblicazione: (2024)

End-to-End Chatbot Evaluation with Adaptive Reasoning and Uncertainty Filtering
di: Dang, Nhi, et al.
Pubblicazione: (2026)

MulCogBench: A Multi-modal Cognitive Benchmark Dataset for Evaluating Chinese and English Computational Language Models
di: Zhang, Yunhao, et al.
Pubblicazione: (2024)

VN-MTEB: Vietnamese Massive Text Embedding Benchmark
di: Pham, Loc, et al.
Pubblicazione: (2025)

Do LLMs Exhibit Human-Like Reasoning? Evaluating Theory of Mind in LLMs for Open-Ended Responses
di: Amirizaniani, Maryam, et al.
Pubblicazione: (2024)

VLUE: A New Benchmark and Multi-task Knowledge Transfer Learning for Vietnamese Natural Language Understanding
di: Do, Phong Nguyen-Thuan, et al.
Pubblicazione: (2024)

Progressive Multi-granular Alignments for Grounded Reasoning in Large Vision-Language Models
di: Le, Quang-Hung, et al.
Pubblicazione: (2024)