Cover Image

Are Large Language Model-based Evaluators the Solution to Scaling Up Multilingual Evaluation?

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Hada, Rishav, Gumma, Varun, de Wynter, Adrian, Diddee, Harshita, Ahmed, Mohamed, Choudhury, Monojit, Bali, Kalika, Sitaram, Sunayana
Format:	Preprint
Published:	2023
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

METAL: Towards Multilingual Meta-Evaluation
by: Hada, Rishav, et al.
Published: (2024)

Contamination Report for Multilingual Benchmarks
by: Ahuja, Sanchit, et al.
Published: (2024)

MEGAVERSE: Benchmarking Large Language Models Across Languages, Modalities, Models and Tasks
by: Ahuja, Sanchit, et al.
Published: (2023)

Akal Badi ya Bias: An Exploratory Study of Gender Bias in Hindi Language Technology
by: Hada, Rishav, et al.
Published: (2024)

Towards Inducing Long-Context Abilities in Multilingual Neural Machine Translation Models
by: Gumma, Varun, et al.
Published: (2024)

PARIKSHA: A Large-Scale Investigation of Human-LLM Evaluator Agreement on Multilingual and Multi-Cultural Data
by: Watts, Ishaan, et al.
Published: (2024)

Beyond Metrics: Evaluating LLMs' Effectiveness in Culturally Nuanced, Low-Resource Real-World Scenarios
by: Ochieng, Millicent, et al.
Published: (2024)

HEALTH-PARIKSHA: Assessing RAG Models for Health Chatbots in Real-World Multilingual Settings
by: Gumma, Varun, et al.
Published: (2024)

MunTTS: A Text-to-Speech System for Mundari
by: Gumma, Varun, et al.
Published: (2024)

Cultural Conditioning or Placebo? On the Effectiveness of Socio-Demographic Prompting
by: Mukherjee, Sagnik, et al.
Published: (2024)

How Deep Is Representational Bias in LLMs? The Cases of Caste and Religion
by: Seth, Agrima, et al.
Published: (2025)

DOSA: A Dataset of Social Artifacts from Different Indian Geographical Subcultures
by: Seth, Agrima, et al.
Published: (2024)

Building Benchmarks from the Ground Up: Community-Centered Evaluation of LLMs in Healthcare Chatbot Settings
by: Hamna, Hamna, et al.
Published: (2025)

MAPLE: Multilingual Evaluation of Parameter Efficient Finetuning of Large Language Models
by: Aggarwal, Divyanshu, et al.
Published: (2024)

Bridging the Language Gap: Dynamic Learning Strategies for Improving Multilingual Performance in LLMs
by: Kumar, Somnath, et al.
Published: (2023)

MAFIA: Multi-Adapter Fused Inclusive LanguAge Models
by: Jain, Prachi, et al.
Published: (2024)

M5 -- A Diverse Benchmark to Assess the Performance of Large Multimodal Models Across Multilingual and Multicultural Vision-Language Tasks
by: Schneider, Florian, et al.
Published: (2024)

Bridging the Gap: Dynamic Learning Strategies for Improving Multilingual Performance in LLMs
by: Kumar, Somnath, et al.
Published: (2024)

A Multilingual, Culture-First Approach to Addressing Misgendering in LLM Applications
by: Sitaram, Sunayana, et al.
Published: (2025)

Customizing Large Language Model Generation Style using Parameter-Efficient Finetuning
by: Liu, Xinyue, et al.
Published: (2024)

UPDESH: Synthesizing Grounded Instruction Tuning Data for 13 Indic Languages
by: Chitale, Pranjal A., et al.
Published: (2025)

Chasing Random: Instruction Selection Strategies Fail to Generalize
by: Diddee, Harshita, et al.
Published: (2024)

BenchBrowser: Retrieving Evidence for Evaluating Benchmark Validity
by: Diddee, Harshita, et al.
Published: (2026)

Evaluating Large Language Models for Health-related Queries with Presuppositions
by: Kaur, Navreet, et al.
Published: (2023)

CIE: Controlling Language Model Text Generations Using Continuous Signals
by: Samuel, Vinay, et al.
Published: (2025)

NoveltyBench: Evaluating Language Models for Humanlike Diversity
by: Zhang, Yiming, et al.
Published: (2025)

Exploring Pretraining via Active Forgetting for Improving Cross Lingual Transfer for Decoder Language Models
by: Aggarwal, Divyanshu, et al.
Published: (2024)

Algorithmically Establishing Trust in Evaluators
by: de Wynter, Adrian
Published: (2025)

User Behavior Prediction as a Generic, Robust, Scalable, and Low-Cost Evaluation Strategy for Estimating Generalization in LLMs
by: Saha, Sougata, et al.
Published: (2025)

Litmus (Re)Agent: A Benchmark and Agentic System for Predictive Evaluation of Multilingual Models
by: Mittal, Avni, et al.
Published: (2026)

AgentCompass: Towards Reliable Evaluation of Agentic Workflows in Production
by: Kartik, NVJK, et al.
Published: (2025)

Exploring Continual Fine-Tuning for Enhancing Language Ability in Large Language Model
by: Aggarwal, Divyanshu, et al.
Published: (2024)

MS-HuBERT: Mitigating Pre-training and Inference Mismatch in Masked Language Modelling methods for learning Speech Representations
by: Yadav, Hemant, et al.
Published: (2024)

A Unified Framework and Dataset for Assessing Societal Bias in Vision-Language Models
by: Sathe, Ashutosh, et al.
Published: (2024)

DEPART: DEcomposing PARiTy across Multilingual LLMs
by: Uppadhyay, Manan, et al.
Published: (2026)

CultureLLM: Incorporating Cultural Differences into Large Language Models
by: Li, Cheng, et al.
Published: (2024)

Improving Self Consistency in LLMs through Probabilistic Tokenization
by: Sathe, Ashutosh, et al.
Published: (2024)

Do Moral Judgment and Reasoning Capability of LLMs Change with Language? A Study using the Multilingual Defining Issues Test
by: Khandelwal, Aditi, et al.
Published: (2024)

"I'd Like to Have an Argument, Please": Argumentative Reasoning in Large Language Models
by: de Wynter, Adrian, et al.
Published: (2023)

An Evaluation on Large Language Model Outputs: Discourse and Memorization
by: de Wynter, Adrian, et al.
Published: (2023)