Cover Image

A Measure for Transparent Comparison of Linguistic Diversity in Multilingual NLP Data Sets

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Samardzic, Tanja, Gutierrez, Ximena, Bentz, Christian, Moran, Steven, Pelloni, Olga
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Tokenization and Representation Biases in Multilingual Models on Dialectal NLP Tasks
by: Kanjirangat, Vani, et al.
Published: (2025)

Cross-Linguistic Transfer in Multilingual NLP: The Role of Language Families and Morphology
by: Bankula, Ajitesh, et al.
Published: (2025)

We Need to Measure Data Diversity in NLP -- Better and Broader
by: Nguyen, Dong, et al.
Published: (2025)

Towards Inclusive NLP: Assessing Compressed Multilingual Transformers across Diverse Language Benchmarks
by: Alshehhi, Maitha, et al.
Published: (2025)

NLP needs Diversity outside of 'Diversity'
by: Tint, Joshua
Published: (2026)

Standardising the NLP Workflow: A Framework for Reproducible Linguistic Analysis
by: Pauli, Yves, et al.
Published: (2025)

Multilingual Large Language Models and Curse of Multilinguality
by: Gurgurov, Daniil, et al.
Published: (2024)

What is "Typological Diversity" in NLP?
by: Ploeger, Esther, et al.
Published: (2024)

How Good is Your Wikipedia? Auditing Data Quality for Low-resource and Multilingual NLP
by: Tatariya, Kushal, et al.
Published: (2024)

Transparent NLP: Using RAG and LLM Alignment for Privacy Q&A
by: Leschanowsky, Anna, et al.
Published: (2025)

Improving Multilingual Instruction Finetuning via Linguistically Natural and Diverse Datasets
by: Indurthi, Sathish Reddy, et al.
Published: (2024)

Linguistically-Informed Multilingual Instruction Tuning: Is There an Optimal Set of Languages to Tune?
by: Soykan, Gürkan, et al.
Published: (2024)

Designing NLP Systems That Adapt to Diverse Worldviews
by: Creanga, Claudiu, et al.
Published: (2024)

Mafoko: Structuring and Building Open Multilingual Terminologies for South African NLP
by: Marivate, Vukosi, et al.
Published: (2025)

MC$^2$: Towards Transparent and Culturally-Aware NLP for Minority Languages in China
by: Zhang, Chen, et al.
Published: (2023)

MELA: Multilingual Evaluation of Linguistic Acceptability
by: Zhang, Ziyin, et al.
Published: (2023)

Linguistically Informed Evaluation of Multilingual ASR for African Languages
by: Chen, Fei-Yueh, et al.
Published: (2026)

Exploring NLP Benchmarks in an Extremely Low-Resource Setting
by: Nuha, Ulin, et al.
Published: (2025)

Benchmark Transparency: Measuring the Impact of Data on Evaluation
by: Kovatchev, Venelin, et al.
Published: (2024)

Ground Truth Generation for Multilingual Historical NLP using LLMs
by: Gladstone, Clovis, et al.
Published: (2025)

Detecting Linguistic Diversity on Social Media
by: Wong, Sidney, et al.
Published: (2025)

Measuring the Robustness of NLP Models to Domain Shifts
by: Calderon, Nitay, et al.
Published: (2023)

Linguistic and Argument Diversity in Synthetic Data for Function-Calling Agents
by: Greenstein, Dan, et al.
Published: (2026)

Benchmarking Sociolinguistic Diversity in Swahili NLP: A Taxonomy-Guided Approach
by: Oketch, Kezia, et al.
Published: (2025)

Benchmarking Linguistic Diversity of Large Language Models
by: Guo, Yanzhu, et al.
Published: (2024)

Multilingual Steering by Design: Multilingual Sparse Autoencoders and Principled Layer Selection
by: Ghussin, Yusser Al, et al.
Published: (2026)

Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation
by: Singh, Shivalika, et al.
Published: (2024)

Improving Multilingual Neural Machine Translation by Utilizing Semantic and Linguistic Features
by: Bu, Mengyu, et al.
Published: (2024)

Linguistic Nepotism: Trading-off Quality for Language Preference in Multilingual RAG
by: Ki, Dayeon, et al.
Published: (2025)

Traversing Emotional Landscapes and Linguistic Patterns in Bernard-Marie Koltès' Plays: An NLP Perspective
by: Pourzarandi, Arezou Zahiri, et al.
Published: (2024)

Facilitating Opinion Diversity through Hybrid NLP Approaches
by: van der Meer, Michiel
Published: (2024)

Decomposed Prompting: Probing Multilingual Linguistic Structure Knowledge in Large Language Models
by: Nie, Ercong, et al.
Published: (2024)

Quantifying the Risks of Tool-assisted Rephrasing to Linguistic Diversity
by: Wang, Mengying, et al.
Published: (2024)

NagaNLP: Bootstrapping NLP for Low-Resource Nagamese Creole with Human-in-the-Loop Synthetic Data
by: Maiti, Agniva, et al.
Published: (2025)

Test Set Quality in Multilingual LLM Evaluation
by: Kranti, Chalamalasetti, et al.
Published: (2025)

Multilingual Language Models Encode Script Over Linguistic Structure
by: Verma, Aastha A K, et al.
Published: (2026)

Undesirable Biases in NLP: Addressing Challenges of Measurement
by: van der Wal, Oskar, et al.
Published: (2022)

MultiBLiMP 1.0: A Massively Multilingual Benchmark of Linguistic Minimal Pairs
by: Jumelet, Jaap, et al.
Published: (2025)

Exploring the Reliability of Large Language Models as Customized Evaluators for Diverse NLP Tasks
by: Li, Qintong, et al.
Published: (2023)

NLP Progress in Indigenous Latin American Languages
by: Tonja, Atnafu Lambebo, et al.
Published: (2024)