Imagen de Portada

Lessons from the Trenches on Reproducible Evaluation of Language Models

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Biderman, Stella, Schoelkopf, Hailey, Sutawika, Lintang, Gao, Leo, Tow, Jonathan, Abbasi, Baber, Aji, Alham Fikri, Ammanamanchi, Pawan Sasanka, Black, Sidney, Clive, Jordan, DiPofi, Anthony, Etxaniz, Julen, Fattori, Benjamin, Forde, Jessica Zosa, Foster, Charles, Hsu, Jeffrey, Jaiswal, Mimansa, Lee, Wilson Y., Li, Haonan, Lovering, Charles, Muennighoff, Niklas, Pavlick, Ellie, Phang, Jason, Skowron, Aviya, Tan, Samson, Tang, Xiangru, Wang, Kevin A., Winata, Genta Indra, Yvon, François, Zou, Andy
Formato:	Preprint
Publicado:	2024
Materias:	Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Gained in Translation: Privileged Pairwise Judges Enhance Multilingual Reasoning
por: Sutawika, Lintang, et al.
Publicado: (2026)

TextGames: Learning to Self-Play Text-Based Puzzle Games via Language Model Reasoning
por: Hudi, Frederikus, et al.
Publicado: (2025)

LinguAlchemy: Fusing Typological and Geographical Elements for Unseen Language Generalization
por: Adilazuarda, Muhammad Farid, et al.
Publicado: (2024)

Not How Many, But Which: Parameter Placement in Low-Rank Adaptation
por: Sehanobish, Arijit, et al.
Publicado: (2026)

Multimodal Large Language Models for Low-Resource Languages: A Case Study for Basque
por: Arana, Lukas, et al.
Publicado: (2025)

Crosslingual Reasoning through Test-Time Scaling
por: Yong, Zheng-Xin, et al.
Publicado: (2025)

Do Language Models Understand Honorific Systems in Javanese?
por: Farhansyah, Mohammad Rifqi, et al.
Publicado: (2025)

MINERS: Multilingual Language Models as Semantic Retrievers
por: Winata, Genta Indra, et al.
Publicado: (2024)

Macaron: Controlled, Human-Written Benchmark for Multilingual and Multicultural Reasoning via Template-Filling
por: Elsetohy, Alaa, et al.
Publicado: (2026)

Vision Language Models are Confused Tourists
por: Irawan, Patrick Amadeus, et al.
Publicado: (2025)

Deep Neural Networks Can Learn Generalizable Same-Different Visual Relations
por: Tartaglini, Alexa R., et al.
Publicado: (2023)

What Causes Knowledge Loss in Multilingual Language Models?
por: Khelli, Maria, et al.
Publicado: (2025)

Suppressing Pink Elephants with Direct Principle Feedback
por: Castricato, Louis, et al.
Publicado: (2024)

La educación femenina en la sociedad nicaragüense: una contrariedad de la Revolución Liberal (1893-1909)
por: Alexander Zosa-Cano
Publicado: (2023)

Linguistics Theory Meets LLM: Code-Switched Text Generation via Equivalence Constrained Large Language Models
por: Kuwanto, Garry, et al.
Publicado: (2024)

IndoPref: A Multi-Domain Pairwise Preference Dataset for Indonesian
por: Wiyono, Vanessa Rebecca, et al.
Publicado: (2025)

Momento: Evaluating Persistent Memory and Reasoning with Multi-Session Agentic Conversations
por: Merin, Adril Putra, et al.
Publicado: (2026)

Towards Efficient and Robust VQA-NLE Data Generation with Large Vision-Language Models
por: Irawan, Patrick Amadeus, et al.
Publicado: (2024)

Spectral statistics and energy-gap scaling in $k-$local spin Hamiltonians
por: Dowarah, Sasanka
Publicado: (2025)

BertaQA: How Much Do Language Models Know About Local Culture?
por: Etxaniz, Julen, et al.
Publicado: (2024)

XNLIeu: a dataset for cross-lingual NLI in Basque
por: Heredia, Maite, et al.
Publicado: (2024)

Investigación en torno a la literatura infantil y juvenil
por: Xabier Etxaniz
Publicado: (2008)

La transmisión de valores en la literatura, desde la tradición oral hasta la LIJ actual
por: Xabier Etxaniz
Publicado: (2011)

The Common Pile v0.1: An 8TB Dataset of Public Domain and Openly Licensed Text
por: Kandpal, Nikhil, et al.
Publicado: (2025)

Narrativa y estética de la database
por: Laura Fattori
Publicado: (2019)

Component parts of the World Heat Flow Data Collection
por: Lovering, Thomas Seward
Publicado: (1948)

Entropy2Vec: Crosslingual Language Modeling Entropy as End-to-End Learnable Language Representations
por: Irawan, Patrick Amadeus, et al.
Publicado: (2025)

Cost-Efficient Estimation of General Abilities Across Benchmarks
por: Krumdick, Michael, et al.
Publicado: (2026)

No Free Labels: Limitations of LLM-as-a-Judge Without Human Grounding
por: Krumdick, Michael, et al.
Publicado: (2025)

MetaMetrics: Calibrating Metrics For Generation Tasks Using Human Preferences
por: Winata, Genta Indra, et al.
Publicado: (2024)

MetaMetrics-MT: Tuning Meta-Metrics for Machine Translation via Human Preference Calibration
por: Anugraha, David, et al.
Publicado: (2024)

The Effect of Sporadic Dormancy on Adaptation under Natural Selection: A Formal Theory
por: Chanda, Sasanka Sekhar
Publicado: (2025)

Autoconcepto físico y satisfacción corporal en mujeres adolescentes según el tipo de deporte practicado
por: Igor Esnaola Etxaniz
Publicado: (2005)

Investigación en la literatura juvenil vasca
por: Xabier Etxaniz Erle
Publicado: (1999)

Lineas de investigación en literatura infantil y juvenil
por: Xabier Etxaniz Erle
Publicado: (2000)

From Text to Emotion: Unveiling the Emotion Annotation Capabilities of LLMs
por: Niu, Minxue, et al.
Publicado: (2024)

Pangea: A Fully Open Multilingual Multimodal LLM for 39 Languages
por: Yue, Xiang, et al.
Publicado: (2024)

HiTZ at VarDial 2025 NorSID: Overcoming Data Scarcity with Language Transfer and Automatic Data Annotation
por: Bengoetxea, Jaione, et al.
Publicado: (2024)

Driving AR Experience to Purchase Intention: Examining the Role of Users' Immersiveness, Perceived Innovativeness and Engagement in Virtual Try‐On
por: Ruturaj Baber, et al.
Publicado: (2026)

A Novel Computational and Modeling Foundation for Automatic Coherence Assessment
por: Maimon, Aviya, et al.
Publicado: (2023)