Image de couverture de livre

Lessons from the Trenches on Reproducible Evaluation of Language Models

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Biderman, Stella, Schoelkopf, Hailey, Sutawika, Lintang, Gao, Leo, Tow, Jonathan, Abbasi, Baber, Aji, Alham Fikri, Ammanamanchi, Pawan Sasanka, Black, Sidney, Clive, Jordan, DiPofi, Anthony, Etxaniz, Julen, Fattori, Benjamin, Forde, Jessica Zosa, Foster, Charles, Hsu, Jeffrey, Jaiswal, Mimansa, Lee, Wilson Y., Li, Haonan, Lovering, Charles, Muennighoff, Niklas, Pavlick, Ellie, Phang, Jason, Skowron, Aviya, Tan, Samson, Tang, Xiangru, Wang, Kevin A., Winata, Genta Indra, Yvon, François, Zou, Andy
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Gained in Translation: Privileged Pairwise Judges Enhance Multilingual Reasoning
par: Sutawika, Lintang, et autres
Publié: (2026)

TextGames: Learning to Self-Play Text-Based Puzzle Games via Language Model Reasoning
par: Hudi, Frederikus, et autres
Publié: (2025)

LinguAlchemy: Fusing Typological and Geographical Elements for Unseen Language Generalization
par: Adilazuarda, Muhammad Farid, et autres
Publié: (2024)

Not How Many, But Which: Parameter Placement in Low-Rank Adaptation
par: Sehanobish, Arijit, et autres
Publié: (2026)

Multimodal Large Language Models for Low-Resource Languages: A Case Study for Basque
par: Arana, Lukas, et autres
Publié: (2025)

Crosslingual Reasoning through Test-Time Scaling
par: Yong, Zheng-Xin, et autres
Publié: (2025)

Do Language Models Understand Honorific Systems in Javanese?
par: Farhansyah, Mohammad Rifqi, et autres
Publié: (2025)

MINERS: Multilingual Language Models as Semantic Retrievers
par: Winata, Genta Indra, et autres
Publié: (2024)

Macaron: Controlled, Human-Written Benchmark for Multilingual and Multicultural Reasoning via Template-Filling
par: Elsetohy, Alaa, et autres
Publié: (2026)

Vision Language Models are Confused Tourists
par: Irawan, Patrick Amadeus, et autres
Publié: (2025)

Deep Neural Networks Can Learn Generalizable Same-Different Visual Relations
par: Tartaglini, Alexa R., et autres
Publié: (2023)

What Causes Knowledge Loss in Multilingual Language Models?
par: Khelli, Maria, et autres
Publié: (2025)

Suppressing Pink Elephants with Direct Principle Feedback
par: Castricato, Louis, et autres
Publié: (2024)

La educación femenina en la sociedad nicaragüense: una contrariedad de la Revolución Liberal (1893-1909)
par: Alexander Zosa-Cano
Publié: (2023)

Linguistics Theory Meets LLM: Code-Switched Text Generation via Equivalence Constrained Large Language Models
par: Kuwanto, Garry, et autres
Publié: (2024)

IndoPref: A Multi-Domain Pairwise Preference Dataset for Indonesian
par: Wiyono, Vanessa Rebecca, et autres
Publié: (2025)

Momento: Evaluating Persistent Memory and Reasoning with Multi-Session Agentic Conversations
par: Merin, Adril Putra, et autres
Publié: (2026)

Towards Efficient and Robust VQA-NLE Data Generation with Large Vision-Language Models
par: Irawan, Patrick Amadeus, et autres
Publié: (2024)

Spectral statistics and energy-gap scaling in $k-$local spin Hamiltonians
par: Dowarah, Sasanka
Publié: (2025)

BertaQA: How Much Do Language Models Know About Local Culture?
par: Etxaniz, Julen, et autres
Publié: (2024)

XNLIeu: a dataset for cross-lingual NLI in Basque
par: Heredia, Maite, et autres
Publié: (2024)

Investigación en torno a la literatura infantil y juvenil
par: Xabier Etxaniz
Publié: (2008)

La transmisión de valores en la literatura, desde la tradición oral hasta la LIJ actual
par: Xabier Etxaniz
Publié: (2011)

The Common Pile v0.1: An 8TB Dataset of Public Domain and Openly Licensed Text
par: Kandpal, Nikhil, et autres
Publié: (2025)

Narrativa y estética de la database
par: Laura Fattori
Publié: (2019)

Component parts of the World Heat Flow Data Collection
par: Lovering, Thomas Seward
Publié: (1948)

Entropy2Vec: Crosslingual Language Modeling Entropy as End-to-End Learnable Language Representations
par: Irawan, Patrick Amadeus, et autres
Publié: (2025)

Cost-Efficient Estimation of General Abilities Across Benchmarks
par: Krumdick, Michael, et autres
Publié: (2026)

No Free Labels: Limitations of LLM-as-a-Judge Without Human Grounding
par: Krumdick, Michael, et autres
Publié: (2025)

MetaMetrics: Calibrating Metrics For Generation Tasks Using Human Preferences
par: Winata, Genta Indra, et autres
Publié: (2024)

MetaMetrics-MT: Tuning Meta-Metrics for Machine Translation via Human Preference Calibration
par: Anugraha, David, et autres
Publié: (2024)

The Effect of Sporadic Dormancy on Adaptation under Natural Selection: A Formal Theory
par: Chanda, Sasanka Sekhar
Publié: (2025)

Autoconcepto físico y satisfacción corporal en mujeres adolescentes según el tipo de deporte practicado
par: Igor Esnaola Etxaniz
Publié: (2005)

Investigación en la literatura juvenil vasca
par: Xabier Etxaniz Erle
Publié: (1999)

Lineas de investigación en literatura infantil y juvenil
par: Xabier Etxaniz Erle
Publié: (2000)

From Text to Emotion: Unveiling the Emotion Annotation Capabilities of LLMs
par: Niu, Minxue, et autres
Publié: (2024)

Pangea: A Fully Open Multilingual Multimodal LLM for 39 Languages
par: Yue, Xiang, et autres
Publié: (2024)

Driving AR Experience to Purchase Intention: Examining the Role of Users' Immersiveness, Perceived Innovativeness and Engagement in Virtual Try‐On
par: Ruturaj Baber, et autres
Publié: (2026)

HiTZ at VarDial 2025 NorSID: Overcoming Data Scarcity with Language Transfer and Automatic Data Annotation
par: Bengoetxea, Jaione, et autres
Publié: (2024)

A Novel Computational and Modeling Foundation for Automatic Coherence Assessment
par: Maimon, Aviya, et autres
Publié: (2023)