Cover Image

Evaluating Large Language Models with Tests of Spanish as a Foreign Language: Pass or Fail?

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Mayor-Rocher, Marina, Melero, Nina, Merino-Gómez, Elena, Grandury, María, Conde, Javier, Reviriego, Pedro
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

It's the same but not the same: Do LLMs distinguish Spanish varieties?
by: Mayor-Rocher, Marina, et al.
Published: (2025)

Spanish and LLM Benchmarks: is MMLU Lost in Translation?
by: Plaza, Irene, et al.
Published: (2024)

Large Language Models and Book Summarization: Reading or Remembering, Which Is Better?
by: Fu, Tairan, et al.
Published: (2026)

Multiple Choice Questions: Reasoning Makes Large Language Models (LLMs) More Self-Confident, Especially When They are Wrong
by: Fu, Tairan, et al.
Published: (2025)

Establishing Vocabulary Tests as a Benchmark for Evaluating Large Language Models
by: Martínez, Gonzalo, et al.
Published: (2023)

Open Conversational LLMs do not know most Spanish words
by: Conde, Javier, et al.
Published: (2024)

Have Multimodal Large Language Models (MLLMs) Really Learned to Tell the Time on Analog Clocks?
by: Fu, Tairan, et al.
Published: (2025)

Lost in the Vibrations: Vision Language Models Fail the Dynamic Gauges Test
by: Fu, Tairan, et al.
Published: (2026)

Psycholinguistic Word Features: a New Approach for the Evaluation of LLMs Alignment with Humans
by: Conde, Javier, et al.
Published: (2025)

Is There a Case for Conversation Optimized Tokenizers in Large Language Models?
by: Ferrando, Raquel, et al.
Published: (2025)

Beware of Words: Evaluating the Lexical Diversity of Conversational LLMs using ChatGPT as Case Study
by: Martínez, Gonzalo, et al.
Published: (2024)

Why Do Large Language Models (LLMs) Struggle to Count Letters?
by: Fu, Tairan, et al.
Published: (2024)

The Generative Energy Arena (GEA): Incorporating Energy Awareness in Large Language Model (LLM) Human Evaluations
by: Arriaga, Carlos, et al.
Published: (2025)

To Words and Beyond: Probing Large Language Models for Sentence-Level Psycholinguistic Norms of Memorability and Reading Times
by: Clark, Thomas Hikaru, et al.
Published: (2026)

La Leaderboard: A Large Language Model Leaderboard for Spanish Varieties and Languages of Spain and Latin America
by: Grandury, María, et al.
Published: (2025)

Playing with words: Comparing the vocabulary and lexical diversity of ChatGPT and humans
by: Reviriego, Pedro, et al.
Published: (2023)

Speed and Conversational Large Language Models: Not All Is About Tokens per Second
by: Conde, Javier, et al.
Published: (2025)

Concurrent Linguistic Error Detection (CLED): a New Methodology for Error Detection in Large Language Models
by: Zhu, Jinhua, et al.
Published: (2024)

The #Somos600M Project: Generating NLP resources that represent the diversity of the languages from LATAM, the Caribbean, and Spain
by: Grandury, María
Published: (2024)

Artificial Intelligence and Misinformation in Art: Can Vision Language Models Judge the Hand or the Machine Behind the Canvas?
by: Fu, Tarian, et al.
Published: (2025)

Adding LLMs to the psycholinguistic norming toolbox: A practical guide to getting the most out of human ratings
by: Conde, Javier, et al.
Published: (2025)

Large Language Models Pass the Turing Test
by: Jones, Cameron R., et al.
Published: (2025)

How does fine-tuning improve sensorimotor representations in large language models?
by: Wu, Minghua, et al.
Published: (2026)

Using large language models to estimate features of multi-word expressions: Concreteness, valence, arousal
by: Martínez, Gonzalo, et al.
Published: (2024)

Lost in Sampling: Assessing Lexical Reachability in LLMs via the Word Coverage Score (WCS)
by: Awad, Samer, et al.
Published: (2026)

Can ChatGPT Learn to Count Letters?
by: Conde, Javier, et al.
Published: (2025)

Failing to Falsify: Evaluating and Mitigating Confirmation Bias in Language Models
by: Jhaveri, Ayush Rajesh, et al.
Published: (2026)

Cost-of-Pass: An Economic Framework for Evaluating Language Models
by: Erol, Mehmet Hamza, et al.
Published: (2025)

A Hard Nut to Crack: Idiom Detection with Conversational Large Language Models
by: Fornaciari, Francesca De Luca, et al.
Published: (2024)

Information Extraction from Electricity Invoices with General-Purpose Large Language Models
by: Gómez, Javier, et al.
Published: (2026)

Investigating the translation capabilities of Large Language Models trained on parallel data only
by: Gilabert, Javier García, et al.
Published: (2024)

Recursive InPainting (RIP): how much information is lost under recursive inferences?
by: Conde, Javier, et al.
Published: (2024)

Why do Large Language Models Fail in Low-resource Translation? Unraveling the Token Dynamics of Large Language Models for Machine Translation
by: Qian, Shenbin, et al.
Published: (2026)

MEL: Legal Spanish Language Model
by: Sánchez, David Betancur, et al.
Published: (2025)

Prompt-Based Simplification for Plain Language using Spanish Language Models
by: Moreno, Lourdes, et al.
Published: (2025)

Language Models Fail to Introspect About Their Knowledge of Language
by: Song, Siyuan, et al.
Published: (2025)

Compositional Bias Control in Large Language Models: Preference Learning Fails, Supervision Succeeds
by: Mahesh, Atij
Published: (2025)

Ask Again, Then Fail: Large Language Models' Vacillations in Judgment
by: Xie, Qiming, et al.
Published: (2023)

Gender Trouble in Language Models: An Empirical Audit Guided by Gender Performativity Theory
by: Hafner, Franziska Sofia, et al.
Published: (2025)

Fine-Tuning or Fine-Failing? Debunking Performance Myths in Large Language Models
by: Barnett, Scott, et al.
Published: (2024)