Copertina

CxMP: A Linguistic Minimal-Pair Benchmark for Evaluating Constructional Understanding in Language Models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Oba, Miyu, Sugawara, Saku
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Can Language Models Induce Grammatical Knowledge from Indirect Evidence?
di: Oba, Miyu, et al.
Pubblicazione: (2024)

RuBLiMP: Russian Benchmark of Linguistic Minimal Pairs
di: Taktasheva, Ekaterina, et al.
Pubblicazione: (2024)

TurBLiMP: A Turkish Benchmark of Linguistic Minimal Pairs
di: Başar, Ezgi, et al.
Pubblicazione: (2025)

A Dual-Task Paradigm to Investigate Sentence Comprehension Strategies in Language Models
di: Emura, Rei, et al.
Pubblicazione: (2026)

What Makes Language Models Good-enough?
di: Asami, Daiki, et al.
Pubblicazione: (2024)

QFrBLiMP: a Quebec-French Benchmark of Linguistic Minimal Pairs
di: Beauchemin, David, et al.
Pubblicazione: (2025)

Rationale-Aware Answer Verification by Pairwise Self-Evaluation
di: Kawabata, Akira, et al.
Pubblicazione: (2024)

Specification-Aware Machine Translation and Evaluation for Purpose Alignment
di: Kayano, Yoko, et al.
Pubblicazione: (2025)

MultiBLiMP 1.0: A Massively Multilingual Benchmark of Linguistic Minimal Pairs
di: Jumelet, Jaap, et al.
Pubblicazione: (2025)

C2: Scalable Rubric-Augmented Reward Modeling from Binary Preferences
di: Kawabata, Akira, et al.
Pubblicazione: (2026)

UrBLiMP: A Benchmark for Evaluating the Linguistic Competence of Large Language Models in Urdu
di: Adeeba, Farah, et al.
Pubblicazione: (2025)

Linguistic Minimal Pairs Elicit Linguistic Similarity in Large Language Models
di: Zhou, Xinyu, et al.
Pubblicazione: (2024)

Are Checklists Really Useful for Automatic Evaluation of Generative Tasks?
di: Furuhashi, Momoka, et al.
Pubblicazione: (2025)

A Systematic Assessment of Language Models with Linguistic Minimal Pairs in Chinese
di: Liu, Yikang, et al.
Pubblicazione: (2024)

Irish-BLiMP: A Linguistic Benchmark for Evaluating Human and Language Model Performance in a Low-Resource Setting
di: McGiff, Josh, et al.
Pubblicazione: (2025)

Targeted Linguistic Analysis of Sign Language Models with Minimal Translation Pairs
di: Karabüklü, Serpil, et al.
Pubblicazione: (2026)

TactfulToM: Do LLMs Have the Theory of Mind Ability to Understand White Lies?
di: Liu, Yiwei, et al.
Pubblicazione: (2025)

BabyLM Challenge: Exploring the Effect of Variation Sets on Language Model Training Efficiency
di: Haga, Akari, et al.
Pubblicazione: (2024)

BQA: Body Language Question Answering Dataset for Video Large Language Models
di: Ozaki, Shintaro, et al.
Pubblicazione: (2024)

Evaluating CxG Generalisation in LLMs via Construction-Based NLI Fine Tuning
di: Mackintosh, Tom, et al.
Pubblicazione: (2025)

Decoding Probing: Revealing Internal Linguistic Structures in Neural Language Models using Minimal Pairs
di: He, Linyang, et al.
Pubblicazione: (2024)

XCOMPS: A Multilingual Benchmark of Conceptual Minimal Pairs
di: He, Linyang, et al.
Pubblicazione: (2025)

What Matters in Memorizing and Recalling Facts? Multifaceted Benchmarks for Knowledge Probing in Language Models
di: Zhao, Xin, et al.
Pubblicazione: (2024)

Which Feedback Works for Whom? Differential Effects of LLM-Generated Feedback Elements Across Learner Profiles
di: Furuhashi, Momoka, et al.
Pubblicazione: (2026)

MoreHopQA: More Than Multi-hop Reasoning
di: Schnitzler, Julian, et al.
Pubblicazione: (2024)

Benchmarking Linguistic Diversity of Large Language Models
di: Guo, Yanzhu, et al.
Pubblicazione: (2024)

Language Models Learn Constructional Semantics, Not To Mention Syntax: Investigating LM Understanding of Paired-Focus Constructions
di: Scivetti, Wesley, et al.
Pubblicazione: (2026)

Quality Text, Robust Vision: The Role of Language in Enhancing Visual Robustness of Vision-Language Models
di: Waseda, Futa, et al.
Pubblicazione: (2025)

Holmes: A Benchmark to Assess the Linguistic Competence of Language Models
di: Waldis, Andreas, et al.
Pubblicazione: (2024)

Minimal Pair-Based Evaluation of Code-Switching
di: Sterner, Igor, et al.
Pubblicazione: (2025)

PragWorld: A Benchmark Evaluating LLMs' Local World Model under Minimal Linguistic Alterations and Conversational Dynamics
di: Vashistha, Sachin, et al.
Pubblicazione: (2025)

From Rosetta to Match-Up: A Paired Corpus of Linguistic Puzzles with Human and LLM Benchmarks
di: Majmudar, Neh, et al.
Pubblicazione: (2026)

TLUE: A Tibetan Language Understanding Evaluation Benchmark
di: Gao, Fan, et al.
Pubblicazione: (2025)

VLURes: Benchmarking VLM Visual and Linguistic Understanding in Low-Resource Languages
di: Atuhurra, Jesse, et al.
Pubblicazione: (2025)

Measuring Human Involvement in AI-Generated Text: A Case Study on Academic Writing
di: Guo, Yuchen, et al.
Pubblicazione: (2025)

Label Effects: Shared Heuristic Reliance in Trust Assessment by Humans and LLM-as-a-Judge
di: Sun, Xin, et al.
Pubblicazione: (2026)

Automatic Feedback Generation for Short Answer Questions using Answer Diagnostic Graphs
di: Furuhashi, Momoka, et al.
Pubblicazione: (2025)

BLUCK: A Benchmark Dataset for Bengali Linguistic Understanding and Cultural Knowledge
di: Kabir, Daeen, et al.
Pubblicazione: (2025)

Evaluating List Construction and Temporal Understanding capabilities of Large Language Models
di: Dumitru, Alexandru, et al.
Pubblicazione: (2025)

The Invalsi Benchmarks: measuring Linguistic and Mathematical understanding of Large Language Models in Italian
di: Puccetti, Giovanni, et al.
Pubblicazione: (2024)