Cover Image

From Rosetta to Match-Up: A Paired Corpus of Linguistic Puzzles with Human and LLM Benchmarks

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Majmudar, Neh, Huang, Anne, Hu, Jinfan Frank, Filatova, Elena
Format:	Preprint
Published:	2026
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Can LLMs Generate and Solve Linguistic Olympiad Puzzles?
by: Majmudar, Neh, et al.
Published: (2025)

Tokenization Strategies for Low-Resource Agglutinative Languages in Word2Vec: Case Study on Turkish and Finnish
by: Hu, Jinfan Frank
Published: (2025)

Hybrid Human-LLM Corpus Construction and LLM Evaluation for Rare Linguistic Phenomena
by: Weissweiler, Leonie, et al.
Published: (2024)

LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
by: Bean, Andrew M., et al.
Published: (2024)

TurBLiMP: A Turkish Benchmark of Linguistic Minimal Pairs
by: Başar, Ezgi, et al.
Published: (2025)

RuBLiMP: Russian Benchmark of Linguistic Minimal Pairs
by: Taktasheva, Ekaterina, et al.
Published: (2024)

Agent-Driven Corpus Linguistics: A Framework for Autonomous Linguistic Discovery
by: Yu, Jia, et al.
Published: (2026)

Rosetta-PL: Propositional Logic as a Benchmark for Large Language Model Reasoning
by: Baek, Shaun, et al.
Published: (2025)

UNVEILING: What Makes Linguistics Olympiad Puzzles Tricky for LLMs?
by: Choudhary, Mukund, et al.
Published: (2025)

QFrBLiMP: a Quebec-French Benchmark of Linguistic Minimal Pairs
by: Beauchemin, David, et al.
Published: (2025)

CxMP: A Linguistic Minimal-Pair Benchmark for Evaluating Constructional Understanding in Language Models
by: Oba, Miyu, et al.
Published: (2026)

MultiBLiMP 1.0: A Massively Multilingual Benchmark of Linguistic Minimal Pairs
by: Jumelet, Jaap, et al.
Published: (2025)

Probing Large Language Models in Reasoning and Translating Complex Linguistic Puzzles
by: Lin, Zheng-Lin, et al.
Published: (2025)

Linguistic Minimal Pairs Elicit Linguistic Similarity in Large Language Models
by: Zhou, Xinyu, et al.
Published: (2024)

PuzzlePlex: Benchmarking Foundation Models on Reasoning and Planning with Puzzles
by: Long, Yitao, et al.
Published: (2025)

Using Natural Language Processing and Networks to Automate Structured Literature Reviews: An Application to Farmers Climate Change Adaptation
by: Gil-Clavel, Sofia, et al.
Published: (2023)

A Systematic Assessment of Language Models with Linguistic Minimal Pairs in Chinese
by: Liu, Yikang, et al.
Published: (2024)

From Arabic Text to Puzzles: LLM-Driven Development of Arabic Educational Crosswords
by: Zeinalipour, Kamyar, et al.
Published: (2025)

CorpusQA: A 10 Million Token Benchmark for Corpus-Level Analysis and Reasoning
by: Lu, Zhiyuan, et al.
Published: (2026)

Leveraging the Cross-Domain & Cross-Linguistic Corpus for Low Resource NMT: A Case Study On Bhili-Hindi-English Parallel Corpus
by: Singh, Pooja, et al.
Published: (2025)

Mind the Gap: Linguistic Divergence and Adaptation Strategies in Human-LLM Assistant vs. Human-Human Interactions
by: Zhang, Fulei, et al.
Published: (2025)

LatEval: An Interactive LLMs Evaluation Benchmark with Incomplete Information from Lateral Thinking Puzzles
by: Huang, Shulin, et al.
Published: (2023)

QFrCoLA: a Quebec-French Corpus of Linguistic Acceptability Judgments
by: Beauchemin, David, et al.
Published: (2025)

NAVCON: A Cognitively Inspired and Linguistically Grounded Corpus for Vision and Language Navigation
by: Wanchoo, Karan, et al.
Published: (2024)

Benchmark^2: Systematic Evaluation of LLM Benchmarks
by: Qian, Qi, et al.
Published: (2026)

ToTRL: Unlock LLM Tree-of-Thoughts Reasoning Potential through Puzzles Solving
by: Wu, Haoyuan, et al.
Published: (2025)

Syntax as a Rosetta Stone: Universal Dependencies for In-Context Coptic Translation
by: Purushothama, Abhishek, et al.
Published: (2026)

Beyond the Rosetta Stone: Unification Forces in Generalization Dynamics
by: Blum, Carter, et al.
Published: (2025)

Scaling Up, Speeding Up: A Benchmark of Speculative Decoding for Efficient LLM Test-Time Scaling
by: Sun, Shengyin, et al.
Published: (2025)

Targeted Linguistic Analysis of Sign Language Models with Minimal Translation Pairs
by: Karabüklü, Serpil, et al.
Published: (2026)

HumanLLM: Benchmarking and Improving LLM Anthropomorphism via Human Cognitive Patterns
by: Wang, Xintao, et al.
Published: (2026)

Rosetta Stone at KSAA-RD Shared Task: A Hop From Language Modeling To Word--Definition Alignment
by: ElBakry, Ahmed, et al.
Published: (2023)

Test-Time-Matching: Decouple Personality, Memory, and Linguistic Style in LLM-based Role-Playing Language Agent
by: Zhan, Xiaoyu, et al.
Published: (2025)

Can AI-Generated Persuasion Be Detected? Persuaficial Benchmark and AI vs. Human Linguistic Differences
by: Modzelewski, Arkadiusz, et al.
Published: (2026)

GPT-NL Public Corpus: A Permissively Licensed, Dutch-First Dataset for LLM Pre-training
by: van Oort, Jesse, et al.
Published: (2026)

No Universal Courtesy: A Cross-Linguistic, Multi-Model Study of Politeness Effects on LLMs Using the PLUM Corpus
by: Mehta, Hitesh, et al.
Published: (2026)

ILiAD: An Interactive Corpus for Linguistic Annotated Data from Twitter Posts
by: Gonzalez, Simon
Published: (2024)

Benchmarking Linguistic Diversity of Large Language Models
by: Guo, Yanzhu, et al.
Published: (2024)

Handling Uncertainty in Health Data using Generative Algorithms
by: Loodaricheh, Mahdi Arab, et al.
Published: (2025)

Irish-BLiMP: A Linguistic Benchmark for Evaluating Human and Language Model Performance in a Low-Resource Setting
by: McGiff, Josh, et al.
Published: (2025)