Image de couverture de livre

SYNTHEVAL: Hybrid Behavioral Testing of NLP Models with Synthetic CheckLists

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Zhao, Raoyuan, Köksal, Abdullatif, Liu, Yihong, Weissweiler, Leonie, Korhonen, Anna, Schütze, Hinrich
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Hybrid Human-LLM Corpus Construction and LLM Evaluation for Rare Linguistic Phenomena
par: Weissweiler, Leonie, et autres
Publié: (2024)

LongForm: Effective Instruction Tuning with Reverse Instructions
par: Köksal, Abdullatif, et autres
Publié: (2023)

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing
par: Zhao, Raoyuan, et autres
Publié: (2025)

How far can bias go? Tracing bias from pretraining data to alignment
par: Thaler, Marion, et autres
Publié: (2024)

TurkishMMLU: Measuring Massive Multitask Language Understanding in Turkish
par: Yüksel, Arda, et autres
Publié: (2024)

Consistent Document-Level Relation Extraction via Counterfactuals
par: Modarressi, Ali, et autres
Publié: (2024)

Large Reasoning Models Are (Not Yet) Multilingual Latent Reasoners
par: Liu, Yihong, et autres
Publié: (2026)

CRAFT Your Dataset: Task-Specific Synthetic Dataset Generation Through Corpus Retrieval and Augmentation
par: Ziegler, Ingo, et autres
Publié: (2024)

Crosslingual On-Policy Self-Distillation for Multilingual Reasoning
par: Liu, Yihong, et autres
Publié: (2026)

A Comprehensive Evaluation of Multilingual Chain-of-Thought Reasoning: Performance, Consistency, and Faithfulness Across Languages
par: Zhao, Raoyuan, et autres
Publié: (2025)

MURI: High-Quality Instruction Tuning Datasets for Low-Resource Languages via Reverse Instructions
par: Köksal, Abdullatif, et autres
Publié: (2024)

Evaluating Robustness of Large Language Models Against Multilingual Typographical Errors
par: Zhao, Raoyuan, et autres
Publié: (2025)

Taxonomy-based CheckList for Large Language Model Evaluation
par: Zhang, Damin
Publié: (2023)

ReverseMath: Answer Inversion for Scalable and Verifiable Mathematical Problem Generation
par: Zhao, Raoyuan, et autres
Publié: (2026)

MemLLM: Finetuning LLMs to Use An Explicit Read-Write Memory
par: Modarressi, Ali, et autres
Publié: (2024)

Verbing Weirds Language (Models): Evaluation of English Zero-Derivation in Five LLMs
par: Mortensen, David R., et autres
Publié: (2024)

Derivational Morphology Reveals Analogical Generalization in Large Language Models
par: Hofmann, Valentin, et autres
Publié: (2024)

Constructions Are So Difficult That Even Large Language Models Get Them Right for the Wrong Reasons
par: Zhou, Shijia, et autres
Publié: (2024)

Evaluating Contextually Mediated Factual Recall in Multilingual Large Language Models
par: Liu, Yihong, et autres
Publié: (2026)

Breaking the Script Barrier in Multilingual Pre-Trained Language Models with Transliteration-Based Post-Training Alignment
par: Xhelili, Orgest, et autres
Publié: (2024)

Why Better Cross-Lingual Alignment Fails for Better Cross-Lingual Transfer: Case of Encoders
par: Veitsman, Yana, et autres
Publié: (2026)

Beyond Input Understanding: Diagnosing Multilingual Mathematical Reasoning with Directed Acyclic Trace Graphs
par: Zhang, Jiaqiao, et autres
Publié: (2026)

Relational Linearity is a Predictor of Hallucinations
par: Lu, Yuetian, et autres
Publié: (2026)

TransliCo: A Contrastive Learning Framework to Address the Script Barrier in Multilingual Pretrained Language Models
par: Liu, Yihong, et autres
Publié: (2024)

TransMI: A Framework to Create Strong Baselines from Multilingual Pretrained Language Models for Transliterated Data
par: Liu, Yihong, et autres
Publié: (2024)

Your Pretrained Model Tells the Difficulty Itself: A Self-Adaptive Curriculum Learning Paradigm for Natural Language Understanding
par: Feng, Qi, et autres
Publié: (2025)

HYPEROFA: Expanding LLM Vocabulary to New Languages via Hypernetwork-Based Embedding Initialization
par: Özeren, Enes, et autres
Publié: (2025)

How Programming Concepts and Neurons Are Shared in Code Language Models
par: Kargaran, Amir Hossein, et autres
Publié: (2025)

MoSECroT: Model Stitching with Static Word Embeddings for Crosslingual Zero-shot Transfer
par: Ye, Haotian, et autres
Publié: (2024)

Left, Right, or Center? Evaluating LLM Framing in News Classification and Generation
par: Kennedy, Molly, et autres
Publié: (2026)

OFA: A Framework of Initializing Unseen Subword Embeddings for Efficient Large-scale Multilingual Continued Pretraining
par: Liu, Yihong, et autres
Publié: (2023)

On the Entity-Level Alignment in Crosslingual Consistency
par: Liu, Yihong, et autres
Publié: (2025)

Exploring the Role of Transliteration in In-Context Learning for Low-resource Languages Written in Non-Latin Scripts
par: Ma, Chunlan, et autres
Publié: (2024)

BabyLM's First Constructions: Causal probing provides a signal of learning
par: Rozner, Joshua, et autres
Publié: (2025)

Linguistic Generalizations are not Rules: Impacts on Evaluation of LMs
par: Weissweiler, Leonie, et autres
Publié: (2025)

Enhancing Robustness of Autoregressive Language Models against Orthographic Attacks via Pixel-based Approach
par: Yang, Han, et autres
Publié: (2025)

Both Direct and Indirect Evidence Contribute to Dative Alternation Preferences in Language Models
par: Yao, Qing, et autres
Publié: (2025)

LangSAMP: Language-Script Aware Multilingual Pretraining
par: Liu, Yihong, et autres
Publié: (2024)

Understanding In-Context Machine Translation for Low-Resource Languages: A Case Study on Manchu
par: Pei, Renhao, et autres
Publié: (2025)

Refusal Direction is Universal Across Safety-Aligned Languages
par: Wang, Xinpeng, et autres
Publié: (2025)