Cover Image

Contrastive Decoding for Synthetic Data Generation in Low-Resource Language Modeling

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Ulm, Jannek, Du, Kevin, Snæbjarnarson, Vésteinn
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Gumbel Counterfactual Generation From Language Models
by: Ravfogel, Shauli, et al.
Published: (2024)

Activation Scaling for Steering and Interpreting Language Models
by: Stoehr, Niklas, et al.
Published: (2024)

TharuChat: Bootstrapping Large Language Models for a Low-Resource Language via Synthetic Data and Human Validation
by: Panth, Prajwal, et al.
Published: (2026)

LoQT: Low-Rank Adapters for Quantized Pretraining
by: Loeschcke, Sebastian, et al.
Published: (2024)

Active Layer-Contrastive Decoding Reduces Hallucination in Large Language Model Generation
by: Zhang, Hongxiang, et al.
Published: (2025)

Better as Generators Than Classifiers: Leveraging LLMs and Synthetic Data for Low-Resource Multilingual Classification
by: Pecher, Branislav, et al.
Published: (2026)

ConLID: Supervised Contrastive Learning for Low-Resource Language Identification
by: Foroutan, Negar, et al.
Published: (2025)

Dependency Parsing Across the Resource Spectrum: Evaluating Architectures on High and Low-Resource Languages
by: Guan, Kevin, et al.
Published: (2026)

LexC-Gen: Generating Data for Extremely Low-Resource Languages with Large Language Models and Bilingual Lexicons
by: Yong, Zheng-Xin, et al.
Published: (2024)

DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models
by: Chuang, Yung-Sung, et al.
Published: (2023)

Distillation Contrastive Decoding: Improving LLMs Reasoning with Contrastive Decoding and Distillation
by: Phan, Phuc, et al.
Published: (2024)

BARE: Leveraging Base Language Models for Few-Shot Synthetic Data Generation
by: Zhu, Alan, et al.
Published: (2025)

DiffLM: Controllable Synthetic Data Generation via Diffusion Language Models
by: Zhou, Ying, et al.
Published: (2024)

Text2Data: Low-Resource Data Generation with Textual Control
by: Wang, Shiyu, et al.
Published: (2024)

Kakugo: Distillation of Low-Resource Languages into Small Language Models
by: Devine, Peter, et al.
Published: (2026)

Fill In The Gaps: Model Calibration and Generalization with Synthetic Data
by: Ba, Yang, et al.
Published: (2024)

A Multi-Faceted Evaluation Framework for Assessing Synthetic Data Generated by Large Language Models
by: Yuan, Yefeng, et al.
Published: (2024)

Socially Aware Synthetic Data Generation for Suicidal Ideation Detection Using Large Language Models
by: Ghanadian, Hamideh, et al.
Published: (2024)

Language Ranker: A Metric for Quantifying LLM Performance Across High and Low-Resource Languages
by: Li, Zihao, et al.
Published: (2024)

Large Multimodal Models for Low-Resource Languages: A Survey
by: Lupascu, Marian, et al.
Published: (2025)

CodecLM: Aligning Language Models with Tailored Synthetic Data
by: Wang, Zifeng, et al.
Published: (2024)

RAVEN: In-Context Learning with Retrieval-Augmented Encoder-Decoder Language Models
by: Huang, Jie, et al.
Published: (2023)

Generalizable and Stable Finetuning of Pretrained Language Models on Low-Resource Texts
by: Somayajula, Sai Ashish, et al.
Published: (2024)

Small Language Models for Privacy-Preserving Clinical Information Extraction in Low-Resource Languages
by: Ghaffarzadeh-Esfahani, Mohammadreza, et al.
Published: (2026)

Emotion Classification in Low and Moderate Resource Languages
by: Tafreshi, Shabnam, et al.
Published: (2024)

AI-Generated Text Detection in Low-Resource Languages: A Case Study on Urdu
by: Ammar, Muhammad, et al.
Published: (2025)

Few-Shot Cross-Lingual Transfer for Prompting Large Language Models in Low-Resource Languages
by: Toukmaji, Christopher
Published: (2024)

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models
by: Raj, Bharath, et al.
Published: (2024)

Reasoning-Driven Synthetic Data Generation and Evaluation
by: Davidson, Tim R., et al.
Published: (2026)

Paired by the Teacher: Turning Unpaired Data into High-Fidelity Pairs for Low-Resource Text Generation
by: Lu, Yen-Ju, et al.
Published: (2025)

Enhancing Clinical Documentation with Synthetic Data: Leveraging Generative Models for Improved Accuracy
by: Biswas, Anjanava, et al.
Published: (2024)

TURNA: A Turkish Encoder-Decoder Language Model for Enhanced Understanding and Generation
by: Uludoğan, Gökçe, et al.
Published: (2024)

Decoding-time Realignment of Language Models
by: Liu, Tianlin, et al.
Published: (2024)

Controlled Decoding from Language Models
by: Mudgal, Sidharth, et al.
Published: (2023)

Looking to Learn: Token-wise Dynamic Gating for Low-Resource Vision-Language Modelling
by: Ganescu, Bianca-Mihaela, et al.
Published: (2025)

How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse
by: Seddik, Mohamed El Amine, et al.
Published: (2024)

Surveying the Effects of Quality, Diversity, and Complexity in Synthetic Data From Large Language Models
by: Havrilla, Alex, et al.
Published: (2024)

CaLM: Contrasting Large and Small Language Models to Verify Grounded Generation
by: Hsu, I-Hung, et al.
Published: (2024)

Out-of-Distribution Detection using Synthetic Data Generation
by: Abbas, Momin, et al.
Published: (2025)

Dynamic Context Evolution for Scalable Synthetic Data Generation
by: Lingo, Ryan, et al.
Published: (2026)