Cover Image

Configurable Safety Tuning of Language Models with Synthetic Preference Data

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Gallego, Victor
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Configurable Preference Tuning with Rubric-Guided Synthetic Data
by: Gallego, Víctor
Published: (2025)

MetaSC: Test-Time Safety Specification Optimization for Language Models
by: Gallego, Víctor
Published: (2025)

Self-Boosting Large Language Models with Synthetic Preference Data
by: Dong, Qingxiu, et al.
Published: (2024)

Discovering Agentic Safety Specifications from 1-Bit Danger Signals
by: Gallego, Víctor
Published: (2026)

Domain-Adaptation through Synthetic Data: Fine-Tuning Large Language Models for German Law
by: Bashir, Ali Hamza, et al.
Published: (2026)

Safety-Aware Fine-Tuning of Large Language Models
by: Choi, Hyeong Kyu, et al.
Published: (2024)

Icon$^{2}$: Aligning Large Language Models Using Self-Synthetic Preference Data via Inherent Regulation
by: Chen, Qiyuan, et al.
Published: (2025)

Course-Correction: Safety Alignment Using Synthetic Preferences
by: Xu, Rongwu, et al.
Published: (2024)

Scaling Laws of Synthetic Data for Language Models
by: Qin, Zeyu, et al.
Published: (2025)

SyntheT2C: Generating Synthetic Data for Fine-Tuning Large Language Models on the Text2Cypher Task
by: Zhong, Ziije, et al.
Published: (2024)

Fine Tuning Large Language Models for Medicine: The Role and Importance of Direct Preference Optimization
by: Savage, Thomas, et al.
Published: (2024)

Beyond Under-Alignment: Atomic Preference Enhanced Factuality Tuning for Large Language Models
by: Yuan, Hongbang, et al.
Published: (2024)

Safer-Instruct: Aligning Language Models with Automated Preference Data
by: Shi, Taiwei, et al.
Published: (2023)

Merging Improves Self-Critique Against Jailbreak Attacks
by: Gallego, Victor
Published: (2024)

Specification Self-Correction: Mitigating In-Context Reward Hacking Through Test-Time Refinement
by: Gallego, Víctor
Published: (2025)

Bi-Factorial Preference Optimization: Balancing Safety-Helpfulness in Language Models
by: Zhang, Wenxuan, et al.
Published: (2024)

Synthetic Data Generation for Phrase Break Prediction with Large Language Model
by: Lee, Hoyeon, et al.
Published: (2025)

Grounding Synthetic Data Evaluations of Language Models in Unsupervised Document Corpora
by: Majurski, Michael, et al.
Published: (2025)

Precision Knowledge Editing: Enhancing Safety in Large Language Models
by: Li, Xuying, et al.
Published: (2024)

The Fellowship of the LLMs: Multi-Model Workflows for Synthetic Preference Optimization Dataset Generation
by: Arif, Samee, et al.
Published: (2024)

CantonMT: Cantonese to English NMT Platform with Fine-Tuned Models Using Synthetic Back-Translation Data
by: Hong, Kung Yin, et al.
Published: (2024)

Preference Packing: Efficient Preference Optimization for Large Language Models
by: Cho, Jaekyung
Published: (2026)

CURATRON: Complete and Robust Preference Data for Rigorous Alignment of Large Language Models
by: Nguyen, Son The, et al.
Published: (2024)

Forging Time Series with Language: A Large Language Model Approach to Synthetic Data Generation
by: Rousseau, Cécile, et al.
Published: (2025)

ANGOFA: Leveraging OFA Embedding Initialization and Synthetic Data for Angolan Language Model
by: Quinjica, Osvaldo Luamba, et al.
Published: (2024)

Scaling Data Diversity for Fine-Tuning Language Models in Human Alignment
by: Song, Feifan, et al.
Published: (2024)

Capturing Nuanced Preferences: Preference-Aligned Distillation for Small Language Models
by: Gu, Yanggan, et al.
Published: (2025)

Advancing Speech Language Models by Scaling Supervised Fine-Tuning with Over 60,000 Hours of Synthetic Speech Dialogue Data
by: Zhao, Shuaijiang, et al.
Published: (2024)

Scaling Instruction-Tuned LLMs to Million-Token Contexts via Hierarchical Synthetic Data Generation
by: He, Linda, et al.
Published: (2025)

GraphGen: Enhancing Supervised Fine-Tuning for LLMs with Knowledge-Driven Synthetic Data Generation
by: Chen, Zihong, et al.
Published: (2025)

SafeMERGE: Preserving Safety Alignment in Fine-Tuned Large Language Models via Selective Layer-Wise Model Merging
by: Djuhera, Aladin, et al.
Published: (2025)

Alignment Tuning for Large Language Models: A Data-Centric Lens on Alignment Data Pipelines
by: Song, Hwanjun
Published: (2026)

A Modular Approach for Clinical SLMs Driven by Synthetic Data with Pre-Instruction Tuning, Model Merging, and Clinical-Tasks Alignment
by: Corbeil, Jean-Philippe, et al.
Published: (2025)

Modular Techniques for Synthetic Long-Context Data Generation in Language Model Training and Evaluation
by: Subramanian, Seganrasan, et al.
Published: (2025)

Assessing and Mitigating Data Memorization Risks in Fine-Tuned Large Language Models
by: Ramakrishnan, Badrinath, et al.
Published: (2025)

Aligning Large Language Models with Searcher Preferences
by: Wu, Wei, et al.
Published: (2026)

Probing Persona-Dependent Preferences in Language Models
by: Gilg, Oscar, et al.
Published: (2026)

Align Generative Artificial Intelligence with Human Preferences: A Novel Large Language Model Fine-Tuning Method for Online Review Management
by: Wang, Yanan, et al.
Published: (2026)

Generating Synthetic Datasets for Few-shot Prompt Tuning
by: Guo, Xu, et al.
Published: (2024)

West-of-N: Synthetic Preferences for Self-Improving Reward Models
by: Pace, Alizée, et al.
Published: (2024)