Cover Image

Configurable Preference Tuning with Rubric-Guided Synthetic Data

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Gallego, Víctor
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Configurable Safety Tuning of Language Models with Synthetic Preference Data
by: Gallego, Victor
Published: (2024)

Optimsyn: Influence-Guided Rubrics Optimization for Synthetic Data Generation
by: Fan, Zhiting, et al.
Published: (2026)

Self-Preference Bias in Rubric-Based Evaluation of Large Language Models
by: Pombal, José, et al.
Published: (2026)

Rubric-Guided Process Reward for Stepwise Model Routing
by: Ye, Shenghao, et al.
Published: (2026)

Rubrics as an Attack Surface: Stealthy Preference Drift in LLM Judges
by: Ding, Ruomeng, et al.
Published: (2026)

Self-Boosting Large Language Models with Synthetic Preference Data
by: Dong, Qingxiu, et al.
Published: (2024)

ReviewGrounder: Improving Review Substantiveness with Rubric-Guided, Tool-Integrated Agents
by: Li, Zhuofeng, et al.
Published: (2026)

AdaRubric: Task-Adaptive Rubrics for Reliable LLM Agent Evaluation and Reward Learning
by: Ding, Liang
Published: (2026)

Generating Data-Driven Reasoning Rubrics for Domain-Adaptive Reward Modeling
by: Sanders, Kate, et al.
Published: (2026)

Specification Self-Correction: Mitigating In-Context Reward Hacking Through Test-Time Refinement
by: Gallego, Víctor
Published: (2025)

MetaSC: Test-Time Safety Specification Optimization for Language Models
by: Gallego, Víctor
Published: (2025)

Discovering Agentic Safety Specifications from 1-Bit Danger Signals
by: Gallego, Víctor
Published: (2026)

Merging Improves Self-Critique Against Jailbreak Attacks
by: Gallego, Victor
Published: (2024)

Rubrics to Tokens: Bridging Response-level Rubrics and Token-level Rewards in Instruction Following Tasks
by: Xu, Tianze, et al.
Published: (2026)

SibylSense: Adaptive Rubric Learning via Memory Tuning and Adversarial Probing
by: Xu, Yifei, et al.
Published: (2026)

Autorubric: Unifying Rubric-based LLM Evaluation
by: Rao, Delip, et al.
Published: (2026)

Scaling Instruction-Tuned LLMs to Million-Token Contexts via Hierarchical Synthetic Data Generation
by: He, Linda, et al.
Published: (2025)

GraphGen: Enhancing Supervised Fine-Tuning for LLMs with Knowledge-Driven Synthetic Data Generation
by: Chen, Zihong, et al.
Published: (2025)

Domain-Adaptation through Synthetic Data: Fine-Tuning Large Language Models for German Law
by: Bashir, Ali Hamza, et al.
Published: (2026)

ResearchRubrics: A Benchmark of Prompts and Rubrics For Evaluating Deep Research Agents
by: Sharma, Manasi, et al.
Published: (2025)

Icon$^{2}$: Aligning Large Language Models Using Self-Synthetic Preference Data via Inherent Regulation
by: Chen, Qiyuan, et al.
Published: (2025)

Automated Rubrics for Reliable Evaluation of Medical Dialogue Systems
by: Chen, Yinzhu, et al.
Published: (2026)

Xpertbench: Expert Level Tasks with Rubrics-Based Evaluation
by: Liu, Xue, et al.
Published: (2026)

AMARIS: A Memory-Augmented Rubric Improvement System for Rubric-Based Reinforcement Learning
by: Wu, Peilin, et al.
Published: (2026)

Generating Synthetic Datasets for Few-shot Prompt Tuning
by: Guo, Xu, et al.
Published: (2024)

Evaluating Legal Reasoning Traces with Legal Issue Tree Rubrics
by: Lee, Jinu, et al.
Published: (2025)

Confusion-Aware Rubric Optimization for LLM-based Automated Grading
by: Chu, Yucheng, et al.
Published: (2026)

DREsS: Dataset for Rubric-based Essay Scoring on EFL Writing
by: Yoo, Haneul, et al.
Published: (2024)

Reinforcement Learning with Rubric Anchors
by: Huang, Zenan, et al.
Published: (2025)

CantonMT: Cantonese to English NMT Platform with Fine-Tuned Models Using Synthetic Back-Translation Data
by: Hong, Kung Yin, et al.
Published: (2024)

Enhancing Judgment Document Generation via Agentic Legal Information Collection and Rubric-Guided Optimization
by: Su, Weihang, et al.
Published: (2026)

Semantic Loss Guided Data Efficient Supervised Fine Tuning for Safe Responses in LLMs
by: Lu, Yuxiao, et al.
Published: (2024)

Refined Direct Preference Optimization with Synthetic Data for Behavioral Alignment of LLMs
by: Gallego, Víctor
Published: (2024)

The Fellowship of the LLMs: Multi-Model Workflows for Synthetic Preference Optimization Dataset Generation
by: Arif, Samee, et al.
Published: (2024)

Preference Curriculum: LLMs Should Always Be Pretrained on Their Preferred Data
by: Zhang, Xuemiao, et al.
Published: (2025)

Context Pruning for Coding Agents via Multi-Rubric Latent Reasoning
by: Wang, Jingjing, et al.
Published: (2026)

LP-Eval: Rubric and Dataset for Measuring the Quality of Legal Proposition Generation
by: Xu, Shanshan, et al.
Published: (2026)

SyntheT2C: Generating Synthetic Data for Fine-Tuning Large Language Models on the Text2Cypher Task
by: Zhong, Ziije, et al.
Published: (2024)

Retrieval-Augmented Fine-Tuning With Preference Optimization For Visual Program Generation
by: Kang, Deokhyung, et al.
Published: (2025)

CroCo: Cross-Lingual Contrastive Preference Tuning on Self-Generations
by: Zhang, Mike, et al.
Published: (2026)