Copertina

Clean First, Align Later: Benchmarking Preference Data Cleaning for Reliable LLM Alignment

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Yeh, Samuel, Li, Sharon
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Artificial Intelligence Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Token Cleaning: Fine-Grained Data Selection for LLM Supervised Fine-Tuning
di: Pang, Jinlong, et al.
Pubblicazione: (2025)

EchoDistill:Alignment Noisy-to-Clean Self-Distillation for Robust Audio LLMs
di: Lin, Liang, et al.
Pubblicazione: (2026)

Auditing Stealth Sycophancy in Mental-Health Dialogue: Structured Clinical-State Diagnostics and Clean Matched Benchmarks
di: Han, Tianze, et al.
Pubblicazione: (2026)

CleanPatrick: A Benchmark for Image Data Cleaning
di: Gröger, Fabian, et al.
Pubblicazione: (2025)

AlpsBench: An LLM Personalization Benchmark for Real-Dialogue Memorization and Preference Alignment
di: Xiao, Jianfei, et al.
Pubblicazione: (2026)

The KL3M Data Project: Copyright-Clean Training Resources for Large Language Models
di: Bommarito II, Michael J, et al.
Pubblicazione: (2025)

CleanGraph: Human-in-the-loop Knowledge Graph Refinement and Completion
di: Bikaun, Tyler, et al.
Pubblicazione: (2024)

ConceptRM: The Quest to Mitigate Alert Fatigue through Consensus-Based Purity-Driven Data Cleaning for Reflection Modelling
di: Yu, Yongda, et al.
Pubblicazione: (2026)

Enhancing Grammatical Error Detection using BERT with Cleaned Lang-8 Dataset
di: Nihalani, Rahul, et al.
Pubblicazione: (2024)

LLM-Powered Benchmark Factory: Reliable, Generic, and Efficient
di: Yuan, Peiwen, et al.
Pubblicazione: (2025)

MathClean: A Benchmark for Synthetic Mathematical Data Cleaning
di: Liang, Hao, et al.
Pubblicazione: (2025)

Capturing Nuanced Preferences: Preference-Aligned Distillation for Small Language Models
di: Gu, Yanggan, et al.
Pubblicazione: (2025)

Safer-Instruct: Aligning Language Models with Automated Preference Data
di: Shi, Taiwei, et al.
Pubblicazione: (2023)

Less is More: Improving LLM Alignment via Preference Data Selection
di: Deng, Xun, et al.
Pubblicazione: (2025)

TriAlign: Towards Universal Truth Consistency in Personalized LLM Alignment
di: Nguyen, Thi-Nhung, et al.
Pubblicazione: (2026)

The Multilingual Alignment Prism: Aligning Global and Local Preferences to Reduce Harm
di: Aakanksha, et al.
Pubblicazione: (2024)

Can LLMs Clean Up Your Mess? A Survey of Application-Ready Data Preparation with LLMs
di: Zhou, Wei, et al.
Pubblicazione: (2026)

PMMT: Preference Alignment in Multilingual Machine Translation via LLM Distillation
di: Sun, Shuqiao, et al.
Pubblicazione: (2024)

Pref-CTRL: Preference Driven LLM Alignment using Representation Editing
di: Ashrafi, Imranul, et al.
Pubblicazione: (2026)

A Highly Clean Recipe Dataset with Ingredient States Annotation for State Probing Task
di: Toyooka, Mashiro, et al.
Pubblicazione: (2025)

LLM-Align: Utilizing Large Language Models for Entity Alignment in Knowledge Graphs
di: Chen, Xuan, et al.
Pubblicazione: (2024)

Preference Ranking Optimization for Human Alignment
di: Song, Feifan, et al.
Pubblicazione: (2023)

TODO: Enhancing LLM Alignment with Ternary Preferences
di: Guo, Yuxiang, et al.
Pubblicazione: (2024)

Spread Preference Annotation: Direct Preference Judgment for Efficient LLM Alignment
di: Kim, Dongyoung, et al.
Pubblicazione: (2024)

LifeAlign: Lifelong Alignment for Large Language Models with Memory-Augmented Focalized Preference Optimization
di: Li, Junsong, et al.
Pubblicazione: (2025)

Disentangling Preference Representation and Text Generation for Efficient Individual Preference Alignment
di: Zhang, Jianfei, et al.
Pubblicazione: (2024)

From Meta-Thought to Execution: Cognitively Aligned Post-Training for Generalizable and Reliable LLM Reasoning
di: Wang, Shaojie, et al.
Pubblicazione: (2026)

AlignBench: Benchmarking Chinese Alignment of Large Language Models
di: Liu, Xiao, et al.
Pubblicazione: (2023)

Is On-Policy Data always the Best Choice for Direct Preference Optimization-based LM Alignment?
di: Sun, Zetian, et al.
Pubblicazione: (2025)

Aligning Large Language Models with Searcher Preferences
di: Wu, Wei, et al.
Pubblicazione: (2026)

Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing
di: Xu, Zhangchen, et al.
Pubblicazione: (2024)

Acquiring Clean Language Models from Backdoor Poisoned Datasets by Downscaling Frequency Space
di: Wu, Zongru, et al.
Pubblicazione: (2024)

Adversarial Preference Optimization: Enhancing Your Alignment via RM-LLM Game
di: Cheng, Pengyu, et al.
Pubblicazione: (2023)

CURATRON: Complete and Robust Preference Data for Rigorous Alignment of Large Language Models
di: Nguyen, Son The, et al.
Pubblicazione: (2024)

ALIGN: Prompt-based Attribute Alignment for Reliable, Responsible, and Personalized LLM-based Decision-Making
di: Ravichandran, Bharadwaj, et al.
Pubblicazione: (2025)

Word Alignment as Preference for Machine Translation
di: Wu, Qiyu, et al.
Pubblicazione: (2024)

Dissecting Human and LLM Preferences
di: Li, Junlong, et al.
Pubblicazione: (2024)

Fake Alignment: Are LLMs Really Aligned Well?
di: Wang, Yixu, et al.
Pubblicazione: (2023)

ASGEA: Exploiting Logic Rules from Align-Subgraphs for Entity Alignment
di: Luo, Yangyifei, et al.
Pubblicazione: (2024)

More RLHF, More Trust? On The Impact of Preference Alignment On Trustworthiness
di: Li, Aaron J., et al.
Pubblicazione: (2024)