Imagen de Portada

Data Selection for LLM Alignment Using Fine-Grained Preferences

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Zhang, Jia, Liu, Yao, Zhang, Chen-Xi, Liu, Yi, Jin, Yi-Xuan, Guo, Lan-Zhe, Li, Yu-Feng
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

D3: Diversity, Difficulty, and Dependability-Aware Data Selection for Sample-Efficient LLM Instruction Tuning
por: Zhang, Jia, et al.
Publicado: (2025)

Contrast-Aware Calibration for Fine-Tuned CLIP: Leveraging Image-Text Alignment
por: Lv, Song-Lin, et al.
Publicado: (2025)

LearnAlign: Data Selection for LLM Reinforcement Learning with Improved Gradient Alignment
por: Li, Shipeng, et al.
Publicado: (2025)

Less is More: Improving LLM Alignment via Preference Data Selection
por: Deng, Xun, et al.
Publicado: (2025)

Data-Centric Human Preference with Rationales for Direct Preference Alignment
por: Just, Hoang Anh, et al.
Publicado: (2024)

ProDS: Preference-oriented Data Selection for Instruction Tuning
por: Guo, Wenya, et al.
Publicado: (2025)

Fine-Grained GRPO for Precise Preference Alignment in Flow Models
por: Zhou, Yujie, et al.
Publicado: (2025)

Reward-Augmented Data Enhances Direct Preference Alignment of LLMs
por: Zhang, Shenao, et al.
Publicado: (2024)

PRISM: Preference-Aware Influence Function Based Data Selection Method for Efficient Fine-Tuning
por: Lin, Qihao, et al.
Publicado: (2026)

Vision-Language Model Selection and Reuse for Downstream Adaptation
por: Tan, Hao-Zhe, et al.
Publicado: (2025)

Adversarial Preference Learning for Robust LLM Alignment
por: Wang, Yuanfu, et al.
Publicado: (2025)

Stackelberg Self-Annotation: A Robust Approach to Data-Efficient LLM Alignment
por: Chu, Xu, et al.
Publicado: (2025)

Realistic Evaluation of TabPFN v2 in Open Environments
por: Cheng, Zi-Jian, et al.
Publicado: (2025)

TabFSBench: Tabular Benchmark for Feature Shifts in Open Environments
por: Cheng, Zi-Jian, et al.
Publicado: (2025)

One Algorithm, Two Goals: Dual Scoring for Parameter and Data Selection in LLM Fine-Tuning
por: Chen, Xinrui, et al.
Publicado: (2026)

PAL: Pluralistic Alignment Framework for Learning from Heterogeneous Preferences
por: Chen, Daiwei, et al.
Publicado: (2024)

LLM-VA: Resolving the Jailbreak-Overrefusal Trade-off via Vector Alignment
por: Zhang, Haonan, et al.
Publicado: (2026)

Nearly Optimal Active Preference Learning and Its Application to LLM Alignment
por: Zhao, Yao, et al.
Publicado: (2026)

Sentence-level Reward Model can Generalize Better for Aligning LLM from Human Preference
por: Qiu, Wenjie, et al.
Publicado: (2025)

Difficulty-Based Preference Data Selection by DPO Implicit Reward Gap
por: Qi, Xuan, et al.
Publicado: (2025)

Directly Aligning the Full Diffusion Trajectory with Fine-Grained Human Preference
por: Shen, Xiangwei, et al.
Publicado: (2025)

An Optimization Algorithm for Multimodal Data Alignment
por: Zhang, Wei, et al.
Publicado: (2025)

Data Difficulty and the Generalization--Extrapolation Tradeoff in LLM Fine-Tuning
por: Liu, Siyuan, et al.
Publicado: (2026)

Towards Understanding Valuable Preference Data for Large Language Model Alignment
por: Zhang, Zizhuo, et al.
Publicado: (2025)

Greedy Information Projection for LLM Data Selection
por: Dong, Victor Ye, et al.
Publicado: (2026)

MallowsPO: Fine-Tune Your LLM with Preference Dispersions
por: Chen, Haoxian, et al.
Publicado: (2024)

ProteinOPD: Towards Effective and Efficient Preference Alignment for Protein Design
por: Zhang, Yulin, et al.
Publicado: (2026)

Fully Test-time Adaptation for Tabular Data
por: Zhou, Zhi, et al.
Publicado: (2024)

QuietPaw: Learning Quadrupedal Locomotion with Versatile Noise Preference Alignment
por: Zhang, Yuyou, et al.
Publicado: (2025)

TeamTR: Trust-Region Fine-Tuning for Multi-Agent LLM Coordination
por: Xie, Yi, et al.
Publicado: (2026)

Weakly Supervised Anomaly Detection via Knowledge-Data Alignment
por: Zhao, Haihong, et al.
Publicado: (2024)

Q-Adapter: Customizing Pre-trained LLMs to New Preferences with Forgetting Mitigation
por: Li, Yi-Chen, et al.
Publicado: (2024)

GRAPE: Generalizing Robot Policy via Preference Alignment
por: Zhang, Zijian, et al.
Publicado: (2024)

TRACE: Distilling Where It Matters via Token-Routed Self On-Policy Alignment
por: Wang, Jiaxuan, et al.
Publicado: (2026)

BoostLLM: Boosting-inspired LLM Fine-tuning for Few-shot Tabular Classification
por: Wang, Yi-Siang, et al.
Publicado: (2026)

A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning
por: Zhou, Zhi, et al.
Publicado: (2025)

Selection of LLM Fine-Tuning Data based on Orthogonal Rules
por: Li, Xiaomin, et al.
Publicado: (2024)

Double-I Watermark: Protecting Model Copyright for LLM Fine-tuning
por: Li, Shen, et al.
Publicado: (2024)

Bridging the Gap Between Preference Alignment and Machine Unlearning
por: Feng, Xiaohua, et al.
Publicado: (2025)

Efficient Preference-Based Reinforcement Learning Using Learned Dynamics Models
por: Liu, Yi, et al.
Publicado: (2023)