Cover Image

Diversity as a Reward: Fine-Tuning LLMs on a Mixture of Domain-Undetermined Data

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Ling, Zhenqing, Chen, Daoyuan, Yao, Liuyi, Shen, Qianli, Li, Yaliang, Shen, Ying
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

On the Convergence of Zeroth-Order Federated Tuning for Large Language Models
by: Ling, Zhenqing, et al.
Published: (2024)

MindGYM: What Matters in Question Synthesis for Thinking-Centric Fine-Tuning?
by: Xu, Zhe, et al.
Published: (2025)

Federated Fine-tuning of Large Language Models under Heterogeneous Tasks and Client Resources
by: Bai, Jiamu, et al.
Published: (2024)

BiMix: A Bivariate Data Mixing Law for Language Model Pretraining
by: Ge, Ce, et al.
Published: (2024)

Double-I Watermark: Protecting Model Copyright for LLM Fine-tuning
by: Li, Shen, et al.
Published: (2024)

BOTS: A Unified Framework for Bayesian Online Task Selection in LLM Reinforcement Finetuning
by: Shen, Qianli, et al.
Published: (2025)

Parameter-Efficient Fine-Tuning of LLMs with Mixture of Space Experts
by: Zhang, Buze, et al.
Published: (2026)

Children's English Reading Story Generation via Supervised Fine-Tuning of Compact LLMs with Controllable Difficulty and Safety
by: Shen, Qian, et al.
Published: (2026)

Parameter-Efficient Routed Fine-Tuning: Mixture-of-Experts Demands Mixture of Adaptation Modules
by: Liu, Yilun, et al.
Published: (2025)

Calibration Collapse Under Sycophancy Fine-Tuning: How Reward Hacking Breaks Uncertainty Quantification in LLMs
by: Sahoo, Subramanyam
Published: (2026)

EE-Tuning: An Economical yet Scalable Solution for Tuning Early-Exit Large Language Models
by: Pan, Xuchen, et al.
Published: (2024)

Fine-Tuning Language Models with Reward Learning on Policy
by: Lang, Hao, et al.
Published: (2024)

The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspective
by: Qin, Zhen, et al.
Published: (2024)

Reward-Robust RLHF in LLMs
by: Yan, Yuzi, et al.
Published: (2024)

Fine-Tuning LLMs for Report Summarization: Analysis on Supervised and Unsupervised Data
by: Rallapalli, Swati, et al.
Published: (2025)

Deconfounded Causality-aware Parameter-Efficient Fine-Tuning for Problem-Solving Improvement of LLMs
by: Wang, Ruoyu, et al.
Published: (2024)

Can LLMs Speak For Diverse People? Tuning LLMs via Debate to Generate Controllable Controversial Statements
by: Li, Ming, et al.
Published: (2024)

ClusterUCB: Efficient Gradient-Based Data Selection for Targeted Fine-Tuning of LLMs
by: Wang, Zige, et al.
Published: (2025)

Get more for less: Principled Data Selection for Warming Up Fine-Tuning in LLMs
by: Kang, Feiyang, et al.
Published: (2024)

NeuroAda: Activating Each Neuron's Potential for Parameter-Efficient Fine-Tuning
by: Zhang, Zhi, et al.
Published: (2025)

DavIR: Data Selection via Implicit Reward for Large Language Models
by: Zhou, Haotian, et al.
Published: (2023)

Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs
by: Ovadia, Oded, et al.
Published: (2023)

Teaching LLMs How to Learn with Contextual Fine-Tuning
by: Choi, Younwoo, et al.
Published: (2025)

It Takes Two: On the Seamlessness between Reward and Policy Model in RLHF
by: Lu, Taiming, et al.
Published: (2024)

Supervised Fine-Tuning Needs to Unlock the Potential of Token Priority
by: Shen, Zhanming, et al.
Published: (2026)

SMART: Submodular Data Mixture Strategy for Instruction Tuning
by: Renduchintala, H S V N S Kowndinya, et al.
Published: (2024)

SEAL: Safety-enhanced Aligned LLM Fine-tuning via Bilevel Data Selection
by: Shen, Han, et al.
Published: (2024)

EBFT: Effective and Block-Wise Fine-Tuning for Sparse LLMs
by: Guo, Song, et al.
Published: (2024)

An Empirical Study on Preference Tuning Generalization and Diversity Under Domain Shift
by: Karouzos, Constantinos, et al.
Published: (2026)

NeuronTune: Fine-Grained Neuron Modulation for Balanced Safety-Utility Alignment in LLMs
by: Pan, Birong, et al.
Published: (2025)

Tag-LLM: Repurposing General-Purpose LLMs for Specialized Domains
by: Shen, Junhong, et al.
Published: (2024)

Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis
by: Wang, Xu, et al.
Published: (2025)

Efficient Differentially Private Fine-Tuning of LLMs via Reinforcement Learning
by: Khadangi, Afshin, et al.
Published: (2025)

Hallucination Detection in LLMs: Fast and Memory-Efficient Fine-Tuned Models
by: Arteaga, Gabriel Y., et al.
Published: (2024)

Position-Aware Parameter Efficient Fine-Tuning Approach for Reducing Positional Bias in LLMs
by: Zhang, Zheng, et al.
Published: (2024)

LLMSurgeon: Diagnosing Data Mixture of Large Language Models
by: Luo, Yaxin, et al.
Published: (2026)

Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards
by: Wang, Haoxiang, et al.
Published: (2024)

Principled Fine-tuning of LLMs from User-Edits: A Medley of Preference, Supervision, and Reward
by: Misra, Dipendra, et al.
Published: (2026)

LLMem: Estimating GPU Memory Usage for Fine-Tuning Pre-Trained LLMs
by: Kim, Taeho, et al.
Published: (2024)

ALKAFI-LLAMA3: Fine-Tuning LLMs for Precise Legal Understanding in Palestine
by: Qasem, Rabee, et al.
Published: (2024)