Cover Image

An Empirical Study on Preference Tuning Generalization and Diversity Under Domain Shift

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Karouzos, Constantinos, Tan, Xingwei, Aletras, Nikolaos
Format:	Preprint
Published:	2026
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Where does output diversity collapse in post-training?
by: Karouzos, Constantinos, et al.
Published: (2026)

Incorporating Attribution Importance for Improving Faithfulness Metrics
by: Zhao, Zhixue, et al.
Published: (2023)

Analysing Chain of Thought Dynamics: Active Guidance or Unfaithful Post-hoc Rationalisation?
by: Lewis-Lim, Samuel, et al.
Published: (2025)

An Empirical Study on Cross-lingual Vocabulary Adaptation for Efficient Language Model Inference
by: Yamaguchi, Atsuki, et al.
Published: (2024)

Fine-Tuning on Noisy Instructions: Effects on Generalization and Performance
by: Alajrami, Ahmed, et al.
Published: (2025)

An Empirical Study on Context Length for Open-Domain Dialog Generation
by: Shen, Xinyi, et al.
Published: (2024)

Parameter Efficient Instruction Tuning: An Empirical Study
by: He, Pengfei
Published: (2024)

Stay Tuned: An Empirical Study of the Impact of Hyperparameters on LLM Tuning in Real-World Applications
by: Halfon, Alon, et al.
Published: (2024)

Stable Preference Optimization: A Bilevel Approach to Catastrophic Preference Shift
by: Jian, Chengtao, et al.
Published: (2025)

Diversity as a Reward: Fine-Tuning LLMs on a Mixture of Domain-Undetermined Data
by: Ling, Zhenqing, et al.
Published: (2025)

Reasoning Dynamics and the Limits of Monitoring Modality Reliance in Vision-Language Models
by: Villegas, Danae Sánchez, et al.
Published: (2026)

Compliance versus Sensibility: On the Reasoning Controllability in Large Language Models
by: Tan, Xingwei, et al.
Published: (2026)

Efficient Prompt Tuning by Multi-Space Projection and Prompt Fusion
by: Lan, Pengxiang, et al.
Published: (2024)

Picky LLMs and Unreliable RMs: An Empirical Study on Safety Alignment after Instruction Tuning
by: Li, Guanlin, et al.
Published: (2025)

Comparing Explanation Faithfulness between Multilingual and Monolingual Fine-tuned Language Models
by: Zhao, Zhixue, et al.
Published: (2024)

Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards
by: Wang, Haoxiang, et al.
Published: (2024)

Preference Tuning For Toxicity Mitigation Generalizes Across Languages
by: Li, Xiaochen, et al.
Published: (2024)

Accidental Vulnerability: Factors in Fine-Tuning that Shift Model Safeguards
by: Pandey, Punya Syon, et al.
Published: (2025)

HelpSteer3-Preference: Open Human-Annotated Preference Data across Diverse Tasks and Languages
by: Wang, Zhilin, et al.
Published: (2025)

Text Classification Under Class Distribution Shift: A Survey
by: Costache, Adriana Valentina, et al.
Published: (2025)

Domain-Adaptive Pre-Training for Arabic Aspect-Based Sentiment Analysis: A Comparative Study of Domain Adaptation and Fine-Tuning Strategies
by: Alyami, Salha, et al.
Published: (2025)

Understanding Multimodal LLMs Under Distribution Shifts: An Information-Theoretic Approach
by: Oh, Changdae, et al.
Published: (2025)

Can LLMs Speak For Diverse People? Tuning LLMs via Debate to Generate Controllable Controversial Statements
by: Li, Ming, et al.
Published: (2024)

Aggregated Knowledge Model: Enhancing Domain-Specific QA with Fine-Tuned and Retrieval-Augmented Generation Models
by: Liu, Fengchen, et al.
Published: (2024)

MaxMin-RLHF: Alignment with Diverse Human Preferences
by: Chakraborty, Souradip, et al.
Published: (2024)

Empirical Characterization of Rationale Stability Under Controlled Perturbations for Explainable Pattern Recognition
by: Sakib, Abu Noman Md, et al.
Published: (2026)

DIVERS-Bench: Evaluating Language Identification Across Domain Shifts and Code-Switching
by: Ojo, Jessica, et al.
Published: (2025)

Sparse but Critical: A Token-Level Analysis of Distributional Shifts in RLVR Fine-Tuning of LLMs
by: Meng, Haoming, et al.
Published: (2026)

From Language Modeling to Instruction Following: Understanding the Behavior Shift in LLMs after Instruction Tuning
by: Wu, Xuansheng, et al.
Published: (2023)

MetaMetrics-MT: Tuning Meta-Metrics for Machine Translation via Human Preference Calibration
by: Anugraha, David, et al.
Published: (2024)

Diversity-driven Data Selection for Language Model Tuning through Sparse Autoencoder
by: Yang, Xianjun, et al.
Published: (2025)

Diversity Measures: Domain-Independent Proxies for Failure in Language Model Queries
by: Ngu, Noel, et al.
Published: (2023)

Domain-Shift-Aware Conformal Prediction for Large Language Models
by: Lin, Zhexiao, et al.
Published: (2025)

Relative Preference Optimization: Enhancing LLM Alignment through Contrasting Responses across Identical and Diverse Prompts
by: Yin, Yueqin, et al.
Published: (2024)

Generative Representational Instruction Tuning
by: Muennighoff, Niklas, et al.
Published: (2024)

Bridging the Gap: From Ad-hoc to Proactive Search in Conversations
by: Meng, Chuan, et al.
Published: (2025)

How Can We Effectively Expand the Vocabulary of LLMs with 0.01GB of Target Language Text?
by: Yamaguchi, Atsuki, et al.
Published: (2024)

Low-Rank Adaptation for Multilingual Summarization: An Empirical Study
by: Whitehouse, Chenxi, et al.
Published: (2023)

On the Effect of Instruction Tuning Loss on Generalization
by: Chatterjee, Anwoy, et al.
Published: (2025)

On the Role of Preference Variance in Preference Optimization
by: Guo, Jiacheng, et al.
Published: (2025)