Cover Image

Shallow Preference Signals: Large Language Model Aligns Even Better with Truncated Data?

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Qi, Xuan, Qiu, Jiahao, Juan, Xinzhe, Wu, Yue, Wang, Mengdi
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

On the Role of Preference Variance in Preference Optimization
by: Guo, Jiacheng, et al.
Published: (2025)

GenEnv: Difficulty-Aligned Co-Evolution Between LLM Agents and Environment Simulators
by: Guo, Jiacheng, et al.
Published: (2025)

Learning Agent Routing From Early Experience
by: Wang, Yimin, et al.
Published: (2026)

Temporal Consistency for LLM Reasoning Process Error Identification
by: Guo, Jiacheng, et al.
Published: (2025)

MetaAlign: Align Large Language Models with Diverse Preferences during Inference Time
by: Zhang, Mozhi, et al.
Published: (2024)

Aligning Large Language Models with Searcher Preferences
by: Wu, Wei, et al.
Published: (2026)

Inverse-Q*: Token Level Reinforcement Learning for Aligning Large Language Models Without Preference Data
by: Xia, Han, et al.
Published: (2024)

Aligning Large Language Models with Human Preferences through Representation Engineering
by: Liu, Wenhao, et al.
Published: (2023)

EmoAgent: Assessing and Safeguarding Human-AI Interaction for Mental Health Safety
by: Qiu, Jiahao, et al.
Published: (2025)

Checklists Are Better Than Reward Models For Aligning Language Models
by: Viswanathan, Vijay, et al.
Published: (2025)

Mixed Preference Optimization: Reinforcement Learning with Data Selection and Better Reference Model
by: Gou, Qi, et al.
Published: (2024)

Sentence-level Reward Model can Generalize Better for Aligning LLM from Human Preference
by: Qiu, Wenjie, et al.
Published: (2025)

Latent Preference Coding: Aligning Large Language Models via Discrete Latent Codes
by: Gong, Zhuocheng, et al.
Published: (2025)

Efficient Response Generation Strategy Selection for Fine-Tuning Large Language Models Through Self-Aligned Perplexity
by: Ren, Xuan, et al.
Published: (2025)

Aligning Large Language Models with Implicit Preferences from User-Generated Content
by: Tan, Zhaoxuan, et al.
Published: (2025)

ROPO: Robust Preference Optimization for Large Language Models
by: Liang, Xize, et al.
Published: (2024)

Brief Is Better: Non-Monotonic Chain-of-Thought Budget Effects in Function-Calling Language Agents
by: Qi, Xuan
Published: (2026)

The Real, the Better: Aligning Large Language Models with Online Human Behaviors
by: Jiang, Guanying, et al.
Published: (2024)

Aligning Large Language Model Behavior with Human Citation Preferences
by: Ando, Kenichiro, et al.
Published: (2026)

RuleAlign: Making Large Language Models Better Physicians with Diagnostic Rule Alignment
by: Wang, Xiaohan, et al.
Published: (2024)

Safer-Instruct: Aligning Language Models with Automated Preference Data
by: Shi, Taiwei, et al.
Published: (2023)

Icon$^{2}$: Aligning Large Language Models Using Self-Synthetic Preference Data via Inherent Regulation
by: Chen, Qiyuan, et al.
Published: (2025)

Preference-Oriented Supervised Fine-Tuning: Favoring Target Model Over Aligned Large Language Models
by: Fan, Yuchen, et al.
Published: (2024)

Making Large Language Models Better Reasoners with Orchestrated Streaming Experiences
by: Liu, Xiangyang, et al.
Published: (2025)

ADELIE: Aligning Large Language Models on Information Extraction
by: Qi, Yunjia, et al.
Published: (2024)

LLM-Align: Utilizing Large Language Models for Entity Alignment in Knowledge Graphs
by: Chen, Xuan, et al.
Published: (2024)

Aligning Language Models with Human Preferences via a Bayesian Approach
by: Wang, Jiashuo, et al.
Published: (2023)

TPO: Aligning Large Language Models with Multi-branch & Multi-step Preference Trees
by: Liao, Weibin, et al.
Published: (2024)

Can Large Language Models Understand You Better? An MBTI Personality Detection Dataset Aligned with Population Traits
by: Li, Bohan, et al.
Published: (2024)

Preference-Guided Reflective Sampling for Aligning Language Models
by: Ye, Hai, et al.
Published: (2024)

TreeBoN: Enhancing Inference-Time Alignment with Speculative Tree-Search and Best-of-N Sampling
by: Qiu, Jiahao, et al.
Published: (2024)

Steering Risk Preferences in Large Language Models by Aligning Behavioral and Neural Representations
by: Zhu, Jian-Qiao, et al.
Published: (2025)

AlignSum: Data Pyramid Hierarchical Fine-tuning for Aligning with Human Summarization Preference
by: Han, Yang, et al.
Published: (2024)

Ask Optimal Questions: Aligning Large Language Models with Retriever's Preference in Conversation
by: Yoon, Chanwoong, et al.
Published: (2024)

Discriminative Finetuning of Generative Large Language Models without Reward Models and Human Preference Data
by: Guo, Siqi, et al.
Published: (2025)

Quantifying Uncertainty in Natural Language Explanations of Large Language Models for Question Answering
by: Li, Yangyi, et al.
Published: (2025)

I Learn Better If You Speak My Language: Understanding the Superior Performance of Fine-Tuning Large Language Models with LLM-Generated Responses
by: Ren, Xuan, et al.
Published: (2024)

Capturing Nuanced Preferences: Preference-Aligned Distillation for Small Language Models
by: Gu, Yanggan, et al.
Published: (2025)

WenyanGPT: A Large Language Model for Classical Chinese Tasks
by: Yao, Xinyu, et al.
Published: (2025)

Aligning Modalities in Vision Large Language Models via Preference Fine-tuning
by: Zhou, Yiyang, et al.
Published: (2024)