Cover Image

Test-Time Preference Optimization: On-the-Fly Alignment via Iterative Textual Feedback

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Li, Yafu, Hu, Xuyang, Qu, Xiaoye, Li, Linjie, Cheng, Yu
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Reasoning over Boundaries: Enhancing Specification Alignment via Test-time Deliberation
by: Zhang, Haoran, et al.
Published: (2025)

SEE: Continual Fine-tuning with Sequential Ensemble of Experts
by: Wang, Zhilin, et al.
Published: (2025)

Characterizing, Evaluating, and Optimizing Complex Reasoning
by: Zhang, Haoran, et al.
Published: (2026)

Scaling Reasoning, Losing Control: Evaluating Instruction Following in Large Reasoning Models
by: Fu, Tingchen, et al.
Published: (2025)

FaithRL: Learning to Reason Faithfully through Step-Level Faithfulness Maximization
by: Gui, Runquan, et al.
Published: (2026)

LLaMA-MoE v2: Exploring Sparsity of LLaMA from Perspective of Mixture-of-Experts with Post-Training
by: Qu, Xiaoye, et al.
Published: (2024)

Textual Self-attention Network: Test-Time Preference Optimization through Textual Gradient-based Attention
by: Mo, Shibing, et al.
Published: (2025)

Test-Time Alignment for Large Language Models via Textual Model Predictive Control
by: Wang, Kuang-Da, et al.
Published: (2025)

Learning to Reason under Off-Policy Guidance
by: Yan, Jianhao, et al.
Published: (2025)

New Skills or Sharper Primitives? A Probabilistic Perspective on the Emergence of Reasoning in RLVR
by: Wang, Zhilin, et al.
Published: (2026)

LatentMem: Customizing Latent Memory for Multi-Agent Systems
by: Fu, Muxin, et al.
Published: (2026)

Make LoRA Great Again: Boosting LoRA with Adaptive Singular Values and Mixture-of-Experts Optimization Alignment
by: Fan, Chenghao, et al.
Published: (2025)

Advancing Multimodal Reasoning: From Optimized Cold Start to Staged Reinforcement Learning
by: Chen, Shuang, et al.
Published: (2025)

Preference Ranking Optimization for Human Alignment
by: Song, Feifan, et al.
Published: (2023)

ExGRPO: Learning to Reason from Experience
by: Zhan, Runzhe, et al.
Published: (2025)

Margin Matching Preference Optimization: Enhanced Model Alignment with Granular Feedback
by: Kim, Kyuyoung, et al.
Published: (2024)

Draft-OPD: On-Policy Distillation for Speculative Draft Models
by: Lei, Haodi, et al.
Published: (2026)

Adversarial Preference Optimization: Enhancing Your Alignment via RM-LLM Game
by: Cheng, Pengyu, et al.
Published: (2023)

PDDLEGO: Iterative Planning in Textual Environments
by: Zhang, Li, et al.
Published: (2024)

Thinking on the Fly: Test-Time Reasoning Enhancement via Latent Thought Policy Optimization
by: Ye, Wengao, et al.
Published: (2025)

Towards an AI Musician: Synthesizing Sheet Music Problems for Musical Reasoning
by: Wang, Zhilin, et al.
Published: (2025)

Multi-LLM Collaborative Search for Complex Problem Solving
by: Yang, Sen, et al.
Published: (2025)

Iterative Reasoning Preference Optimization
by: Pang, Richard Yuanzhe, et al.
Published: (2024)

FrameThinker: Learning to Think with Long Videos via Multi-Turn Frame Spotlighting
by: He, Zefeng, et al.
Published: (2025)

ProSocialAlign: Preference Conditioned Test Time Alignment in Language Models
by: Banerjee, Somnath, et al.
Published: (2025)

Preference Optimization for Reasoning with Pseudo Feedback
by: Jiao, Fangkai, et al.
Published: (2024)

Language-Guided Tuning: Enhancing Numeric Optimization with Textual Feedback
by: Lu, Yuxing, et al.
Published: (2025)

CiPO: Counterfactual Unlearning for Large Reasoning Models through Iterative Preference Optimization
by: Li, Junyi, et al.
Published: (2026)

ARTIS: Agentic Risk-Aware Test-Time Scaling via Iterative Simulation
by: Zeng, Xingshan, et al.
Published: (2026)

Proximalized Preference Optimization for Diverse Feedback Types: A Decomposed Perspective on DPO
by: Guo, Kaiyang, et al.
Published: (2025)

PA-RAG: RAG Alignment via Multi-Perspective Preference Optimization
by: Wu, Jiayi, et al.
Published: (2024)

From Drafts to Answers: Unlocking LLM Potential via Aggregation Fine-Tuning
by: Li, Yafu, et al.
Published: (2025)

AIPO: Improving Training Objective for Iterative Preference Optimization
by: Shen, Yaojie, et al.
Published: (2024)

AERO: Autonomous Evolutionary Reasoning Optimization via Endogenous Dual-Loop Feedback
by: Gao, Zhitao, et al.
Published: (2026)

Is On-Policy Data always the Best Choice for Direct Preference Optimization-based LM Alignment?
by: Sun, Zetian, et al.
Published: (2025)

Textual-to-Visual Iterative Self-Verification for Slide Generation
by: Xu, Yunqing, et al.
Published: (2025)

Reverse Preference Optimization for Complex Instruction Following
by: Huang, Xiang, et al.
Published: (2025)

ISSR: Iterative Selection with Self-Review for Vocabulary Test Distractor Generation
by: Liu, Yu-Cheng, et al.
Published: (2025)

RoVRM: A Robust Visual Reward Model Optimized via Auxiliary Textual Preference Data
by: Wang, Chenglong, et al.
Published: (2024)

BPO: Towards Balanced Preference Optimization between Knowledge Breadth and Depth in Alignment
by: Wang, Sizhe, et al.
Published: (2024)