Cover Image

Iterative Reasoning Preference Optimization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Pang, Richard Yuanzhe, Yuan, Weizhe, Cho, Kyunghyun, He, He, Sukhbaatar, Sainbayar, Weston, Jason
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Self-Rewarding Language Models
by: Yuan, Weizhe, et al.
Published: (2024)

Some things are more CRINGE than others: Iterative Preference Optimization with the Pairwise Cringe Loss
by: Xu, Jing, et al.
Published: (2023)

Self-Consistency Preference Optimization
by: Prasad, Archiki, et al.
Published: (2024)

System-Level Natural Language Feedback
by: Yuan, Weizhe, et al.
Published: (2023)

StepWiser: Stepwise Generative Judges for Wiser Reasoning
by: Xiong, Wei, et al.
Published: (2025)

Thinking LLMs: General Instruction Following with Thought Generation
by: Wu, Tianhao, et al.
Published: (2024)

Contextual Position Encoding: Learning to Count What's Important
by: Golovneva, Olga, et al.
Published: (2024)

Reverse Training to Nurse the Reversal Curse
by: Golovneva, Olga, et al.
Published: (2024)

R.I.P.: Better Models by Survival of the Fittest Prompts
by: Yu, Ping, et al.
Published: (2025)

Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judge
by: Wu, Tianhao, et al.
Published: (2024)

Self-Challenging Language Model Agents
by: Zhou, Yifei, et al.
Published: (2025)

Following Length Constraints in Instructions
by: Yuan, Weizhe, et al.
Published: (2024)

CoT-Self-Instruct: Building high-quality synthetic prompts for reasoning and non-reasoning tasks
by: Yu, Ping, et al.
Published: (2025)

Leveraging Implicit Feedback from Deployment Data in Dialogue
by: Pang, Richard Yuanzhe, et al.
Published: (2023)

Diverse Preference Optimization
by: Lanchantin, Jack, et al.
Published: (2025)

Adaptive Decoding via Latent Preference Optimization
by: Dhuliawala, Shehzaad, et al.
Published: (2024)

Multi-Token Attention
by: Golovneva, Olga, et al.
Published: (2025)

Self-Taught Evaluators
by: Wang, Tianlu, et al.
Published: (2024)

Reasoning over mathematical objects: on-policy reward modeling and test time aggregation
by: Aggarwal, Pranjal, et al.
Published: (2026)

Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLM
by: Sukhbaatar, Sainbayar, et al.
Published: (2024)

SPICE: Self-Play In Corpus Environments Improves Reasoning
by: Liu, Bo, et al.
Published: (2025)

An Overview of Large Language Models for Statisticians
by: Ji, Wenlong, et al.
Published: (2025)

Self-Improving Pretraining: using post-trained models to pretrain better models
by: Tan, Ellen Xiaoqing, et al.
Published: (2026)

Preference Packing: Efficient Preference Optimization for Large Language Models
by: Cho, Jaekyung
Published: (2026)

Training Large Language Models to Reason in a Continuous Latent Space
by: Hao, Shibo, et al.
Published: (2024)

Preference Learning Algorithms Do Not Learn Preference Rankings
by: Chen, Angelica, et al.
Published: (2024)

Accelerated Preference Optimization for Large Language Model Alignment
by: He, Jiafan, et al.
Published: (2024)

Learning to Plan & Reason for Evaluation with Thinking-LLM-as-a-Judge
by: Saha, Swarnadeep, et al.
Published: (2025)

Efficient semantic uncertainty quantification in language models via diversity-steered sampling
by: Park, Ji Won, et al.
Published: (2025)

Iterative Deepening Sampling as Efficient Test-Time Scaling
by: Chen, Weizhe, et al.
Published: (2025)

Code-Switching In-Context Learning for Cross-Lingual Transfer of Large Language Models
by: Yoo, Haneul, et al.
Published: (2025)

Step-level Value Preference Optimization for Mathematical Reasoning
by: Chen, Guoxin, et al.
Published: (2024)

Retrieval-Augmented Fine-Tuning With Preference Optimization For Visual Program Generation
by: Kang, Deokhyung, et al.
Published: (2025)

Spontaneous Reward Hacking in Iterative Self-Refinement
by: Pan, Jane, et al.
Published: (2024)

Dualformer: Controllable Fast and Slow Thinking by Learning with Randomized Reasoning Traces
by: Su, DiJia, et al.
Published: (2024)

Enhancing LLM Reasoning via Non-Human-Like Reasoning Path Preference Optimization
by: Lu, Junjie, et al.
Published: (2025)

Transformers Struggle to Learn to Search
by: Saparov, Abulhair, et al.
Published: (2024)

Aioli: A Unified Optimization Framework for Language Model Data Mixing
by: Chen, Mayee F., et al.
Published: (2024)

Distilling System 2 into System 1
by: Yu, Ping, et al.
Published: (2024)

Beyond the Binary: Capturing Diverse Preferences With Reward Regularization
by: Padmakumar, Vishakh, et al.
Published: (2024)