Cover Image

Geometric-Averaged Preference Optimization for Soft Preference Labels

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Furuta, Hiroki, Lee, Kuang-Huei, Gu, Shixiang Shane, Matsuo, Yutaka, Faust, Aleksandra, Zen, Heiga, Gur, Izzeddin
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Exposing Limitations of Language Model Agents in Sequential-Task Compositions on the Web
by: Furuta, Hiroki, et al.
Published: (2023)

Multimodal Web Navigation with Instruction-Finetuned Foundation Models
by: Furuta, Hiroki, et al.
Published: (2023)

A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis
by: Gur, Izzeddin, et al.
Published: (2023)

Improving Dynamic Object Interactions in Text-to-Video Generation with AI Feedback
by: Furuta, Hiroki, et al.
Published: (2024)

Rethinking Evaluation of Sparse Autoencoders through the Representation of Polysemous Words
by: Minegishi, Gouki, et al.
Published: (2025)

Beyond Induction Heads: In-Context Meta Learning Induces Multi-Phase Circuit Emergence
by: Minegishi, Gouki, et al.
Published: (2025)

A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts
by: Lee, Kuang-Huei, et al.
Published: (2024)

Collective Intelligence for 2D Push Manipulations with Mobile Robots
by: Kuroki, So, et al.
Published: (2022)

Inference-Aware Fine-Tuning for Best-of-N Sampling in Large Language Models
by: Chow, Yinlam, et al.
Published: (2024)

FocalPO: Enhancing Preference Optimizing by Focusing on Correct Preference Rankings
by: Liu, Tong, et al.
Published: (2025)

Towards Empirical Interpretation of Internal Circuits and Properties in Grokked Transformers on Modular Polynomials
by: Furuta, Hiroki, et al.
Published: (2024)

Creative Preference Optimization
by: Ismayilzada, Mete, et al.
Published: (2025)

On the Role of Preference Variance in Preference Optimization
by: Guo, Jiacheng, et al.
Published: (2025)

Preference Packing: Efficient Preference Optimization for Large Language Models
by: Cho, Jaekyung
Published: (2026)

CAPO: Confidence Aware Preference Optimization Learning for Multilingual Preferences
by: Pokharel, Rhitabrat, et al.
Published: (2025)

ORPO: Monolithic Preference Optimization without Reference Model
by: Hong, Jiwoo, et al.
Published: (2024)

Capturing Nuanced Preferences: Preference-Aligned Distillation for Small Language Models
by: Gu, Yanggan, et al.
Published: (2025)

CLUTR: Curriculum Learning via Unsupervised Task Representation Learning
by: Azad, Abdus Salam, et al.
Published: (2022)

Weights-Rotated Preference Optimization for Large Language Models
by: Yang, Chenxu, et al.
Published: (2025)

Multiplayer Nash Preference Optimization
by: Wu, Fang, et al.
Published: (2025)

Iterative Reasoning Preference Optimization
by: Pang, Richard Yuanzhe, et al.
Published: (2024)

Accelerated Preference Optimization for Large Language Model Alignment
by: He, Jiafan, et al.
Published: (2024)

Topology of Reasoning: Understanding Large Reasoning Models through Reasoning Graph Properties
by: Minegishi, Gouki, et al.
Published: (2025)

Orthogonal Finetuning for Direct Preference Optimization
by: Yang, Chenxu, et al.
Published: (2024)

SGPO: Self-Generated Preference Optimization based on Self-Improver
by: Lee, Hyeonji, et al.
Published: (2025)

Self-supervised Preference Optimization: Enhance Your Language Model with Preference Degree Awareness
by: Li, Jian, et al.
Published: (2024)

Preference Ranking Optimization for Human Alignment
by: Song, Feifan, et al.
Published: (2023)

Token-level Direct Preference Optimization
by: Zeng, Yongcheng, et al.
Published: (2024)

Retrieval-Augmented Fine-Tuning With Preference Optimization For Visual Program Generation
by: Kang, Deokhyung, et al.
Published: (2025)

VPO: Leveraging the Number of Votes in Preference Optimization
by: Cho, Jae Hyeon, et al.
Published: (2024)

Self-Play Preference Optimization for Language Model Alignment
by: Wu, Yue, et al.
Published: (2024)

Not All Preferences are What You Need for Post-Training: Selective Alignment Strategy for Preference Optimization
by: Dong, Zhijin
Published: (2025)

Iterative Label Refinement Matters More than Preference Optimization under Weak Supervision
by: Ye, Yaowen, et al.
Published: (2025)

Stable Preference Optimization: A Bilevel Approach to Catastrophic Preference Shift
by: Jian, Chengtao, et al.
Published: (2025)

AMPO: Active Multi-Preference Optimization for Self-play Preference Selection
by: Gupta, Taneesh, et al.
Published: (2025)

Understanding Emergent Misalignment via Feature Superposition Geometry
by: Minegishi, Gouki, et al.
Published: (2026)

Filtered Direct Preference Optimization
by: Morimura, Tetsuro, et al.
Published: (2024)

Direct Preference Optimization with an Offset
by: Amini, Afra, et al.
Published: (2024)

Self-Consistency Preference Optimization
by: Prasad, Archiki, et al.
Published: (2024)

2D-DPO: Scaling Direct Preference Optimization with 2-Dimensional Supervision
by: Li, Shilong, et al.
Published: (2024)