Cover Image

KTO: Model Alignment as Prospect Theoretic Optimization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Ethayarajh, Kawin, Xu, Winnie, Muennighoff, Niklas, Jurafsky, Dan, Kiela, Douwe
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Humanline: Online Alignment as Perceptual Loss
by: Liu, Sijia, et al.
Published: (2025)

Anchored Preference Optimization and Contrastive Revisions: Addressing Underspecification in Alignment
by: D'Oosterlinck, Karel, et al.
Published: (2024)

Understanding Dataset Difficulty with $\mathcal{V}$-Usable Information
by: Ethayarajh, Kawin, et al.
Published: (2021)

Generative Representational Instruction Tuning
by: Muennighoff, Niklas, et al.
Published: (2024)

Anchor Points: Benchmarking Models with Much Fewer Examples
by: Vivek, Rajan, et al.
Published: (2023)

Reflective Context Learning: Studying the Optimization Primitives of Context Space
by: Vassilyev, Nikita, et al.
Published: (2026)

Step-KTO: Optimizing Mathematical Reasoning through Stepwise Binary Feedback
by: Lin, Yen-Ting, et al.
Published: (2025)

Mecha-nudges for Machines
by: Frey, Giulio, et al.
Published: (2026)

ExtractBench: A Benchmark and Evaluation Methodology for Complex Structured Extraction
by: Ferguson, Nick, et al.
Published: (2026)

Great Models Think Alike and this Undermines AI Oversight
by: Goel, Shashwat, et al.
Published: (2025)

OLMoE: Open Mixture-of-Experts Language Models
by: Muennighoff, Niklas, et al.
Published: (2024)

BlitzRank: Principled Zero-shot Ranking Agents with Tournament Graphs
by: Agrawal, Sheshansh, et al.
Published: (2026)

Cooking Up Creativity: Enhancing LLM Creativity through Structured Recombination
by: Mizrahi, Moran, et al.
Published: (2025)

ReFT: Representation Finetuning for Language Models
by: Wu, Zhengxuan, et al.
Published: (2024)

Data Checklist: On Unit-Testing Datasets with Usable Information
by: Zhang, Heidi C., et al.
Published: (2024)

Theoretical Analysis of Sparse Optimization with Reparameterization, Weight Decay, and Adaptive Learning Rate
by: Xu, Huangyu, et al.
Published: (2026)

Scaling Data-Constrained Language Models
by: Muennighoff, Niklas, et al.
Published: (2023)

Meta-Aligner: Bidirectional Preference-Policy Optimization for Multi-Objective LLMs Alignment
by: Xu, Wenzhe, et al.
Published: (2026)

Inference-Time Alignment of Diffusion Models with Direct Noise Optimization
by: Tang, Zhiwei, et al.
Published: (2024)

Dual Alignment Maximin Optimization for Offline Model-based RL
by: Zhou, Chi, et al.
Published: (2025)

Information-Theoretic Safe Bayesian Optimization
by: Bottero, Alessandro G., et al.
Published: (2024)

I am a Strange Dataset: Metalinguistic Tests for Language Models
by: Thrush, Tristan, et al.
Published: (2024)

HOLOGRAPH: Active Causal Discovery via Sheaf-Theoretic Alignment of Large Language Model Priors
by: Kim, Hyunjun
Published: (2025)

Ecosystem-level Analysis of Deployed Machine Learning Reveals Homogeneous Outcomes
by: Toups, Connor, et al.
Published: (2023)

A Unified Theoretical Analysis of Private and Robust Offline Alignment: from RLHF to DPO
by: Zhou, Xingyu, et al.
Published: (2025)

Robust LLM Alignment via Distributionally Robust Direct Preference Optimization
by: Xu, Zaiyan, et al.
Published: (2025)

An Optimization Algorithm for Multimodal Data Alignment
by: Zhang, Wei, et al.
Published: (2025)

A Multi-target Bayesian Transformer Framework for Predicting Cardiovascular Disease Biomarkers during Pandemics
by: Inekwe, Trusting, et al.
Published: (2025)

Lynx: An Open Source Hallucination Evaluation Model
by: Ravi, Selvan Sunitha, et al.
Published: (2024)

Doubly Robust Alignment for Large Language Models
by: Xu, Erhan, et al.
Published: (2025)

The Reward Model Selection Crisis in Personalized Alignment
by: Rezk, Fady, et al.
Published: (2025)

Theoretical Analysis of Meta Reinforcement Learning: Generalization Bounds and Convergence Guarantees
by: Wang, Cangqing, et al.
Published: (2024)

Trust-Region Noise Search for Black-Box Alignment of Diffusion and Flow Models
by: Schweiger, Niklas, et al.
Published: (2026)

Model Alignment Search
by: Grant, Satchel
Published: (2025)

Binary Classifier Optimization for Large Language Model Alignment
by: Jung, Seungjae, et al.
Published: (2024)

Accelerated Preference Optimization for Large Language Model Alignment
by: He, Jiafan, et al.
Published: (2024)

Self-Play Preference Optimization for Language Model Alignment
by: Wu, Yue, et al.
Published: (2024)

Stepwise Alignment for Constrained Language Model Policy Optimization
by: Wachi, Akifumi, et al.
Published: (2024)

Gradient-Adaptive Policy Optimization: Towards Multi-Objective Alignment of Large Language Models
by: Li, Chengao, et al.
Published: (2025)

A Theoretical Survey on Foundation Models
by: Fu, Shi, et al.
Published: (2024)