Cover Image

CiPO: Counterfactual Unlearning for Large Reasoning Models through Iterative Preference Optimization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Li, Junyi, Chen, Yongqiang, Ding, Ningning
Format:	Preprint
Published:	2026
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Iterative Reasoning Preference Optimization
by: Pang, Richard Yuanzhe, et al.
Published: (2024)

LongPO: Long Context Self-Evolution of Large Language Models through Short-to-Long Preference Optimization
by: Chen, Guanzheng, et al.
Published: (2025)

Alternate Preference Optimization for Unlearning Factual Knowledge in Large Language Models
by: Mekala, Anmol, et al.
Published: (2024)

SeaPO: Strategic Error Amplification for Robust Preference Optimization of Large Language Models
by: Rao, Jun, et al.
Published: (2025)

LiPO: Listwise Preference Optimization through Learning-to-Rank
by: Liu, Tianqi, et al.
Published: (2024)

DiffPO: Diffusion-styled Preference Optimization for Efficient Inference-Time Alignment of Large Language Models
by: Chen, Ruizhe, et al.
Published: (2025)

Enhancing Multilingual Counterfactual Generation through Alignment-as-Preference Optimization
by: Wang, Yilong, et al.
Published: (2026)

PrefPO: Pairwise Preference Prompt Optimization
by: Singhal, Rahul, et al.
Published: (2026)

R-TOFU: Unlearning in Large Reasoning Models
by: Yoon, Sangyeon, et al.
Published: (2025)

Reasoning Models Hallucinate More: Factuality-Aware Reinforcement Learning for Large Reasoning Models
by: Li, Junyi, et al.
Published: (2025)

AMaPO: Adaptive Margin-attached Preference Optimization for Language Model Alignment
by: Deng, Ruibo, et al.
Published: (2025)

Evaluating Counterfactual Strategic Reasoning in Large Language Models
by: Georgousis, Dimitrios, et al.
Published: (2026)

CounterBench: Evaluating and Improving Counterfactual Reasoning in Large Language Models
by: Chen, Yuefei, et al.
Published: (2025)

Reasoning Strategies in Large Language Models: Can They Follow, Prefer, and Optimize?
by: Zhang, Yanjian, et al.
Published: (2025)

FocalPO: Enhancing Preference Optimizing by Focusing on Correct Preference Rankings
by: Liu, Tong, et al.
Published: (2025)

AIPO: Improving Training Objective for Iterative Preference Optimization
by: Shen, Yaojie, et al.
Published: (2024)

MidPO: Dual Preference Optimization for Safety and Helpfulness in Large Language Models via a Mixture of Experts Framework
by: Qi, Yupeng, et al.
Published: (2025)

MAPO: Advancing Multilingual Reasoning through Multilingual Alignment-as-Preference Optimization
by: She, Shuaijie, et al.
Published: (2024)

M3PO: Multimodal-Model-Guided Preference Optimization for Visual Instruction Following
by: Gao, Ruirui, et al.
Published: (2025)

SimPO: Simple Preference Optimization with a Reference-Free Reward
by: Meng, Yu, et al.
Published: (2024)

Deciphering the Impact of Pretraining Data on Large Language Models through Machine Unlearning
by: Zhao, Yang, et al.
Published: (2024)

Learn while Unlearn: An Iterative Unlearning Framework for Generative Language Models
by: Tang, Haoyu, et al.
Published: (2024)

ComPO: Community Preferences for Language Model Personalization
by: Kumar, Sachin, et al.
Published: (2024)

A Neuro-inspired Interpretation of Unlearning in Large Language Models through Sample-level Unlearning Difficulty
by: Feng, Xiaohua, et al.
Published: (2025)

HiPO: Hybrid Policy Optimization for Dynamic Reasoning in LLMs
by: Deng, Ken, et al.
Published: (2025)

Preference Optimization for Reasoning with Pseudo Feedback
by: Jiao, Fangkai, et al.
Published: (2024)

Towards Safer Large Language Models through Machine Unlearning
by: Liu, Zheyuan, et al.
Published: (2024)

Self-Evolutionary Large Language Models through Uncertainty-Enhanced Preference Optimization
by: Wang, Jianing, et al.
Published: (2024)

CANDLE: Iterative Conceptualization and Instantiation Distillation from Large Language Models for Commonsense Reasoning
by: Wang, Weiqi, et al.
Published: (2024)

$f$-PO: Generalizing Preference Optimization with $f$-divergence Minimization
by: Han, Jiaqi, et al.
Published: (2024)

LamPO: A Lambda Style Policy Optimization for Reasoning Language Models
by: arXiv, Redacted by
Published: (2026)

LambdaPO: A Lambda Style Policy Optimization for Reasoning Language Models
by: arXiv, Redacted by
Published: (2026)

Test-Time Preference Optimization: On-the-Fly Alignment via Iterative Textual Feedback
by: Li, Yafu, et al.
Published: (2025)

Towards Reasoning-Preserving Unlearning in Multimodal Large Language Models
by: Li, Hongji, et al.
Published: (2025)

Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization
by: Wang, Weiyun, et al.
Published: (2024)

BFS-PO: Best-First Search for Large Reasoning Models
by: Parascandolo, Fiorenzo, et al.
Published: (2026)

RFEval: Benchmarking Reasoning Faithfulness under Counterfactual Reasoning Intervention in Large Reasoning Models
by: Han, Yunseok, et al.
Published: (2026)

Avoiding Copyright Infringement via Large Language Model Unlearning
by: Dou, Guangyao, et al.
Published: (2024)

EmPO: Emotion Grounding for Empathetic Response Generation through Preference Optimization
by: Sotolar, Ondrej, et al.
Published: (2024)

Integrating Physician Diagnostic Logic into Large Language Models: Preference Learning from Process Feedback
by: Dou, Chengfeng, et al.
Published: (2024)