Cover Image

Ratio-Variance Regularized Policy Optimization for Efficient LLM Fine-tuning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Luo, Yu, Han, Shuo, Hu, Yihan, Li, Dong, Hao, Jianye
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Adaptive Divergence Regularized Policy Optimization for Fine-tuning Generative Models
by: Fan, Jiajun, et al.
Published: (2025)

The Blessing of Dimensionality in LLM Fine-tuning: A Variance-Curvature Perspective
by: Liang, Qiyao, et al.
Published: (2026)

A Variance-Reduced Cubic-Regularized Newton for Policy Optimization
by: Sun, Cheng, et al.
Published: (2025)

Hierarchical Balance Packing: Towards Efficient Supervised Fine-tuning for Long-Context LLM
by: Yao, Yongqiang, et al.
Published: (2025)

Parameter Efficient Fine-tuning via Explained Variance Adaptation
by: Paischer, Fabian, et al.
Published: (2024)

Rethinking Safety in LLM Fine-tuning: An Optimization Perspective
by: Kim, Minseon, et al.
Published: (2025)

CurvZO: Adaptive Curvature-Guided Sparse Zeroth-Order Optimization for Efficient LLM Fine-Tuning
by: Wang, Shuo, et al.
Published: (2026)

Fine-tuning Pocket-Aware Diffusion Models via Denoising Policy Optimization
by: Xue, Yuan, et al.
Published: (2026)

Fine-tuning Diffusion Policies with Backpropagation Through Diffusion Timesteps
by: Yang, Ningyuan, et al.
Published: (2025)

RoRA: Efficient Fine-Tuning of LLM with Reliability Optimization for Rank Adaptation
by: Liu, Jun, et al.
Published: (2025)

S$^{2}$FT: Efficient, Scalable and Generalizable LLM Fine-tuning by Structured Sparsity
by: Yang, Xinyu, et al.
Published: (2024)

Stable Asynchrony: Variance-Controlled Off-Policy RL for LLMs
by: Huang, Luke J., et al.
Published: (2026)

Reference-guided Policy Optimization for Molecular Optimization via LLM Reasoning
by: Li, Xuan, et al.
Published: (2026)

EVPO: Explained Variance Policy Optimization for Adaptive Critic Utilization in LLM Post-Training
by: Pan, Chengjun, et al.
Published: (2026)

HELENE: Hessian Layer-wise Clipping and Gradient Annealing for Accelerating Fine-tuning LLM with Zeroth-order Optimization
by: Zhao, Huaqin, et al.
Published: (2024)

SC-LoRA: Balancing Efficient Fine-tuning and Knowledge Preservation via Subspace-Constrained LoRA
by: Luo, Minrui, et al.
Published: (2025)

Information Guided Regularization for Fine-tuning Language Models
by: Sharma, Mandar, et al.
Published: (2024)

LLM-Based Scientific Equation Discovery via Physics-Informed Token-Regularized Policy Optimization
by: Wang, Boxiao, et al.
Published: (2026)

State Regularized Policy Optimization on Data with Dynamics Shift
by: Xue, Zhenghai, et al.
Published: (2023)

Safeguarding LLM Fine-tuning via Push-Pull Distributional Alignment
by: Wang, Haozhong, et al.
Published: (2026)

SEAL: Safety-enhanced Aligned LLM Fine-tuning via Bilevel Data Selection
by: Shen, Han, et al.
Published: (2024)

Fine-tuning Behavioral Cloning Policies with Preference-Based Reinforcement Learning
by: Macuglia, Maël, et al.
Published: (2025)

PrunePEFT: Iterative Hybrid Pruning for Parameter-Efficient Fine-tuning of LLMs
by: Yu, Tongzhou, et al.
Published: (2025)

Outlier-weighed Layerwise Sampling for LLM Fine-tuning
by: Li, Pengxiang, et al.
Published: (2024)

Complexity-Regularized Proximal Policy Optimization
by: Serfilippi, Luca, et al.
Published: (2025)

Symmetric Behavior Regularized Policy Optimization
by: Zhu, Lingwei, et al.
Published: (2025)

AdaMeZO: Adam-style Zeroth-Order Optimizer for LLM Fine-tuning Without Maintaining the Moments
by: Cai, Zhijie, et al.
Published: (2026)

Model-Based Epistemic Variance of Values for Risk-Aware Policy Optimization
by: Luis, Carlos E., et al.
Published: (2023)

Behaviour Policy Optimization: Provably Lower Variance Return Estimates for Off-Policy Reinforcement Learning
by: Goodall, Alexander W., et al.
Published: (2025)

Behavior-Regularized Diffusion Policy Optimization for Offline Reinforcement Learning
by: Gao, Chen-Xiao, et al.
Published: (2025)

Revisiting Regularized Policy Optimization for Stable and Efficient Reinforcement Learning in Two-Player Games
by: Ota, Kazuki, et al.
Published: (2026)

Efficient End-to-end Language Model Fine-tuning on Graphs
by: Xue, Rui, et al.
Published: (2023)

MODULI: Unlocking Preference Generalization via Diffusion Models for Offline Multi-Objective Reinforcement Learning
by: Yuan, Yifu, et al.
Published: (2024)

Skrull: Towards Efficient Long Context Fine-tuning through Dynamic Data Scheduling
by: Xu, Hongtao, et al.
Published: (2025)

Demystifying Design Choices of Reinforcement Fine-tuning: A Batched Contextual Bandit Learning Perspective
by: Xie, Hong, et al.
Published: (2026)

Federated Offline Policy Optimization with Dual Regularization
by: Yue, Sheng, et al.
Published: (2024)

Confidence-aware Reward Optimization for Fine-tuning Text-to-Image Models
by: Kim, Kyuyoung, et al.
Published: (2024)

GVPO: Group Variance Policy Optimization for Large Language Model Post-Training
by: Zhang, Kaichen, et al.
Published: (2025)

Double-I Watermark: Protecting Model Copyright for LLM Fine-tuning
by: Li, Shen, et al.
Published: (2024)

AdaFlow: Imitation Learning with Variance-Adaptive Flow-Based Policies
by: Hu, Xixi, et al.
Published: (2024)