Cover Image

ESPO: Entropy Importance Sampling Policy Optimization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Sheng, Yuepeng, Huang, Yuwei, Liu, Shuman, Zeng, Anxiang, Zhang, Haibo
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

ESPO: Early-Stopping Proximal Policy Optimization
by: Li, Zihang, et al.
Published: (2026)

Each Prompt Matters: Scaling Reinforcement Learning Without Wasting Rollouts on Hundred-Billion-Scale MoE
by: Zeng, Anxiang, et al.
Published: (2025)

Compass-Thinker-7B Technical Report
by: Zeng, Anxiang, et al.
Published: (2025)

GIPO: Gaussian Importance Sampling Policy Optimization
by: Lu, Chengxuan, et al.
Published: (2026)

Optimal Transport-Based Token Weighting scheme for Enhanced Preference Optimization
by: Li, Meng, et al.
Published: (2025)

Beyond Importance Sampling: Rejection-Gated Policy Optimization
by: Sun, Ziwu, et al.
Published: (2026)

AIS: Adaptive Importance Sampling for Quantized RL
by: Zhou, Jiajun, et al.
Published: (2026)

Rethinking Importance Sampling in LLM Policy Optimization: A Cumulative Token Perspective
by: Zhang, Yuheng, et al.
Published: (2026)

Preference as Reward, Maximum Preference Optimization with Importance Sampling
by: Jiang, Zaifan, et al.
Published: (2023)

SPEC-RL: Accelerating On-Policy Reinforcement Learning with Speculative Rollouts
by: Liu, Bingshuai, et al.
Published: (2025)

Measuring Sample Importance in Data Pruning for Language Models based on Information Entropy
by: Kim, Minsang, et al.
Published: (2024)

When Maximum Entropy Misleads Policy Optimization
by: Zhang, Ruipeng, et al.
Published: (2025)

MISA: Memory-Efficient LLMs Optimization with Module-wise Importance Sampling
by: Liu, Yuxi, et al.
Published: (2025)

Demystifying the Paradox of Importance Sampling with an Estimated History-Dependent Behavior Policy in Off-Policy Evaluation
by: Zhou, Hongyi, et al.
Published: (2025)

Quotient DAGs for Off-Policy Evaluation:Forward-Flow Importance Sampling and Exact Slate Propensities
by: Xie, Ziwen, et al.
Published: (2026)

Relative Importance Sampling for off-Policy Actor-Critic in Deep Reinforcement Learning
by: Humayoo, Mahammad, et al.
Published: (2018)

Understanding and Preventing Entropy Collapse in RLVR with On-Policy Entropy Flow Optimization
by: Xu, Huimin, et al.
Published: (2026)

A Step Back: Prefix Importance Ratio Stabilizes Policy Optimization
by: Lei, Shiye, et al.
Published: (2026)

Unified PAC-Bayesian Study of Pessimism for Offline Policy Learning with Regularized Importance Sampling
by: Aouali, Imad, et al.
Published: (2024)

Entropy-Regularized Token-Level Policy Optimization for Language Agent Reinforcement
by: Wen, Muning, et al.
Published: (2024)

Importance Sampling for Nonlinear Models
by: Rajmohan, Prakash Palanivelu, et al.
Published: (2025)

TIP: Token Importance in On-Policy Distillation
by: Xu, Yuanda, et al.
Published: (2026)

SEBA: Sample-Efficient Black-Box Attacks on Visual Reinforcement Learning
by: Huang, Tairan, et al.
Published: (2025)

On the Sample Complexity of Differentially Private Policy Optimization
by: He, Yi, et al.
Published: (2025)

Entropy Meets Importance: A Unified Head Importance-Entropy Score for Stable and Efficient Transformer Pruning
by: Choi, Minsik, et al.
Published: (2025)

Hybrid Group Relative Policy Optimization: A Multi-Sample Approach to Enhancing Policy Optimization
by: Sane, Soham
Published: (2025)

Variational Learning of Gaussian Process Latent Variable Models through Stochastic Gradient Annealed Importance Sampling
by: Xu, Jian, et al.
Published: (2024)

ERPO: Token-Level Entropy-Regulated Policy Optimization for Large Reasoning Models
by: Yu, Song, et al.
Published: (2026)

Maximum Entropy Reinforcement Learning with Diffusion Policy
by: Dong, Xiaoyi, et al.
Published: (2025)

UCPO: Uncertainty-Aware Policy Optimization
by: Zeng, Xianzhou, et al.
Published: (2026)

Offline Multi-Agent Reinforcement Learning via In-Sample Sequential Policy Optimization
by: Liu, Zongkai, et al.
Published: (2024)

Entropy-Gated Selective Policy Optimization:Token-Level Gradient Allocation for Hybrid Training of Large Language Models
by: Hu, Yuelin, et al.
Published: (2026)

Flexible Entropy Control in RLVR with a Gradient-Preserving Perspective
by: Chen, Kun, et al.
Published: (2026)

PORTool: Importance-Aware Policy Optimization with Rewarded Tree for Multi-Tool-Integrated Reasoning
by: Wu, Feijie, et al.
Published: (2025)

Stabilizing Off-Policy Training for Long-Horizon LLM Agent via Turn-Level Importance Sampling and Clipping-Triggered Normalization
by: Li, Chenliang, et al.
Published: (2025)

Unifying Group-Relative and Self-Distillation Policy Optimization via Sample Routing
by: Li, Gengsheng, et al.
Published: (2026)

Agentic Entropy-Balanced Policy Optimization
by: Dong, Guanting, et al.
Published: (2025)

Federated Offline Policy Optimization with Dual Regularization
by: Yue, Sheng, et al.
Published: (2024)

Maximum Entropy On-Policy Actor-Critic via Entropy Advantage Estimation
by: Choe, Jean Seong Bjorn, et al.
Published: (2024)

EP-GRPO: Entropy-Progress Aligned Group Relative Policy Optimization with Implicit Process Guidance
by: Yu, Song, et al.
Published: (2026)