Cover Image

Entropy Ratio Clipping as a Soft Global Constraint for Stable Reinforcement Learning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Su, Zhenpeng, Pan, Leiyu, Lv, Minxuan, Mei, Tiehua, Lin, Zijia, Li, Yuntao, Hu, Wenping, Tang, Ruiming, Gai, Kun, Zhou, Guorui
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

CE-GPPO: Coordinating Entropy via Gradient-Preserving Clipping Policy Optimization in Reinforcement Learning
by: Su, Zhenpeng, et al.
Published: (2025)

Klear-Reasoner: Advancing Reasoning Capability via Gradient-Preserving Clipping Policy Optimization
by: Su, Zhenpeng, et al.
Published: (2025)

Good Reasoning Makes Good Demonstrations: Implicit Reasoning Quality Supervision via In-Context Reinforcement Learning
by: Mei, Tiehua, et al.
Published: (2026)

GoLongRL: Capability-Oriented Long Context Reinforcement Learning with Multitask Alignment
by: Lv, Minxuan, et al.
Published: (2026)

DSMoE: Matrix-Partitioned Experts with Dynamic Routing for Computation-Efficient Dense LLMs
by: Lv, Minxuan, et al.
Published: (2025)

Finedeep: Mitigating Sparse Activation in Dense LLMs via Multi-Layer Fine-Grained Experts
by: Pan, Leiyu, et al.
Published: (2025)

When Importance Sampling Misallocates Credit: Asymmetric Ratios for Outcome-Supervised RL
by: Wang, Jiakang, et al.
Published: (2025)

CartesianMoE: Boosting Knowledge Sharing among Experts via Cartesian Product Routing in Mixture-of-Experts
by: Su, Zhenpeng, et al.
Published: (2024)

GRank: Towards Target-Aware and Streamlined Industrial Retrieval with a Generate-Rank Framework
by: Sun, Yijia, et al.
Published: (2025)

DiffGRM: Diffusion-based Generative Recommendation Model
by: Liu, Zhao, et al.
Published: (2025)

PROMISE: Process Reward Models Unlock Test-Time Scaling Laws in Generative Recommendations
by: Guo, Chengcheng, et al.
Published: (2026)

Clip-Low Increases Entropy and Clip-High Decreases Entropy in Reinforcement Learning of Large Language Models
by: Park, Jaesung R., et al.
Published: (2025)

MiLe Loss: a New Entropy-Weighed Loss for Mitigating the Bias of Learning Difficulties in Large Language Models
by: Su, Zhenpeng, et al.
Published: (2023)

GEMs: Breaking the Long-Sequence Barrier in Generative Recommendation with a Multi-Stream Decoder
by: Zhou, Yu, et al.
Published: (2026)

DADF: A Distribution-Aware Debiasing Framework for Watch-Time Regression in Recommender Systems
by: Yang, Yiqing, et al.
Published: (2026)

DARL: Encouraging Diverse Answers for General Reasoning without Verifiers
by: Huang, Chongxuan, et al.
Published: (2026)

Adaptive-Boundary-Clipping GRPO: Ensuring Bounded Ratios for Stable and Generalizable Training
by: Liu, Chi, et al.
Published: (2026)

An Empirical Study on the Robustness of Massively Multilingual Neural Machine Translation
by: Supryadi, et al.
Published: (2024)

CAPTS: Channel-Aware, Preference-Aligned Trigger Selection for Multi-Channel Item-to-Item Retrieval
by: Zhou, Xiaoyou, et al.
Published: (2026)

DeepSynth-Eval: Objectively Evaluating Information Consolidation in Deep Survey Writing
by: Zhang, Hongzhi, et al.
Published: (2026)

HoME: Hierarchy of Multi-Gate Experts for Multi-Task Learning at Kuaishou
by: Wang, Xu, et al.
Published: (2024)

Entropy formula for surface diffeomorphisms
by: Zang, Yuntao
Published: (2026)

OneLoc: Geo-Aware Generative Recommender Systems for Local Life Service
by: Wei, Zhipeng, et al.
Published: (2025)

Unleashing the Native Recommendation Potential: LLM-Based Generative Recommendation via Structured Term Identifiers
by: Zhang, Zhiyang, et al.
Published: (2026)

Towards Constraint-Based Adaptive Hypergraph Learning for Solving Vehicle Routing: An End-to-End Solution
by: Wang, Zhenwei, et al.
Published: (2025)

From Tags to Trees: Structuring Fine-Grained Knowledge for Controllable Data Selection in LLM Instruction Tuning
by: Niu, Zihan, et al.
Published: (2026)

SoftAdaClip: A Smooth Clipping Strategy for Fair and Private Model Training
by: Soleymani, Dorsa, et al.
Published: (2025)

DPWriter: Reinforcement Learning with Diverse Planning Branching for Creative Writing
by: Cao, Qian, et al.
Published: (2026)

FIM: Frequency-Aware Multi-View Interest Modeling for Local-Life Service Recommendation
by: Wang, Guoquan, et al.
Published: (2025)

Clip-and-Verify: Linear Constraint-Driven Domain Clipping for Accelerating Neural Network Verification
by: Zhou, Duo, et al.
Published: (2025)

PPO-Clip Attains Global Optimality: Towards Deeper Understandings of Clipping
by: Huang, Nai-Chieh, et al.
Published: (2023)

ConfClip: Confidence-Weighted and Clipped Reward for Reinforcement Learning in LLMs
by: Zhang, Bonan, et al.
Published: (2025)

BandPO: Bridging Trust Regions and Ratio Clipping via Probability-Aware Bounds for LLM Reinforcement Learning
by: Li, Yuan, et al.
Published: (2026)

Reinforcement Learning for Option Hedging: Static Implied-Volatility Fit versus Shortfall-Aware Performance
by: Chen, Ziheng, et al.
Published: (2026)

Autonomous AI Agents for Option Hedging: Enhancing Financial Stability through Shortfall Aware Reinforcement Learning
by: Hu, Minxuan, et al.
Published: (2026)

The RNA-seq raw data
by: Pan, Wenping
Published: (2030)

From $\log π$ to $π$: Taming Divergence in Soft Clipping via Bilateral Decoupled Decay of Probability Gradient Weight
by: Fu, Xiaoliang, et al.
Published: (2026)

Clipped Stochastic Gradient Tracking For Locally Smooth Functions
by: Mei, Leilei, et al.
Published: (2026)

LANDeRMT: Detecting and Routing Language-Aware Neurons for Selectively Finetuning LLMs to Machine Translation
by: Zhu, Shaolin, et al.
Published: (2024)

Agentic Entropy-Balanced Policy Optimization
by: Dong, Guanting, et al.
Published: (2025)