Image de couverture de livre

PPO-Clip Attains Global Optimality: Towards Deeper Understandings of Clipping

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Huang, Nai-Chieh, Hsieh, Ping-Chun, Ho, Kuo-Hao, Wu, I-Chen
Format:	Preprint
Publié:	2023
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Non-Asymptotic Global Convergence of PPO-Clip
par: Liu, Yin, et autres
Publié: (2025)

CacheClip: Accelerating RAG with Effective KV Cache Reuse
par: Yang, Bin, et autres
Publié: (2025)

Clip-and-Verify: Linear Constraint-Driven Domain Clipping for Accelerating Neural Network Verification
par: Zhou, Duo, et autres
Publié: (2025)

Weight Clipping for Deep Continual and Reinforcement Learning
par: Elsayed, Mohamed, et autres
Publié: (2024)

Generalisation of RLHF under Reward Shift and Clipped KL Regularisation
par: Tang, Kenton, et autres
Publié: (2026)

Graph-attention-based Casual Discovery with Trust Region-navigated Clipping Policy Optimization
par: Liu, Shixuan, et autres
Publié: (2024)

Enhanced Generative Model Evaluation with Clipped Density and Coverage
par: Salvy, Nicolas, et autres
Publié: (2025)

DCPO: Dynamic Clipping Policy Optimization
par: Yang, Shihui, et autres
Publié: (2025)

Learning Human-Like RL Agents Through Trajectory Optimization With Action Quantization
par: Guo, Jian-Ting, et autres
Publié: (2025)

Adaptive-Boundary-Clipping GRPO: Ensuring Bounded Ratios for Stable and Generalizable Training
par: Liu, Chi, et autres
Publié: (2026)

Robust Federated Learning Over the Air: Combating Heavy-Tailed Noise with Median Anchored Clipping
par: Li, Jiaxing, et autres
Publié: (2024)

Exploration vs Exploitation: Rethinking RLVR through Clipping, Entropy, and Spurious Reward
par: Chen, Peter, et autres
Publié: (2025)

Adaptive Clipping for Privacy-Preserving Few-Shot Learning: Enhancing Generalization with Limited Data
par: Ranaweera, Kanishka, et autres
Publié: (2025)

It's Not You, It's Clipping: A Soft Trust-Region via Probability Smoothing for LLM RL
par: Dwyer, Madeleine, et autres
Publié: (2025)

Exploiting Estimation Bias in Clipped Double Q-Learning for Continous Control Reinforcement Learning Tasks
par: Turcato, Niccolò, et autres
Publié: (2024)

Synthetic ALS-EEG Data Augmentation for ALS Diagnosis Using Conditional WGAN with Weight Clipping
par: Mutlu, Abdulvahap, et autres
Publié: (2025)

Momentum Streams for Optimizer-Inspired Transformers
par: Gai, Jingchu, et autres
Publié: (2026)

DC-SGD: Differentially Private SGD with Dynamic Clipping through Gradient Norm Distribution Estimation
par: Wei, Chengkun, et autres
Publié: (2025)

Klear-Reasoner: Advancing Reasoning Capability via Gradient-Preserving Clipping Policy Optimization
par: Su, Zhenpeng, et autres
Publié: (2025)

Clip Your Sequences Fairly: Enforcing Length Fairness for Sequence-Level RL
par: Mao, Hanyi, et autres
Publié: (2025)

BAPO: Stabilizing Off-Policy Reinforcement Learning for LLMs via Balanced Policy Optimization with Adaptive Clipping
par: Xi, Zhiheng, et autres
Publié: (2025)

HELENE: Hessian Layer-wise Clipping and Gradient Annealing for Accelerating Fine-tuning LLM with Zeroth-order Optimization
par: Zhao, Huaqin, et autres
Publié: (2024)

BandPO: Bridging Trust Regions and Ratio Clipping via Probability-Aware Bounds for LLM Reinforcement Learning
par: Li, Yuan, et autres
Publié: (2026)

From $\log π$ to $π$: Taming Divergence in Soft Clipping via Bilateral Decoupled Decay of Probability Gradient Weight
par: Fu, Xiaoliang, et autres
Publié: (2026)

To Clip or not to Clip: the Dynamics of SGD with Gradient Clipping in High-Dimensions
par: Marshall, Noah, et autres
Publié: (2024)

CoPE: Clipped RoPE as A Scalable Free Lunch for Long Context LLMs
par: Li, Haoran, et autres
Publié: (2026)

Rotate, Clip, and Partition: Towards W2A4KV4 Quantization by Integrating Rotation and Learnable Non-uniform Quantizer
par: Choi, Euntae, et autres
Publié: (2025)

Self-Reinforced Graph Contrastive Learning
par: Hsieh, Chou-Ying, et autres
Publié: (2025)

Semantic-Clipping: Efficient Vision-Language Modeling with Semantic-Guidedd Visual Selection
par: Li, Bangzheng, et autres
Publié: (2025)

DP-Adam-AC: Privacy-preserving Fine-Tuning of Localizable Language Models Using Adam Optimization with Adaptive Clipping
par: Yang, Ruoxing
Publié: (2025)

ConfClip: Confidence-Weighted and Clipped Reward for Reinforcement Learning in LLMs
par: Zhang, Bonan, et autres
Publié: (2025)

ClipGrader: Leveraging Vision-Language Models for Robust Label Quality Assessment in Object Detection
par: Lu, Hong, et autres
Publié: (2025)

TexAVi: Generating Stereoscopic VR Video Clips from Text Descriptions
par: Srihari, Vriksha, et autres
Publié: (2025)

Stabilizing Off-Policy Training for Long-Horizon LLM Agent via Turn-Level Importance Sampling and Clipping-Triggered Normalization
par: Li, Chenliang, et autres
Publié: (2025)

Byzantine Robustness and Partial Participation Can Be Achieved at Once: Just Clip Gradient Differences
par: Malinovsky, Grigory, et autres
Publié: (2023)

Byzantines can also Learn from History: Fall of Centered Clipping in Federated Learning
par: Ozfatura, Kerem, et autres
Publié: (2022)

EFRame: Deeper Reasoning via Exploration-Filter-Replay Reinforcement Learning Framework
par: Wang, Chen, et autres
Publié: (2025)

Mode-Dependent Rectification for Stable PPO Training
par: Mohamad, Mohamad, et autres
Publié: (2026)

ResNets Are Deeper Than You Think
par: Mehmeti-Göpel, Christian H. X. Ali, et autres
Publié: (2025)

DP-LAC: Lightweight Adaptive Clipping for Differentially Private Federated Fine-tuning of Language Models
par: Mehmood, Haaris, et autres
Publié: (2026)