Cover Image

Dynamic Vocabulary Pruning: Stable LLM-RL by Taming the Tail

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Li, Yingru, Xu, Jiawei, Liu, Jiacai, Tong, Yuxuan, Li, Ziniu, Cai, Tianle, Zhang, Ge, Liu, Qian, Wang, Baoxiang
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Trust Region Masking for Long-Horizon LLM Reinforcement Learning
by: Li, Yingru, et al.
Published: (2025)

The Optimal Token Baseline: Variance Reduction for Long-Horizon LLM-RL
by: Li, Yingru, et al.
Published: (2026)

A Note on Hybrid Online Reinforcement and Imitation Learning for LLMs: Formulations and Algorithms
by: Li, Yingru, et al.
Published: (2025)

Beyond Precision: Training-Inference Mismatch is an Optimization Problem and Simple LR Scheduling Fixes It
by: Zhang, Yaxiang, et al.
Published: (2026)

Scalable Exploration via Ensemble++
by: Li, Yingru, et al.
Published: (2024)

Logit Dynamics in Softmax Policy Gradient Methods
by: Li, Yingru
Published: (2025)

Taming the Long-Tail: Efficient Reasoning RL Training with Adaptive Drafter
by: Hu, Qinghao, et al.
Published: (2025)

Dr. Kernel: Reinforcement Learning Done Right for Triton Kernel Generations
by: Liu, Wei, et al.
Published: (2026)

Reshaping Reasoning in LLMs: A Theoretical Analysis of RL Training Dynamics through Pattern Selection
by: Chen, Xingwu, et al.
Published: (2025)

Q-Star Meets Scalable Posterior Sampling: Bridging Theory and Practice via HyperAgent
by: Li, Yingru, et al.
Published: (2024)

Divergence-Augmented Policy Optimization
by: Wang, Qing, et al.
Published: (2025)

RAP: Runtime Adaptive Pruning for LLM Inference
by: Liu, Huanrong, et al.
Published: (2025)

Dynamic Vocabulary Pruning in Early-Exit LLMs
by: Vincenti, Jort, et al.
Published: (2024)

SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild
by: Zeng, Weihao, et al.
Published: (2025)

ADG: Ambient Diffusion-Guided Dataset Recovery for Corruption-Robust Offline Reinforcement Learning
by: Liu, Zeyuan, et al.
Published: (2025)

Token-Efficient RL for LLM Reasoning
by: Lee, Alan, et al.
Published: (2025)

Taming the Long Tail in Human Mobility Prediction
by: Xu, Xiaohang, et al.
Published: (2024)

Relative Policy-Transition Optimization for Fast Policy Transfer
by: Xu, Jiawei, et al.
Published: (2022)

Knapsack RL: Unlocking Exploration of LLMs via Optimizing Budget Allocation
by: Li, Ziniu, et al.
Published: (2025)

QaRL: Rollout-Aligned Quantization-Aware RL for Fast and Stable Training under Training--Inference Mismatch
by: Gu, Hao, et al.
Published: (2026)

Mitigating LLM Hallucination via Behaviorally Calibrated Reinforcement Learning
by: Wu, Jiayun, et al.
Published: (2025)

Taming LLMs by Scaling Learning Rates with Gradient Grouping
by: Li, Siyuan, et al.
Published: (2025)

Protein Design with Dynamic Protein Vocabulary
by: Liu, Nuowei, et al.
Published: (2025)

Sparse-RL: Breaking the Memory Wall in LLM Reinforcement Learning via Stable Sparse Rollouts
by: Luo, Sijia, et al.
Published: (2026)

On Effectiveness and Efficiency of Agentic Tool-calling and RL Training
by: Liu, Tong, et al.
Published: (2026)

Unbiased Dynamic Pruning for Efficient Group-Based Policy Optimization
by: Zhu, Haodong, et al.
Published: (2026)

Dataset Distillation for Offline Reinforcement Learning
by: Light, Jonathan, et al.
Published: (2024)

PruneSymNet: A Symbolic Neural Network and Pruning Algorithm for Symbolic Regression
by: Wu, Min, et al.
Published: (2024)

Learning from Complexity: Exploring Dynamic Sample Pruning of Spatio-Temporal Training
by: Chen, Wei, et al.
Published: (2026)

Prior-dependent analysis of posterior sampling reinforcement learning with function approximation
by: Li, Yingru, et al.
Published: (2024)

Taming the Titans: A Survey of Efficient LLM Inference Serving
by: Zhen, Ranran, et al.
Published: (2025)

MaskPrune: Mask-based LLM Pruning for Layer-wise Uniform Structures
by: Qin, Jiayu, et al.
Published: (2025)

Tackling Data Corruption in Offline Reinforcement Learning via Sequence Modeling
by: Xu, Jiawei, et al.
Published: (2024)

Momentum Boosted Episodic Memory for Improving Learning in Long-Tailed RL Environments
by: Fernandes, Dolton, et al.
Published: (2025)

Taming the Tail in Class-Conditional GANs: Knowledge Sharing via Unconditional Training at Lower Resolutions
by: Khorram, Saeed, et al.
Published: (2024)

ProxyKV: Cross-Model Proxy Pruning for Efficient Long-Context LLM Inference
by: Li, Junjie, et al.
Published: (2026)

Beyond Markovian: Reflective Exploration via Bayes-Adaptive RL for LLM Reasoning
by: Zhang, Shenao, et al.
Published: (2025)

Taming the Tail: NoI Topology Synthesis for Mixed DL Workloads on Chiplet-Based Accelerators
by: Shukla, Arnav, et al.
Published: (2025)

GPrune-LLM: Generalization-Aware Structured Pruning for Large Language Models
by: Liu, Xiaoyun, et al.
Published: (2026)

FlowNet: Modeling Dynamic Spatio-Temporal Systems via Flow Propagation
by: Feng, Yutong, et al.
Published: (2025)