Cover Image

Do Not Let Low-Probability Tokens Over-Dominate in RL for LLMs

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yang, Zhihe, Luo, Xufang, Wang, Zilong, Han, Dongqi, He, Zhiyuan, Li, Dongsheng, Xu, Yunjian
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

SortedRL: Accelerating RL Training for LLMs through Online Length-Aware Scheduling
by: Zhang, Yiqi, et al.
Published: (2026)

Mitigating Hallucinations in Large Vision-Language Models via DPO: On-Policy Data Hold the Key
by: Yang, Zhihe, et al.
Published: (2025)

LLM-RadJudge: Achieving Radiologist-Level Evaluation for X-Ray Report Generation
by: Wang, Zilong, et al.
Published: (2024)

Agent Lightning: Train ANY AI Agents with Reinforcement Learning
by: Luo, Xufang, et al.
Published: (2025)

Do LLMs Encode Functional Importance of Reasoning Tokens?
by: Singh, Janvijay, et al.
Published: (2026)

RST-LoRA: A Discourse-Aware Low-Rank Adaptation for Long Document Abstractive Summarization
by: Liu, Dongqi, et al.
Published: (2024)

Understanding Reasoning in LLMs through Strategic Information Allocation under Uncertainty
by: Kim, Jeonghye, et al.
Published: (2026)

Principled RL for Diffusion LLMs Emerges from a Sequence-Level Perspective
by: Ou, Jingyang, et al.
Published: (2025)

Understanding Token Probability Encoding in Output Embeddings
by: Cho, Hakaze, et al.
Published: (2024)

Knapsack RL: Unlocking Exploration of LLMs via Optimizing Budget Allocation
by: Li, Ziniu, et al.
Published: (2025)

Position Engineering: Boosting Large Language Models through Positional Information Manipulation
by: He, Zhiyuan, et al.
Published: (2024)

Let the Code LLM Edit Itself When You Edit the Code
by: He, Zhenyu, et al.
Published: (2024)

VL Norm: Rethink Loss Aggregation in RLVR
by: He, Zhiyuan, et al.
Published: (2025)

POMP: Probability-driven Meta-graph Prompter for LLMs in Low-resource Unsupervised Neural Machine Translation
by: Pan, Shilong, et al.
Published: (2024)

pMoE: Prompting Diverse Experts Together Wins More in Visual Adaptation
by: Mo, Shentong, et al.
Published: (2026)

Exploratory Memory-Augmented LLM Agent via Hybrid On- and Off-Policy Optimization
by: Liu, Zeyuan, et al.
Published: (2026)

ChatGPT vs Human-authored Text: Insights into Controllable Text Summarization and Sentence Style Transfer
by: Liu, Dongqi, et al.
Published: (2023)

ADG: Ambient Diffusion-Guided Dataset Recovery for Corruption-Robust Offline Reinforcement Learning
by: Liu, Zeyuan, et al.
Published: (2025)

TruthRL: Incentivizing Truthful LLMs via Reinforcement Learning
by: Wei, Zhepei, et al.
Published: (2025)

The Price of Format: Diversity Collapse in LLMs
by: Yun, Longfei, et al.
Published: (2025)

Nemotron-Cascade 2: Post-Training LLMs with Cascade RL and Multi-Domain On-Policy Distillation
by: Yang, Zhuolin, et al.
Published: (2026)

Habits and goals in synergy: a variational Bayesian framework for behavior
by: Han, Dongqi, et al.
Published: (2023)

State-Action Inpainting Diffuser for Continuous Control with Delay
by: Han, Dongqi, et al.
Published: (2026)

What Makes a Good Diffusion Planner for Decision Making?
by: Lu, Haofei, et al.
Published: (2025)

Learning to Route LLMs with Confidence Tokens
by: Chuang, Yu-Neng, et al.
Published: (2024)

Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training
by: Liu, Mingjie, et al.
Published: (2025)

RHYTHM: Reasoning with Hierarchical Temporal Tokenization for Human Mobility
by: He, Haoyu, et al.
Published: (2025)

XAI4LLM. Let Machine Learning Models and LLMs Collaborate for Enhanced In-Context Learning in Healthcare
by: Nazary, Fatemeh, et al.
Published: (2024)

GaLore$+$: Boosting Low-Rank Adaptation for LLMs with Cross-Head Projection
by: Liao, Xutao, et al.
Published: (2024)

Team Ryu's Submission to SIGMORPHON 2024 Shared Task on Subword Tokenization
by: Li, Zilong
Published: (2024)

Hypertokens: Holographic Associative Memory in Tokenized LLMs
by: Augeri, Christopher James
Published: (2025)

Critical Tokens Matter: Token-Level Contrastive Estimation Enhances LLM's Reasoning Capability
by: Lin, Zicheng, et al.
Published: (2024)

Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wisely
by: Zhao, Siyun, et al.
Published: (2024)

Incorporating Distributions of Discourse Structure for Long Document Abstractive Summarization
by: Liu, Dongqi, et al.
Published: (2023)

Let Models Speak Ciphers: Multiagent Debate through Embeddings
by: Pham, Chau, et al.
Published: (2023)

How Likely Do LLMs with CoT Mimic Human Reasoning?
by: Bao, Guangsheng, et al.
Published: (2024)

Memento-Skills: Let Agents Design Agents
by: Zhou, Huichi, et al.
Published: (2026)

InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory
by: Xiao, Chaojun, et al.
Published: (2024)

SPEC-RL: Accelerating On-Policy Reinforcement Learning with Speculative Rollouts
by: Liu, Bingshuai, et al.
Published: (2025)

LSPT: Long-term Spatial Prompt Tuning for Visual Representation Learning
by: Mo, Shentong, et al.
Published: (2024)