Cover Image

GTPO and GRPO-S: Token and Sequence-Level Reward Shaping with Policy Entropy

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Tan, Hongze, Wang, Zihan, Pan, Jianfei, Lin, Jinghao, Wang, Hao, Wu, Yifan, Chen, Tao, Zheng, Zhihang, Tang, Zhihao, Yang, Haihua
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

GTPO: Stabilizing Group Relative Policy Optimization via Gradient and Entropy Control
by: Simoni, Marco, et al.
Published: (2025)

Token-Level Policy Optimization: Linking Group-Level Rewards to Token-Level Aggregation via Sequence-Level Likelihood
by: Lin, Xingyu, et al.
Published: (2026)

SEED-GRPO: Semantic Entropy Enhanced GRPO for Uncertainty-Aware Policy Optimization
by: Chen, Minghan, et al.
Published: (2025)

Token-Level Policy Optimization: Linking Group-Level Rewards to Token-Level Aggregation via Markov Likelihood
by: Lin, Xingyu, et al.
Published: (2025)

Discriminative Policy Optimization for Token-Level Reward Models
by: Chen, Hongzhan, et al.
Published: (2025)

Smaller Models are Natural Explorers for Policy-Level Diversity in GRPO
by: Ren, Yiming, et al.
Published: (2026)

Improving DAPO from a Mixed-Policy Perspective
by: Tan, Hongze, et al.
Published: (2025)

$λ$-GRPO: Unifying the GRPO Frameworks with Learnable Token Preferences
by: Wang, Yining, et al.
Published: (2025)

DEEPMED: Building a Medical DeepResearch Agent via Multi-hop Med-Search Data and Turn-Controlled Agentic Training & Inference
by: Wang, Zihan, et al.
Published: (2026)

ResT: Reshaping Token-Level Policy Gradients for Tool-Use Large Language Models
by: Lin, Zihan, et al.
Published: (2025)

Entropy-Regularized Token-Level Policy Optimization for Language Agent Reinforcement
by: Wen, Muning, et al.
Published: (2024)

Shaping Explanations: Semantic Reward Modeling with Encoder-Only Transformers for GRPO
by: Pappone, Francesco, et al.
Published: (2025)

Pref-GRPO: Pairwise Preference Reward-based GRPO for Stable Text-to-Image Reinforcement Learning
by: Wang, Yibin, et al.
Published: (2025)

DNAMotifTokenizer: Towards Biologically Informed Tokenization of Genomic Sequences
by: Zhou, Xiaoxiao, et al.
Published: (2025)

RC-GRPO: Reward-Conditioned Group Relative Policy Optimization for Multi-Turn Tool Calling Agents
by: Zhong, Haitian, et al.
Published: (2026)

GRPO and Reflection Reward for Mathematical Reasoning in Large Language Models
by: Wang, Zhijie
Published: (2026)

ShapE-GRPO: Shapley-Enhanced Reward Allocation for Multi-Candidate LLM Training
by: Ai, Rui, et al.
Published: (2026)

T-REG: Preference Optimization with Token-Level Reward Regularization
by: Zhou, Wenxuan, et al.
Published: (2024)

SafeGRPO: Self-Rewarded Multimodal Safety Alignment via Rule-Governed Policy Optimization
by: Rong, Xuankun, et al.
Published: (2025)

OP-GRPO: Efficient Off-Policy GRPO for Flow-Matching Models
by: Zhang, Liyu, et al.
Published: (2026)

BaseReward: A Strong Baseline for Multimodal Reward Model
by: Zhang, Yi-Fan, et al.
Published: (2025)

Learning to Generate Secure Code via Token-Level Rewards
by: Quan, Jiazheng, et al.
Published: (2026)

LaF-GRPO: In-Situ Navigation Instruction Generation for the Visually Impaired via GRPO with LLM-as-Follower Reward
by: Zhao, Yi, et al.
Published: (2025)

Design Conditions for Intra-Group Learning of Sequence-Level Rewards: Token Gradient Cancellation
by: Ding, Fei, et al.
Published: (2026)

Beyond GRPO and On-Policy Distillation: An Empirical Sparse-to-Dense Reward Principle for Language-Model Post-Training
by: Xu, Yuanda, et al.
Published: (2026)

Learning from Medical Entity Trees: An Entity-Centric Medical Data Engineering Framework for MLLMs
by: Lin, Jianghang, et al.
Published: (2026)

GROW: Aligning GRPO with State-Action Modeling for Open-World VLM Agents
by: Wu, Xiongbin, et al.
Published: (2026)

Improving Generalization in Intent Detection: GRPO with Reward-Based Curriculum Sampling
by: Feng, Zihao, et al.
Published: (2025)

ARES: Multimodal Adaptive Reasoning via Difficulty-Aware Token-Level Entropy Shaping
by: Chen, Shuang, et al.
Published: (2025)

Lattice Genome Framework for Regionally Tailored Component‐Level Multi‐Objective Design in Additive Manufacturing
by: Haoyuan Deng, et al.
Published: (2026)

ERPO: Token-Level Entropy-Regulated Policy Optimization for Large Reasoning Models
by: Yu, Song, et al.
Published: (2026)

SR-GRPO: Stable Rank as an Intrinsic Geometric Reward for Large Language Model Alignment
by: Tang, Yixuan, et al.
Published: (2025)

EDGE-GRPO: Entropy-Driven GRPO with Guided Error Correction for Advantage Diversity
by: Zhang, Xingjian, et al.
Published: (2025)

MMR-GRPO: Accelerating GRPO-Style Training through Diversity-Aware Reward Reweighting
by: Wei, Kangda, et al.
Published: (2026)

Entropy-Regularized Process Reward Model
by: Zhang, Hanning, et al.
Published: (2024)

GRPO is Secretly a Process Reward Model
by: Sullivan, Michael, et al.
Published: (2025)

Route Experts by Sequence, not by Token
by: Wen, Tiansheng, et al.
Published: (2025)

RED: Unleashing Token-Level Rewards from Holistic Feedback via Reward Redistribution
by: Li, Jiahui, et al.
Published: (2024)

Lossless Token Sequence Compression via Meta-Tokens
by: Harvill, John, et al.
Published: (2025)

Within the Dynamic Context: Inertia-aware 3D Human Modeling with Pose Sequence
by: Chen, Yutong, et al.
Published: (2024)