Cover Image

Design Conditions for Intra-Group Learning of Sequence-Level Rewards: Token Gradient Cancellation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Ding, Fei, Zhang, Yongkang, wang, youwei, Zeng, Zijian
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Rethinking the Comparison Unit in Sequence-Level Reinforcement Learning: An Equal-Length Paired Training Framework from Loss Correction to Sample Construction
by: Ding, Fei, et al.
Published: (2026)

Internalizing Outcome Supervision into Process Supervision: A New Paradigm for Reinforcement Learning for Reasoning
by: Ding, Fei, et al.
Published: (2026)

Reducing Credit Assignment Variance via Counterfactual Reasoning Paths
by: Ding, Fei, et al.
Published: (2026)

Intra-Trajectory Consistency for Reward Modeling
by: Zhou, Chaoyang, et al.
Published: (2025)

T-REG: Preference Optimization with Token-Level Reward Regularization
by: Zhou, Wenxuan, et al.
Published: (2024)

HELM: Harness-Enhanced Long-horizon Memory for Vision-Language-Action Manipulation
by: Zeng, Zijian, et al.
Published: (2026)

Learning to Compare Hardware Designs for High-Level Synthesis
by: Bai, Yunsheng, et al.
Published: (2024)

Balance Reward and Safety Optimization for Safe Reinforcement Learning: A Perspective of Gradient Manipulation
by: Gu, Shangding, et al.
Published: (2024)

Lossless Token Sequence Compression via Meta-Tokens
by: Harvill, John, et al.
Published: (2025)

Selective Preference Optimization via Token-Level Reward Function Estimation
by: Yang, Kailai, et al.
Published: (2024)

Asymmetric On-Policy Distillation: Bridging Exploitation and Imitation at the Token Level
by: Jia, Nan, et al.
Published: (2026)

Cross-Modality Program Representation Learning for Electronic Design Automation with High-Level Synthesis
by: Qin, Zongyue, et al.
Published: (2024)

Subwords as Skills: Tokenization for Sparse-Reward Reinforcement Learning
by: Yunis, David, et al.
Published: (2023)

Beyond Semantic Manipulation: Token-Space Attacks on Reward Models
by: Zhang, Yuheng, et al.
Published: (2026)

DUET: Optimize Token-Budget Allocation for Reinforcement Learning with Verifiable Rewards
by: Hu, Haoyu, et al.
Published: (2026)

Taming LLMs by Scaling Learning Rates with Gradient Grouping
by: Li, Siyuan, et al.
Published: (2025)

Tiered Reward: Designing Rewards for Specification and Fast Learning of Desired Behavior
by: Zhou, Zhiyuan, et al.
Published: (2022)

TGDPO: Harnessing Token-Level Reward Guidance for Enhancing Direct Preference Optimization
by: Zhu, Mingkang, et al.
Published: (2025)

Temporal Logic Specification-Conditioned Decision Transformer for Offline Safe Reinforcement Learning
by: Guo, Zijian, et al.
Published: (2024)

NExT-Guard: Training-Free Streaming Safeguard without Token-Level Labels
by: Fang, Junfeng, et al.
Published: (2026)

Hierarchical Mixture of Experts: Generalizable Learning for High-Level Synthesis
by: Li, Weikai, et al.
Published: (2024)

Reinforcement Learning with Conditional Expectation Reward
by: Xiao, Changyi, et al.
Published: (2026)

Collapsing Sequence-Level Data-Policy Coverage via Poisoning Attack in Offline Reinforcement Learning
by: Zhou, Xue, et al.
Published: (2025)

Entropy-Gated Selective Policy Optimization:Token-Level Gradient Allocation for Hybrid Training of Large Language Models
by: Hu, Yuelin, et al.
Published: (2026)

Eureka: Human-Level Reward Design via Coding Large Language Models
by: Ma, Yecheng Jason, et al.
Published: (2023)

Token-Level LLM Collaboration via FusionRoute
by: Xiong, Nuoya, et al.
Published: (2026)

InfoRM: Mitigating Reward Hacking in RLHF via Information-Theoretic Reward Modeling
by: Miao, Yuchun, et al.
Published: (2024)

$K$-Level Policy Gradients for Multi-Agent Reinforcement Learning
by: Reddi, Aryaman, et al.
Published: (2025)

Gradient Regularization Prevents Reward Hacking in Reinforcement Learning from Human Feedback and Verifiable Rewards
by: Ackermann, Johannes, et al.
Published: (2026)

LaSeR: Reinforcement Learning with Last-Token Self-Rewarding
by: Yang, Wenkai, et al.
Published: (2025)

Route Experts by Sequence, not by Token
by: Wen, Tiansheng, et al.
Published: (2025)

Noise-corrected GRPO: From Noisy Rewards to Unbiased Gradients
by: Mansouri, Omar El, et al.
Published: (2025)

Context-Former: Stitching via Latent Conditioned Sequence Modeling
by: Zhang, Ziqi, et al.
Published: (2024)

ORSO: Accelerating Reward Design via Online Reward Selection and Policy Optimization
by: Zhang, Chen Bo Calvin, et al.
Published: (2024)

Distributional Process Reward Models: Calibrated Prediction of Future Rewards via Conditional Optimal Transport
by: Ma, Rachel, et al.
Published: (2026)

MeTok: An Efficient Meteorological Tokenization with Hyper-Aligned Group Learning for Precipitation Nowcasting
by: Jin, Qizhao, et al.
Published: (2026)

TNPAR: Topological Neural Poisson Auto-Regressive Model for Learning Granger Causal Structure from Event Sequences
by: Liu, Yuequn, et al.
Published: (2023)

TOPReward: Token Probabilities as Hidden Zero-Shot Rewards for Robotics
by: Chen, Shirui, et al.
Published: (2026)

TSSR: Two-Stage Swap-Reward-Driven Reinforcement Learning for Character-Level SMILES Generation
by: Levine, Jacob Ede, et al.
Published: (2026)

Verifier-Free RL for LLMs via Intrinsic Gradient-Norm Reward
by: Wen, Xuexiang, et al.
Published: (2026)