Cover Image

TDRM: Smooth Reward Models with Temporal Difference for LLM RL and Inference

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhang, Dan, Cai, Min, Light, Jonathan, Hu, Ziniu, Yue, Yisong, Tang, Jie
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

DataSciBench: An LLM Agent Benchmark for Data Science
by: Zhang, Dan, et al.
Published: (2025)

TreeRL: LLM Reinforcement Learning with On-Policy Tree Search
by: Hou, Zhenyu, et al.
Published: (2025)

ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Search
by: Zhang, Dan, et al.
Published: (2024)

Beyond Numeric Rewards: In-Context Dueling Bandits with LLM Agents
by: Xia, Fanzeng, et al.
Published: (2024)

Strategist: Self-improvement of LLM Decision Making via Bi-Level Tree Search
by: Light, Jonathan, et al.
Published: (2024)

Optimized Multi-Token Joint Decoding with Auxiliary Model for LLM Inference
by: Qin, Zongyue, et al.
Published: (2024)

Discursive Circuits: How Do Language Models Understand Discourse Relations?
by: Miao, Yisong, et al.
Published: (2025)

FlowRL: Matching Reward Distributions for LLM Reasoning
by: Zhu, Xuekai, et al.
Published: (2025)

SceneCraft: An LLM Agent for Synthesizing 3D Scene as Blender Code
by: Hu, Ziniu, et al.
Published: (2024)

Diagnosing and Mitigating System Bias in Self-Rewarding RL
by: Tan, Chuyi, et al.
Published: (2025)

Chart-RL: Generalized Chart Comprehension via Reinforcement Learning with Verifiable Rewards
by: Zhang, Xin, et al.
Published: (2026)

Self-Control of LLM Behaviors by Compressing Suffix Gradient into Prefix Controller
by: Cai, Min, et al.
Published: (2024)

On-Policy RL with Optimal Reward Baseline
by: Hao, Yaru, et al.
Published: (2025)

On Designing Effective RL Reward at Training Time for LLM Reasoning
by: Gao, Jiaxuan, et al.
Published: (2024)

Dataset Distillation for Offline Reinforcement Learning
by: Light, Jonathan, et al.
Published: (2024)

Rewarding the Rare: Uniqueness-Aware RL for Creative Problem Solving in LLMs
by: Hu, Zhiyuan, et al.
Published: (2026)

Exploring RL-based LLM Training for Formal Language Tasks with Programmed Rewards
by: Padula, Alexander G., et al.
Published: (2024)

RewardAnything: Generalizable Principle-Following Reward Models
by: Yu, Zhuohao, et al.
Published: (2025)

On the Power of (Approximate) Reward Models for Inference-Time Scaling
by: Zhu, Youheng, et al.
Published: (2026)

DISC: Dynamic Decomposition Improves LLM Inference Scaling
by: Light, Jonathan, et al.
Published: (2025)

Offline RL by Reward-Weighted Fine-Tuning for Conversation Optimization
by: Mukherjee, Subhojyoti, et al.
Published: (2025)

Personalized RewardBench: Evaluating Reward Models with Human Aligned Personalization
by: Ma, Qiyao, et al.
Published: (2026)

Efficient Ensemble for Fine-tuning Language Models on Multiple Datasets
by: Li, Dongyue, et al.
Published: (2025)

PIANIST: Learning Partially Observable World Models with LLMs for Multi-Agent Decision Making
by: Light, Jonathan, et al.
Published: (2024)

When Sharpening Becomes Collapse: Sampling Bias and Semantic Coupling in RL with Verifiable Rewards
by: Fan, Mingyuan, et al.
Published: (2026)

DISA: Offline Importance Sampling for Distribution-Matching LLM-RL
by: Wang, Shaobo, et al.
Published: (2026)

BWArea Model: Learning World Model, Inverse Dynamics, and Policy for Controllable Language Generation
by: Jia, Chengxing, et al.
Published: (2024)

LLM-PS: Empowering Large Language Models for Time Series Forecasting with Temporal Patterns and Semantics
by: Tang, Jialiang, et al.
Published: (2025)

RewardFlow: Topology-Aware Reward Propagation on State Graphs for Agentic RL with Large Language Models
by: Feng, Xiao, et al.
Published: (2026)

Consolidating Rewarded Perturbations for LLM Post-Training
by: Zhang, Zheyu, et al.
Published: (2026)

Smoothing Out Hallucinations: Mitigating LLM Hallucination with Smoothed Knowledge Distillation
by: Nguyen, Hieu, et al.
Published: (2025)

Knapsack RL: Unlocking Exploration of LLMs via Optimizing Budget Allocation
by: Li, Ziniu, et al.
Published: (2025)

Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoning
by: Setlur, Amrith, et al.
Published: (2024)

Exploration vs Exploitation: Rethinking RLVR through Clipping, Entropy, and Spurious Reward
by: Chen, Peter, et al.
Published: (2025)

Sentence-level Reward Model can Generalize Better for Aligning LLM from Human Preference
by: Qiu, Wenjie, et al.
Published: (2025)

Ltri-LLM: Streaming Long Context Inference for LLMs with Training-Free Dynamic Triangular Attention Pattern
by: Tang, Hongyin, et al.
Published: (2024)

Inference-Time Scaling for Generalist Reward Modeling
by: Liu, Zijun, et al.
Published: (2025)

Self-Comparison for Dataset-Level Membership Inference in Large (Vision-)Language Models
by: Ren, Jie, et al.
Published: (2024)

RubricEM: Meta-RL with Rubric-guided Policy Decomposition beyond Verifiable Rewards
by: Li, Gaotang, et al.
Published: (2026)

ToolRL: Reward is All Tool Learning Needs
by: Qian, Cheng, et al.
Published: (2025)