Copertina

Trajectory Bellman Residual Minimization: A Simple Value-Based Method for LLM Reasoning

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Yuan, Yurun, Chen, Fan, Jia, Zeyu, Rakhlin, Alexander, Xie, Tengyang
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning Artificial Intelligence Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Reinforce LLM Reasoning through Multi-Agent Reflection
di: Yuan, Yurun, et al.
Pubblicazione: (2025)

Breaking the Capability Ceiling of LLM Post-Training by Reintroducing Markov States
di: Yuan, Yurun, et al.
Pubblicazione: (2026)

Do We Need to Verify Step by Step? Rethinking Process Supervision from a Theoretical Perspective
di: Jia, Zeyu, et al.
Pubblicazione: (2025)

Outcome-Based Online Reinforcement Learning: Algorithms and Fundamental Limits
di: Chen, Fan, et al.
Pubblicazione: (2025)

Exploratory Preference Optimization: Harnessing Implicit Q*-Approximation for Sample-Efficient RLHF
di: Xie, Tengyang, et al.
Pubblicazione: (2024)

Offline Reinforcement Learning: Role of State Aggregation and Trajectory Data
di: Jia, Zeyu, et al.
Pubblicazione: (2024)

R$^2$PO: Decoupling Training Trajectories from Inference Responses for LLM Reasoning
di: Wang, Jingchu, et al.
Pubblicazione: (2026)

Self-Play with Adversarial Critic: Provable and Scalable Offline Alignment for Language Models
di: Ji, Xiang, et al.
Pubblicazione: (2024)

Decision Making in Changing Environments: Robustness, Query-Based Learning, and Differential Privacy
di: Chen, Fan, et al.
Pubblicazione: (2025)

GaussMark: A Practical Approach for Structural Watermarking of Language Models
di: Block, Adam, et al.
Pubblicazione: (2025)

SafePred: A Predictive Guardrail for Computer-Using Agents via World Models
di: Chen, Yurun, et al.
Pubblicazione: (2026)

CounterCurate: Enhancing Physical and Semantic Visio-Linguistic Compositional Reasoning via Counterfactual Examples
di: Zhang, Jianrui, et al.
Pubblicazione: (2024)

Advancing LLM Reasoning Generalists with Preference Trees
di: Yuan, Lifan, et al.
Pubblicazione: (2024)

LLM Reasoning as Trajectories: Step-Specific Representation Geometry and Correctness Signals
di: Sun, Lihao, et al.
Pubblicazione: (2026)

Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferences
di: Rosset, Corby, et al.
Pubblicazione: (2024)

POLCA: Stochastic Generative Optimization with LLM
di: Ren, Xuanfei, et al.
Pubblicazione: (2026)

Offline Reinforcement Learning in Large State Spaces: Algorithms and Guarantees
di: Jiang, Nan, et al.
Pubblicazione: (2025)

dLLM: Simple Diffusion Language Modeling
di: Zhou, Zhanhui, et al.
Pubblicazione: (2026)

Trial and Error: Exploration-Based Trajectory Optimization for LLM Agents
di: Song, Yifan, et al.
Pubblicazione: (2024)

R-Stitch: Dynamic Trajectory Stitching for Efficient Reasoning
di: Chen, Zhuokun, et al.
Pubblicazione: (2025)

SFT-then-RL Outperforms Mixed-Policy Methods for LLM Reasoning
di: Limozin, Alexis, et al.
Pubblicazione: (2026)

Distributional Open-Ended Evaluation of LLM Cultural Value Alignment Based on Value Codebook
di: Lee, Jaehyeok, et al.
Pubblicazione: (2026)

AMAQ: Adaptive Mixed-bit Activation Quantization for Collaborative Parameter Efficient Fine-tuning
di: Song, Yurun, et al.
Pubblicazione: (2025)

Beyond Speedup -- Utilizing KV Cache for Sampling and Reasoning
di: Xing, Zeyu, et al.
Pubblicazione: (2026)

Correcting the Mythos of KL-Regularization: Direct Alignment without Overoptimization via Chi-Squared Preference Optimization
di: Huang, Audrey, et al.
Pubblicazione: (2024)

On the Design of KL-Regularized Policy Gradient Algorithms for LLM Reasoning
di: Zhang, Yifan, et al.
Pubblicazione: (2025)

Training-Trajectory-Aware Token Selection
di: Shen, Zhanming, et al.
Pubblicazione: (2026)

Frontier LLMs Still Struggle with Simple Reasoning Tasks
di: Malek, Alan, et al.
Pubblicazione: (2025)

Simple Policy Gradients for Reasoning with Diffusion Language Models
di: Zhan, Anthony
Pubblicazione: (2025)

Low-rank Optimization Trajectories Modeling for LLM RLVR Acceleration
di: Chen, Zhipeng, et al.
Pubblicazione: (2026)

Supervised Reinforcement Learning: From Expert Trajectories to Step-wise Reasoning
di: Deng, Yihe, et al.
Pubblicazione: (2025)

Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective
di: Cheng, Zhoujun, et al.
Pubblicazione: (2025)

An Embarrassingly Simple Defense Against LLM Abliteration Attacks
di: Shairah, Harethah Abu, et al.
Pubblicazione: (2025)

Simple Yet Effective: An Information-Theoretic Approach to Multi-LLM Uncertainty Quantification
di: Kruse, Maya, et al.
Pubblicazione: (2025)

DOTS: Learning to Reason Dynamically in LLMs via Optimal Reasoning Trajectories Search
di: Yue, Murong, et al.
Pubblicazione: (2024)

Towards a Theoretical Understanding of Synthetic Data in LLM Post-Training: A Reverse-Bottleneck Perspective
di: Gan, Zeyu, et al.
Pubblicazione: (2024)

Semi-Clairvoyant Scheduling of Speculative Decoding Requests to Minimize LLM Inference Latency
di: Li, Ruixiao, et al.
Pubblicazione: (2025)

Decoupling KL and Trajectories: A Unified Perspective for SFT, DAgger, Offline RL, and OPD in LLM Distillation
di: Zhao, Anhao, et al.
Pubblicazione: (2026)

CAPO: Towards Enhancing LLM Reasoning through Generative Credit Assignment
di: Xie, Guofu, et al.
Pubblicazione: (2025)

Embedding Trajectory for Out-of-Distribution Detection in Mathematical Reasoning
di: Wang, Yiming, et al.
Pubblicazione: (2024)