Cover Image

Natural Language Actor-Critic: Scalable Off-Policy Learning in Language Space

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Hong, Joey, Liu, Kang, Ling, Zhan, Chen, Jiecao, Levine, Sergey
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Q-SFT: Q-Learning for Language Models via Supervised Fine-Tuning
by: Hong, Joey, et al.
Published: (2024)

Interactive Dialogue Agents via Reinforcement Learning on Hindsight Regenerations
by: Hong, Joey, et al.
Published: (2024)

Scaling Long-Horizon LLM Agent via Context-Folding
by: Sun, Weiwei, et al.
Published: (2025)

Off-Policy Value-Based Reinforcement Learning for Large Language Models
by: Wang, Peng-Yuan, et al.
Published: (2026)

GeoReasoner: Reasoning On Geospatially Grounded Context For Natural Language Understanding
by: Yan, Yibo, et al.
Published: (2024)

Scaling LLM Multi-turn RL with End-to-end Summarization-based Context Management
by: Lu, Miao, et al.
Published: (2025)

MIR-Bench: Can Your LLM Recognize Complicated Patterns via Many-Shot In-Context Reasoning?
by: Yan, Kai, et al.
Published: (2025)

Self-Augmented Preference Optimization: Off-Policy Paradigms for Language Model Alignment
by: Yin, Yueqin, et al.
Published: (2024)

Unfamiliar Finetuning Examples Control How Language Models Hallucinate
by: Kang, Katie, et al.
Published: (2024)

Zero-Overhead Introspection for Adaptive Test-Time Compute
by: Manvi, Rohin, et al.
Published: (2025)

Large Language Model Post-Training: A Unified View of Off-Policy and On-Policy Learning
by: Zhao, Shiwan, et al.
Published: (2026)

Implicit Actor Critic Coupling via a Supervised Learning Framework for RLVR
by: Li, Jiaming, et al.
Published: (2025)

Learning to Reason without External Rewards
by: Zhao, Xuandong, et al.
Published: (2025)

Generalizable End-to-End Tool-Use RL with Synthetic CodeGym
by: Du, Weihua, et al.
Published: (2025)

RLAC: Reinforcement Learning with Adversarial Critic for Free-Form Generation Tasks
by: Wu, Mian, et al.
Published: (2025)

Your Language Model is Its Own Critic: Reinforcement Learning with Value Estimation from Actor's Internal States
by: Choi, Yunho, et al.
Published: (2026)

Simple Policy Gradients for Reasoning with Diffusion Language Models
by: Zhan, Anthony
Published: (2025)

Robust and Scalable Model Editing for Large Language Models
by: Chen, Yingfa, et al.
Published: (2024)

Nested-ReFT: Efficient Reinforcement Learning for Large Language Model Fine-Tuning via Off-Policy Rollouts
by: Heuillet, Maxime, et al.
Published: (2025)

How Off-Policy Can GRPO Be? Mu-GRPO for Efficient LLM Reinforcement Learning
by: Tian, Minghao, et al.
Published: (2026)

Recitation over Reasoning: How Cutting-Edge Language Models Can Fail on Elementary School-Level Reasoning Problems?
by: Yan, Kai, et al.
Published: (2025)

Asynchronous RLHF: Faster and More Efficient Off-Policy RL for Language Models
by: Noukhovitch, Michael, et al.
Published: (2024)

Learning to Compress Prompt in Natural Language Formats
by: Chuang, Yu-Neng, et al.
Published: (2024)

Natural Language Reinforcement Learning
by: Feng, Xidong, et al.
Published: (2024)

ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RL
by: Zhou, Yifei, et al.
Published: (2024)

Research on Optimization of Natural Language Processing Model Based on Multimodal Deep Learning
by: Sun, Dan, et al.
Published: (2024)

Learning to Reason under Off-Policy Guidance
by: Yan, Jianhao, et al.
Published: (2025)

Correlating and Predicting Human Evaluations of Language Models from Natural Language Processing Benchmarks
by: Schaeffer, Rylan, et al.
Published: (2025)

Language Guided Skill Discovery
by: Rho, Seungeun, et al.
Published: (2024)

Self-Play with Adversarial Critic: Provable and Scalable Offline Alignment for Language Models
by: Ji, Xiang, et al.
Published: (2024)

Decoding Diffusion: A Scalable Framework for Unsupervised Analysis of Latent Space Biases and Representations Using Natural Language Prompts
by: Zeng, E. Zhixuan, et al.
Published: (2024)

Evaluating & Reducing Deceptive Dialogue From Language Models with Multi-turn RL
by: Abdulhai, Marwa, et al.
Published: (2025)

Natural Language Reinforcement Learning
by: Feng, Xidong, et al.
Published: (2024)

Safe Reinforcement Learning with Free-form Natural Language Constraints and Pre-Trained Language Models
by: Lou, Xingzhou, et al.
Published: (2024)

SIEVE: Sample-Efficient Parametric Learning from Natural Language
by: Asawa, Parth, et al.
Published: (2026)

Reasoning-Grounded Natural Language Explanations for Language Models
by: Cahlik, Vojtech, et al.
Published: (2025)

LaFFi: Leveraging Hybrid Natural Language Feedback for Fine-tuning Language Models
by: Li, Qianxi, et al.
Published: (2023)

Decentralized Arena: Towards Democratic and Scalable Automatic Evaluation of Language Models
by: Yin, Yanbin, et al.
Published: (2025)

Uncertainty Quantification for In-Context Learning of Large Language Models
by: Ling, Chen, et al.
Published: (2024)

Patch the Distribution Mismatch: RL Rewriting Agent for Stable Off-Policy SFT
by: Wang, Jiacheng, et al.
Published: (2026)