Cover Image

Digi-Q: Learning Q-Value Functions for Training Device-Control Agents

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Bai, Hao, Zhou, Yifei, Li, Li Erran, Levine, Sergey, Kumar, Aviral
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learning
by: Bai, Hao, et al.
Published: (2024)

ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RL
by: Zhou, Yifei, et al.
Published: (2024)

Flow Q-Learning
by: Park, Seohong, et al.
Published: (2025)

Is Value Learning Really the Main Bottleneck in Offline RL?
by: Park, Seohong, et al.
Published: (2024)

Q-learning with Adjoint Matching
by: Li, Qiyang, et al.
Published: (2026)

Proposer-Agent-Evaluator(PAE): Autonomous Skill Discovery For Foundation Model Internet Agents
by: Zhou, Yifei, et al.
Published: (2024)

Decoupled Q-Chunking
by: Li, Qiyang, et al.
Published: (2025)

Steering Your Generalists: Improving Robotic Foundation Models via Value Guidance
by: Nakamoto, Mitsuhiko, et al.
Published: (2024)

Efficient Online Reinforcement Learning Fine-Tuning Need Not Retain Offline Data
by: Zhou, Zhiyuan, et al.
Published: (2024)

Strategically Conservative Q-Learning
by: Shimizu, Yutaka, et al.
Published: (2024)

Q-SFT: Q-Learning for Language Models via Supervised Fine-Tuning
by: Hong, Joey, et al.
Published: (2024)

Compute-Optimal Scaling for Value-Based Deep RL
by: Fu, Preston, et al.
Published: (2025)

Value-Based Deep RL Scales Predictably
by: Rybkin, Oleh, et al.
Published: (2025)

SWEET-RL: Training Multi-Turn LLM Agents on Collaborative Reasoning Tasks
by: Zhou, Yifei, et al.
Published: (2025)

Vision-Language Models Provide Promptable Representations for Reinforcement Learning
by: Chen, William, et al.
Published: (2024)

Scaling Test-Time Compute Without Verification or RL is Suboptimal
by: Setlur, Amrith, et al.
Published: (2025)

DigiData: Training and Evaluating General-Purpose Mobile Control Agents
by: Sun, Yuxuan, et al.
Published: (2025)

Enhancing Q-Value Updates in Deep Q-Learning via Successor-State Prediction
by: Zu, Lipeng, et al.
Published: (2025)

Unfamiliar Finetuning Examples Control How Language Models Hallucinate
by: Kang, Katie, et al.
Published: (2024)

Stop Regressing: Training Value Functions via Classification for Scalable Deep RL
by: Farebrother, Jesse, et al.
Published: (2024)

WebGym: Scaling Training Environments for Visual Web Agents with Realistic Tasks
by: Bai, Hao, et al.
Published: (2026)

ViVa: Video-Trained Value Functions for Guiding Online RL from Diverse Data
by: Dashora, Nitish, et al.
Published: (2025)

RLAC: Reinforcement Learning with Adversarial Critic for Free-Form Generation Tasks
by: Wu, Mian, et al.
Published: (2025)

What Do Learning Dynamics Reveal About Generalization in LLM Reasoning?
by: Kang, Katie, et al.
Published: (2024)

RiskQ: Risk-sensitive Multi-Agent Reinforcement Learning Value Factorization
by: Shen, Siqi, et al.
Published: (2023)

floq: Training Critics via Flow-Matching for Scaling Compute in Value-Based RL
by: Agrawalla, Bhavya, et al.
Published: (2025)

Quantile Q-Learning: Revisiting Offline Extreme Q-Learning with Quantile Regression
by: Gao, Xinming, et al.
Published: (2025)

Improve Value Estimation of Q Function and Reshape Reward with Monte Carlo Tree Search
by: Li, Jiamian
Published: (2024)

Bigger, Regularized, Categorical: High-Capacity Value Functions are Efficient Multi-Task Learners
by: Nauman, Michal, et al.
Published: (2025)

Residual Q-Learning: Offline and Online Policy Customization without Value
by: Li, Chenran, et al.
Published: (2023)

Horizon Reduction Makes RL Scalable
by: Park, Seohong, et al.
Published: (2025)

Towards Adapting Reinforcement Learning Agents to New Tasks: Insights from Q-Values
by: Ramaswamy, Ashwin, et al.
Published: (2024)

Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuning
by: Nakamoto, Mitsuhiko, et al.
Published: (2023)

Reinforcement Learning with Action Chunking
by: Li, Qiyang, et al.
Published: (2025)

Scalable In-Context Q-Learning
by: Liu, Jinmei, et al.
Published: (2025)

Visual Pre-Training on Unlabeled Images using Reinforcement Learning
by: Ghosh, Dibya, et al.
Published: (2025)

QSIM: Mitigating Overestimation in Multi-Agent Reinforcement Learning via Action Similarity Weighted Q-Learning
by: Li, Yuanjun, et al.
Published: (2026)

Peng's Q($λ$) for Conservative Value Estimation in Offline Reinforcement Learning
by: Kim, Byeongchan, et al.
Published: (2026)

Multi-Agent Inverse Q-Learning from Demonstrations
by: Haynam, Nathaniel, et al.
Published: (2025)

VerifierQ: Enhancing LLM Test Time Compute with Q-Learning-based Verifiers
by: Qi, Jianing, et al.
Published: (2024)