Cover Image

The Role of Inherent Bellman Error in Offline Reinforcement Learning with Linear Function Approximation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Golowich, Noah, Moitra, Ankur
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Linear Bellman Completeness Suffices for Efficient Online Reinforcement Learning with Few Actions
by: Golowich, Noah, et al.
Published: (2024)

Edit Distance Robust Watermarks via Indexing Pseudorandom Codes
by: Golowich, Noah, et al.
Published: (2024)

Subliminal Effects in Your Data: A General Mechanism via Log-Linearity
by: Aden-Ali, Ishaq, et al.
Published: (2026)

Exploration is Harder than Prediction: Cryptographically Separating Reinforcement Learning from Supervised Learning
by: Golowich, Noah, et al.
Published: (2024)

Is Efficient PAC Learning Possible with an Oracle That Responds 'Yes' or 'No'?
by: Daskalakis, Constantinos, et al.
Published: (2024)

MICRO: Model-Based Offline Reinforcement Learning with a Conservative Bellman Operator
by: Liu, Xiao-Yin, et al.
Published: (2023)

Bellman Unbiasedness: Toward Provably Efficient Distributional Reinforcement Learning with General Value Function Approximation
by: Cho, Taehyun, et al.
Published: (2024)

Symmetric Q-learning: Reducing Skewness of Bellman Error in Online Reinforcement Learning
by: Omura, Motoki, et al.
Published: (2024)

Bellman Error Centering
by: Chen, Xingguo, et al.
Published: (2025)

A Generalized Projected Bellman Error for Off-policy Value Estimation in Reinforcement Learning
by: Patterson, Andrew, et al.
Published: (2021)

Model Stealing for Any Low-Rank Language Model
by: Liu, Allen, et al.
Published: (2024)

Misspecified $Q$-Learning with Sparse Linear Function Approximation: Tight Bounds on Approximation Error
by: Du, Ally Yalei, et al.
Published: (2024)

Provably Learning from Modern Language Models via Low Logit Rank
by: Golowich, Noah, et al.
Published: (2025)

Theoretical Barriers in Bellman-Based Reinforcement Learning
by: Pinon, Brieuc, et al.
Published: (2025)

Sequences of Logits Reveal the Low Rank Structure of Language Models
by: Golowich, Noah, et al.
Published: (2025)

The Role of Sparsity for Length Generalization in Transformers
by: Golowich, Noah, et al.
Published: (2025)

Gradual Transition from Bellman Optimality Operator to Bellman Operator in Online Reinforcement Learning
by: Omura, Motoki, et al.
Published: (2025)

On the Complexity of Offline Reinforcement Learning with $Q^\star$-Approximation and Partial Coverage
by: Liu, Haolin, et al.
Published: (2026)

Offline Reinforcement Learning: Role of State Aggregation and Trajectory Data
by: Jia, Zeyu, et al.
Published: (2024)

Offline Trajectory Optimization for Offline Reinforcement Learning
by: Zhao, Ziqi, et al.
Published: (2024)

VIPO: Value Function Inconsistency Penalized Offline Reinforcement Learning
by: Chen, Xuyang, et al.
Published: (2025)

The Hidden Game Problem
by: Buzaglo, Gon, et al.
Published: (2025)

Near-Optimal Learning and Planning in Separated Latent MDPs
by: Chen, Fan, et al.
Published: (2024)

Robust Offline Reinforcement Learning with Linearly Structured f-Divergence Regularization
by: Tang, Cheng, et al.
Published: (2024)

Is Value Functions Estimation with Classification Plug-and-play for Offline Reinforcement Learning?
by: Tarasov, Denis, et al.
Published: (2024)

RAMAC: Multimodal Risk-Aware Offline Reinforcement Learning and the Role of Behavior Regularization
by: Fukazawa, Kai, et al.
Published: (2025)

Iterated $Q$-Network: Beyond One-Step Bellman Updates in Deep Reinforcement Learning
by: Vincent, Théo, et al.
Published: (2024)

Preference Elicitation for Offline Reinforcement Learning
by: Pace, Alizée, et al.
Published: (2024)

Simple Ingredients for Offline Reinforcement Learning
by: Cetin, Edoardo, et al.
Published: (2024)

State-Constrained Offline Reinforcement Learning
by: Hepburn, Charles A., et al.
Published: (2024)

Dataset Distillation for Offline Reinforcement Learning
by: Light, Jonathan, et al.
Published: (2024)

Offline Reinforcement Learning with Imputed Rewards
by: Romeo, Carlo, et al.
Published: (2024)

Offline Reinforcement Learning with Imbalanced Datasets
by: Jiang, Li, et al.
Published: (2023)

The Generalization Gap in Offline Reinforcement Learning
by: Mediratta, Ishita, et al.
Published: (2023)

Diverse Randomized Value Functions: A Provably Pessimistic Approach for Offline Reinforcement Learning
by: Yu, Xudong, et al.
Published: (2024)

Path-Coupled Bellman Flows for Distributional Reinforcement Learning
by: Xu, Boyang, et al.
Published: (2026)

OffSim: Offline Simulator for Model-based Offline Inverse Reinforcement Learning
by: Ahn, Woo-Jin, et al.
Published: (2025)

In-Context Compositional Q-Learning for Offline Reinforcement Learning
by: Xu, Qiushui, et al.
Published: (2025)

Imagination-Limited Q-Learning for Offline Reinforcement Learning
by: Liu, Wenhui, et al.
Published: (2025)

Mildly Conservative Q-Learning for Offline Reinforcement Learning
by: Lyu, Jiafei, et al.
Published: (2022)