Cover Image

Demystifying the Paradox of Importance Sampling with an Estimated History-Dependent Behavior Policy in Off-Policy Evaluation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhou, Hongyi, Hanna, Josiah P., Zhu, Jin, Yang, Ying, Shi, Chengchun
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Demystifying Group Relative Policy Optimization: Its Policy Gradient is a U-Statistic
by: Zhou, Hongyi, et al.
Published: (2026)

Quotient DAGs for Off-Policy Evaluation:Forward-Flow Importance Sampling and Exact Slate Propensities
by: Xie, Ziwen, et al.
Published: (2026)

Learn-to-Distance: Distance Learning for Detecting LLM-Generated Text
by: Zhou, Hongyi, et al.
Published: (2026)

Robust Reinforcement Learning from Human Feedback for Large Language Models Fine-Tuning
by: Ye, Kai, et al.
Published: (2025)

Off-OAB: Off-Policy Policy Gradient Method with Optimal Action-Dependent Baseline
by: Meng, Wenjia, et al.
Published: (2024)

Kernel Metric Learning for In-Sample Off-Policy Evaluation of Deterministic RL Policies
by: Lee, Haanvid, et al.
Published: (2024)

GIPO: Gaussian Importance Sampling Policy Optimization
by: Lu, Chengxuan, et al.
Published: (2026)

Statistical Tractability of Off-policy Evaluation of History-dependent Policies in POMDPs
by: Zhang, Yuheng, et al.
Published: (2025)

Causal Deepsets for Off-policy Evaluation under Spatial or Spatio-temporal Interferences
by: Dai, Runpeng, et al.
Published: (2024)

ESPO: Entropy Importance Sampling Policy Optimization
by: Sheng, Yuepeng, et al.
Published: (2025)

Clustering Context in Off-Policy Evaluation
by: Guzman-Olivares, Daniel, et al.
Published: (2025)

Concept-driven Off Policy Evaluation
by: Majumdar, Ritam, et al.
Published: (2024)

Search-Based Adversarial Estimates for Improving Sample Efficiency in Off-Policy Reinforcement Learning
by: Malato, Federico, et al.
Published: (2025)

Beyond Importance Sampling: Rejection-Gated Policy Optimization
by: Sun, Ziwu, et al.
Published: (2026)

Doubly Robust Alignment for Large Language Models
by: Xu, Erhan, et al.
Published: (2025)

Rethinking Importance Sampling in LLM Policy Optimization: A Cumulative Token Perspective
by: Zhang, Yuheng, et al.
Published: (2026)

Low Variance Off-policy Evaluation with State-based Importance Sampling
by: Bossens, David M., et al.
Published: (2022)

Adaptive Exploration for Data-Efficient General Value Function Evaluations
by: Jain, Arushi, et al.
Published: (2024)

Learning Action Embeddings for Off-Policy Evaluation
by: Cief, Matej, et al.
Published: (2023)

On-Policy Policy Gradient Reinforcement Learning Without On-Policy Sampling
by: Corrado, Nicholas E., et al.
Published: (2023)

Relative Importance Sampling for off-Policy Actor-Critic in Deep Reinforcement Learning
by: Humayoo, Mahammad, et al.
Published: (2018)

Data Poisoning Attacks on Off-Policy Policy Evaluation Methods
by: Lobo, Elita, et al.
Published: (2024)

TIP: Token Importance in On-Policy Distillation
by: Xu, Yuanda, et al.
Published: (2026)

AdaDetectGPT: Adaptive Detection of LLM-Generated Text with Statistical Guarantees
by: Zhou, Hongyi, et al.
Published: (2025)

Behaviour Policy Optimization: Provably Lower Variance Return Estimates for Off-Policy Reinforcement Learning
by: Goodall, Alexander W., et al.
Published: (2025)

Automated Off-Policy Estimator Selection via Supervised Learning
by: Felicioni, Nicolò, et al.
Published: (2024)

Group-Relative REINFORCE Is Secretly an Off-Policy Algorithm: Demystifying Some Myths About GRPO and Its Friends
by: Yao, Chaorui, et al.
Published: (2025)

Stabilizing Off-Policy Training for Long-Horizon LLM Agent via Turn-Level Importance Sampling and Clipping-Triggered Normalization
by: Li, Chenliang, et al.
Published: (2025)

Effective Off-Policy Evaluation and Learning in Contextual Combinatorial Bandits
by: Shimizu, Tatsuhiro, et al.
Published: (2024)

Off-Policy Evaluation and Learning for the Future under Non-Stationarity
by: Shimizu, Tatsuhiro, et al.
Published: (2025)

A Unifying View of Coverage in Linear Off-Policy Evaluation
by: Amortila, Philip, et al.
Published: (2026)

Bayesian Off-Policy Evaluation and Learning for Large Action Spaces
by: Aouali, Imad, et al.
Published: (2024)

Unified PAC-Bayesian Study of Pessimism for Offline Policy Learning with Regularized Importance Sampling
by: Aouali, Imad, et al.
Published: (2024)

SOPE: Stabilizing Off-Policy Evaluation for Online RL with Prior Data
by: Romeo, Carlo, et al.
Published: (2026)

Balancing Immediate Revenue and Future Off-Policy Evaluation in Coupon Allocation
by: Nishimura, Naoki, et al.
Published: (2024)

Efficient and Sharp Off-Policy Evaluation in Robust Markov Decision Processes
by: Bennett, Andrew, et al.
Published: (2024)

Towards Assessing and Benchmarking Risk-Return Tradeoff of Off-Policy Evaluation
by: Kiyohara, Haruka, et al.
Published: (2023)

Conformal Off-Policy Evaluation in Markov Decision Processes
by: Foffano, Daniele, et al.
Published: (2023)

Off-Policy Evaluation and Learning for Survival Outcomes under Censoring
by: Kubota, Kohsuke, et al.
Published: (2026)

Towards Off-Policy Reinforcement Learning for Ranking Policies with Human Feedback
by: Xiao, Teng, et al.
Published: (2024)