Intersectional Fairness in Reinforcement Learning with Large State and Constraint Spaces
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913693809246208 |
|---|---|
| author | Eaton, Eric Hussing, Marcel Kearns, Michael Roth, Aaron Sengupta, Sikata Bela Sorrell, Jessica |
| author_facet | Eaton, Eric Hussing, Marcel Kearns, Michael Roth, Aaron Sengupta, Sikata Bela Sorrell, Jessica |
| contents | In traditional reinforcement learning (RL), the learner aims to solve a single objective optimization problem: find the policy that maximizes expected reward. However, in many real-world settings, it is important to optimize over multiple objectives simultaneously. For example, when we are interested in fairness, states might have feature annotations corresponding to multiple (intersecting) demographic groups to whom reward accrues, and our goal might be to maximize the reward of the group receiving the minimal reward. In this work, we consider a multi-objective optimization problem in which each objective is defined by a state-based reweighting of a single scalar reward function. This generalizes the problem of maximizing the reward of the minimum reward group. We provide oracle-efficient algorithms to solve these multi-objective RL problems even when the number of objectives is exponentially large-for tabular MDPs, as well as for large MDPs when the group functions have additional structure. Finally, we experimentally validate our theoretical results and demonstrate applications on a preferential attachment graph MDP. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_11828 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Intersectional Fairness in Reinforcement Learning with Large State and Constraint Spaces Eaton, Eric Hussing, Marcel Kearns, Michael Roth, Aaron Sengupta, Sikata Bela Sorrell, Jessica Machine Learning Computer Science and Game Theory In traditional reinforcement learning (RL), the learner aims to solve a single objective optimization problem: find the policy that maximizes expected reward. However, in many real-world settings, it is important to optimize over multiple objectives simultaneously. For example, when we are interested in fairness, states might have feature annotations corresponding to multiple (intersecting) demographic groups to whom reward accrues, and our goal might be to maximize the reward of the group receiving the minimal reward. In this work, we consider a multi-objective optimization problem in which each objective is defined by a state-based reweighting of a single scalar reward function. This generalizes the problem of maximizing the reward of the minimum reward group. We provide oracle-efficient algorithms to solve these multi-objective RL problems even when the number of objectives is exponentially large-for tabular MDPs, as well as for large MDPs when the group functions have additional structure. Finally, we experimentally validate our theoretical results and demonstrate applications on a preferential attachment graph MDP. |
| title | Intersectional Fairness in Reinforcement Learning with Large State and Constraint Spaces |
| topic | Machine Learning Computer Science and Game Theory |
| url | https://arxiv.org/abs/2502.11828 |