Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2502.09806 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912966492815360 |
|---|---|
| author | Habibi, Mahyar Khanalizadeh, Zahra Ziaeian, Negar |
| author_facet | Habibi, Mahyar Khanalizadeh, Zahra Ziaeian, Negar |
| contents | Online marketplaces frequently run pricing experiments in environments where users choose from a list of items. In these settings, items compete for users' limited attention and demand, creating interference among items within a list: Changing prices for any item can affect the demand for others, biasing estimates from item-level A/B tests. Besides, a key consideration in pricing experiments is preserving platform coherency across prices and item availability. This requirement rules out experimental designs such as user-level A/B tests as they violate platform coherency. We propose Two-Sided Prioritized Ranking (TSPR) to estimate the total average treatment effect of price changes in such settings. TSPR exploits position bias in ranked search results to create variation in treatment exposure without compromising coherency. TSPR randomizes both users and items and reorders ranked lists, prioritizing treated items for one group of users and untreated items for the other. All users see the same items at consistent prices, but differ in exposure to treatment as they pay disproportionate attention across ranks. In semi-synthetic simulations based on Expedia hotel search data, TSPR outperforms baseline coherency-preserving experiment designs by reducing estimation bias and providing sufficient statistical power. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_09806 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Two-Sided Prioritized Ranking: A Coherency-Preserving Design for Marketplace Experiments Habibi, Mahyar Khanalizadeh, Zahra Ziaeian, Negar Econometrics Information Retrieval Social and Information Networks Methodology H.3.3; H.3.5 Online marketplaces frequently run pricing experiments in environments where users choose from a list of items. In these settings, items compete for users' limited attention and demand, creating interference among items within a list: Changing prices for any item can affect the demand for others, biasing estimates from item-level A/B tests. Besides, a key consideration in pricing experiments is preserving platform coherency across prices and item availability. This requirement rules out experimental designs such as user-level A/B tests as they violate platform coherency. We propose Two-Sided Prioritized Ranking (TSPR) to estimate the total average treatment effect of price changes in such settings. TSPR exploits position bias in ranked search results to create variation in treatment exposure without compromising coherency. TSPR randomizes both users and items and reorders ranked lists, prioritizing treated items for one group of users and untreated items for the other. All users see the same items at consistent prices, but differ in exposure to treatment as they pay disproportionate attention across ranks. In semi-synthetic simulations based on Expedia hotel search data, TSPR outperforms baseline coherency-preserving experiment designs by reducing estimation bias and providing sufficient statistical power. |
| title | Two-Sided Prioritized Ranking: A Coherency-Preserving Design for Marketplace Experiments |
| topic | Econometrics Information Retrieval Social and Information Networks Methodology H.3.3; H.3.5 |
| url | https://arxiv.org/abs/2502.09806 |