Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2304.06828 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912993755791360 |
|---|---|
| author | Gordon, Brett R. Moakler, Robert Zettelmeyer, Florian |
| author_facet | Gordon, Brett R. Moakler, Robert Zettelmeyer, Florian |
| contents | Randomized controlled trials (RCTs) provide the most credible estimates of advertising incrementality but are difficult to scale. We propose Predicted Incrementality by Experimentation (PIE), which reframes ad measurement as a campaign-level prediction problem. PIE uses a sample of RCTs to learn a mapping from campaign features to causal effects, then applies it to campaigns not run as RCTs. Because the RCTs identify the causal effects, PIE can incorporate post-determined features -- campaign-level aggregates such as test-group outcomes, exposure rates, and last-click conversions, computed after campaign completion. These metrics reflect the consumer behaviors that generate treatment effects, so they carry predictive information about incrementality even though they would be invalid controls in a causal model. Using 2,226 Meta ad experiments, PIE achieves an out-of-sample $R^2 = 0.88$ for incremental conversions per dollar, compared to $R^2 = 0.19$ for industry-standard 7-day last-click attribution. In a decision-making framework, PIE disagrees with RCT-based decisions in only 8-12% of campaigns, compared to 12-20% for last-click attribution. We conclude that PIE can help scale causal measurement from a limited number of RCTs to a large set of non-experimental campaigns. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2304_06828 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Predicted Incrementality by Experimentation (PIE) for Ad Measurement Gordon, Brett R. Moakler, Robert Zettelmeyer, Florian Econometrics Randomized controlled trials (RCTs) provide the most credible estimates of advertising incrementality but are difficult to scale. We propose Predicted Incrementality by Experimentation (PIE), which reframes ad measurement as a campaign-level prediction problem. PIE uses a sample of RCTs to learn a mapping from campaign features to causal effects, then applies it to campaigns not run as RCTs. Because the RCTs identify the causal effects, PIE can incorporate post-determined features -- campaign-level aggregates such as test-group outcomes, exposure rates, and last-click conversions, computed after campaign completion. These metrics reflect the consumer behaviors that generate treatment effects, so they carry predictive information about incrementality even though they would be invalid controls in a causal model. Using 2,226 Meta ad experiments, PIE achieves an out-of-sample $R^2 = 0.88$ for incremental conversions per dollar, compared to $R^2 = 0.19$ for industry-standard 7-day last-click attribution. In a decision-making framework, PIE disagrees with RCT-based decisions in only 8-12% of campaigns, compared to 12-20% for last-click attribution. We conclude that PIE can help scale causal measurement from a limited number of RCTs to a large set of non-experimental campaigns. |
| title | Predicted Incrementality by Experimentation (PIE) for Ad Measurement |
| topic | Econometrics |
| url | https://arxiv.org/abs/2304.06828 |