Towards Reliable Social A/B Testing: Spillover-Contained Clustering with Robust Post-Experiment Analysis
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915786189176832 |
|---|---|
| author | Min, Xu Yang, Zhaoxu Tan, Kaixuan Yan, Juan Xiong, Xunbin Zhu, Zihao Zhu, Kaiyu Cui, Fenglin Yang, Yang Yang, Sihua Bu, Jianhui |
| author_facet | Min, Xu Yang, Zhaoxu Tan, Kaixuan Yan, Juan Xiong, Xunbin Zhu, Zihao Zhu, Kaiyu Cui, Fenglin Yang, Yang Yang, Sihua Bu, Jianhui |
| contents | A/B testing is the foundation of decision-making in online platforms, yet social products often suffer from network interference: user interactions cause treatment effects to spill over into the control group. Such spillovers bias causal estimates and undermine experimental conclusions. Existing approaches face key limitations: user-level randomization ignores network structure, while cluster-based methods often rely on general-purpose clustering that is not tailored for spillover containment and has difficulty balancing unbiasedness and statistical power at scale. We propose a spillover-contained experimentation framework with two stages. In the pre-experiment stage, we build social interaction graphs and introduce a Balanced Louvain algorithm that produces stable, size-balanced clusters while minimizing cross-cluster edges, enabling reliable cluster-based randomization. In the post-experiment stage, we develop a tailored CUPAC estimator that leverages pre-experiment behavioral covariates to reduce the variance induced by cluster-level assignment, thereby improving statistical power. Together, these components provide both structural spillover containment and robust statistical inference. We validate our approach through large-scale social sharing experiments on Kuaishou, a platform serving hundreds of millions of users. Results show that our method substantially reduces spillover and yields more accurate assessments of social strategies than traditional user-level designs, establishing a reliable and scalable framework for networked A/B testing. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_08569 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Towards Reliable Social A/B Testing: Spillover-Contained Clustering with Robust Post-Experiment Analysis Min, Xu Yang, Zhaoxu Tan, Kaixuan Yan, Juan Xiong, Xunbin Zhu, Zihao Zhu, Kaiyu Cui, Fenglin Yang, Yang Yang, Sihua Bu, Jianhui Social and Information Networks Information Retrieval A/B testing is the foundation of decision-making in online platforms, yet social products often suffer from network interference: user interactions cause treatment effects to spill over into the control group. Such spillovers bias causal estimates and undermine experimental conclusions. Existing approaches face key limitations: user-level randomization ignores network structure, while cluster-based methods often rely on general-purpose clustering that is not tailored for spillover containment and has difficulty balancing unbiasedness and statistical power at scale. We propose a spillover-contained experimentation framework with two stages. In the pre-experiment stage, we build social interaction graphs and introduce a Balanced Louvain algorithm that produces stable, size-balanced clusters while minimizing cross-cluster edges, enabling reliable cluster-based randomization. In the post-experiment stage, we develop a tailored CUPAC estimator that leverages pre-experiment behavioral covariates to reduce the variance induced by cluster-level assignment, thereby improving statistical power. Together, these components provide both structural spillover containment and robust statistical inference. We validate our approach through large-scale social sharing experiments on Kuaishou, a platform serving hundreds of millions of users. Results show that our method substantially reduces spillover and yields more accurate assessments of social strategies than traditional user-level designs, establishing a reliable and scalable framework for networked A/B testing. |
| title | Towards Reliable Social A/B Testing: Spillover-Contained Clustering with Robust Post-Experiment Analysis |
| topic | Social and Information Networks Information Retrieval |
| url | https://arxiv.org/abs/2602.08569 |