AgentA/B: Automated and Scalable Web A/BTesting with Interactive LLM Agents
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915851393826816 |
|---|---|
| author | Lu, Yuxuan Hsu, Ting-Yao Gu, Hansu Cui, Limeng Xie, Yaochen Headden, William Yao, Bingsheng Veeragouni, Akash Liu, Jiapeng Nag, Sreyashi Wang, Jessie Wang, Dakuo |
| author_facet | Lu, Yuxuan Hsu, Ting-Yao Gu, Hansu Cui, Limeng Xie, Yaochen Headden, William Yao, Bingsheng Veeragouni, Akash Liu, Jiapeng Nag, Sreyashi Wang, Jessie Wang, Dakuo |
| contents | A/B testing experiment is a widely adopted method for evaluating UI/UX design decisions in modern web applications. Yet, traditional A/B testing remains constrained by its dependence on the large-scale and live traffic of human participants, and the long time of waiting for the testing result. Through formative interviews with six experienced industry practitioners, we identified critical bottlenecks in current A/B testing workflows. In response, we present AgentA/B, a novel system that leverages Large Language Model-based autonomous agents (LLM Agents) to automatically simulate user interaction behaviors with real webpages. AgentA/B enables scalable deployment of LLM agents with diverse personas, each capable of navigating the dynamic webpage and interactively executing multi-step interactions like search, clicking, filtering, and purchasing. In a demonstrative controlled experiment, we employ AgentA/B to simulate a between-subject A/B testing with 1,000 LLM agents Amazon.com, and compare agent behaviors with real human shopping behaviors at a scale. Our findings suggest AgentA/B can emulate human-like behavior patterns. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_09723 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | AgentA/B: Automated and Scalable Web A/BTesting with Interactive LLM Agents Lu, Yuxuan Hsu, Ting-Yao Gu, Hansu Cui, Limeng Xie, Yaochen Headden, William Yao, Bingsheng Veeragouni, Akash Liu, Jiapeng Nag, Sreyashi Wang, Jessie Wang, Dakuo Human-Computer Interaction Computation and Language A/B testing experiment is a widely adopted method for evaluating UI/UX design decisions in modern web applications. Yet, traditional A/B testing remains constrained by its dependence on the large-scale and live traffic of human participants, and the long time of waiting for the testing result. Through formative interviews with six experienced industry practitioners, we identified critical bottlenecks in current A/B testing workflows. In response, we present AgentA/B, a novel system that leverages Large Language Model-based autonomous agents (LLM Agents) to automatically simulate user interaction behaviors with real webpages. AgentA/B enables scalable deployment of LLM agents with diverse personas, each capable of navigating the dynamic webpage and interactively executing multi-step interactions like search, clicking, filtering, and purchasing. In a demonstrative controlled experiment, we employ AgentA/B to simulate a between-subject A/B testing with 1,000 LLM agents Amazon.com, and compare agent behaviors with real human shopping behaviors at a scale. Our findings suggest AgentA/B can emulate human-like behavior patterns. |
| title | AgentA/B: Automated and Scalable Web A/BTesting with Interactive LLM Agents |
| topic | Human-Computer Interaction Computation and Language |
| url | https://arxiv.org/abs/2504.09723 |