Long-Term Value of Exploration: Measurements, Findings and Algorithms
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866929255371243520 |
|---|---|
| author | Su, Yi Wang, Xiangyu Le, Elaine Ya Liu, Liang Li, Yuening Lu, Haokai Lipshitz, Benjamin Badam, Sriraj Heldt, Lukasz Bi, Shuchao Chi, Ed Goodrow, Cristos Wu, Su-Lin Baugher, Lexi Chen, Minmin |
| author_facet | Su, Yi Wang, Xiangyu Le, Elaine Ya Liu, Liang Li, Yuening Lu, Haokai Lipshitz, Benjamin Badam, Sriraj Heldt, Lukasz Bi, Shuchao Chi, Ed Goodrow, Cristos Wu, Su-Lin Baugher, Lexi Chen, Minmin |
| contents | Effective exploration is believed to positively influence the long-term user experience on recommendation platforms. Determining its exact benefits, however, has been challenging. Regular A/B tests on exploration often measure neutral or even negative engagement metrics while failing to capture its long-term benefits. We here introduce new experiment designs to formally quantify the long-term value of exploration by examining its effects on content corpus, and connecting content corpus growth to the long-term user experience from real-world experiments. Once established the values of exploration, we investigate the Neural Linear Bandit algorithm as a general framework to introduce exploration into any deep learning based ranking systems. We conduct live experiments on one of the largest short-form video recommendation platforms that serves billions of users to validate the new experiment designs, quantify the long-term values of exploration, and to verify the effectiveness of the adopted neural linear bandit algorithm for exploration. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_07764 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Long-Term Value of Exploration: Measurements, Findings and Algorithms Su, Yi Wang, Xiangyu Le, Elaine Ya Liu, Liang Li, Yuening Lu, Haokai Lipshitz, Benjamin Badam, Sriraj Heldt, Lukasz Bi, Shuchao Chi, Ed Goodrow, Cristos Wu, Su-Lin Baugher, Lexi Chen, Minmin Information Retrieval Effective exploration is believed to positively influence the long-term user experience on recommendation platforms. Determining its exact benefits, however, has been challenging. Regular A/B tests on exploration often measure neutral or even negative engagement metrics while failing to capture its long-term benefits. We here introduce new experiment designs to formally quantify the long-term value of exploration by examining its effects on content corpus, and connecting content corpus growth to the long-term user experience from real-world experiments. Once established the values of exploration, we investigate the Neural Linear Bandit algorithm as a general framework to introduce exploration into any deep learning based ranking systems. We conduct live experiments on one of the largest short-form video recommendation platforms that serves billions of users to validate the new experiment designs, quantify the long-term values of exploration, and to verify the effectiveness of the adopted neural linear bandit algorithm for exploration. |
| title | Long-Term Value of Exploration: Measurements, Findings and Algorithms |
| topic | Information Retrieval |
| url | https://arxiv.org/abs/2305.07764 |