What We Know About Using Non-Engagement Signals in Content Ranking
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913230890205184 |
|---|---|
| author | Cunningham, Tom Pandey, Sana Sigerson, Leif Stray, Jonathan Allen, Jeff Barrilleaux, Bonnie Iyer, Ravi Milli, Smitha Kothari, Mohit Rezaei, Behnam |
| author_facet | Cunningham, Tom Pandey, Sana Sigerson, Leif Stray, Jonathan Allen, Jeff Barrilleaux, Bonnie Iyer, Ravi Milli, Smitha Kothari, Mohit Rezaei, Behnam |
| contents | Many online platforms predominantly rank items by predicted user engagement. We believe that there is much unrealized potential in including non-engagement signals, which can improve outcomes both for platforms and for society as a whole. Based on a daylong workshop with experts from industry and academia, we formulate a series of propositions and document each as best we can from public evidence, including quantitative results where possible.
There is strong evidence that ranking by predicted engagement is effective in increasing user retention. However retention can be further increased by incorporating other signals, including item "quality" proxies and asking users what they want to see with "item-level" surveys. There is also evidence that "diverse engagement" is an effective quality signal. Ranking changes can alter the prevalence of self-reported experiences of various kinds (e.g. harassment) but seldom have large enough effects on attitude measures like user satisfaction, well-being, polarization etc. to be measured in typical experiments. User controls over ranking often have low usage rates, but when used they do correlate well with quality and item-level surveys. There was no strong evidence on the impact of transparency/explainability on retention. There is reason to believe that generative AI could be used to create better quality signals and enable new kinds of user controls. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_06831 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | What We Know About Using Non-Engagement Signals in Content Ranking Cunningham, Tom Pandey, Sana Sigerson, Leif Stray, Jonathan Allen, Jeff Barrilleaux, Bonnie Iyer, Ravi Milli, Smitha Kothari, Mohit Rezaei, Behnam Social and Information Networks H.3.3; H.4.3 Many online platforms predominantly rank items by predicted user engagement. We believe that there is much unrealized potential in including non-engagement signals, which can improve outcomes both for platforms and for society as a whole. Based on a daylong workshop with experts from industry and academia, we formulate a series of propositions and document each as best we can from public evidence, including quantitative results where possible. There is strong evidence that ranking by predicted engagement is effective in increasing user retention. However retention can be further increased by incorporating other signals, including item "quality" proxies and asking users what they want to see with "item-level" surveys. There is also evidence that "diverse engagement" is an effective quality signal. Ranking changes can alter the prevalence of self-reported experiences of various kinds (e.g. harassment) but seldom have large enough effects on attitude measures like user satisfaction, well-being, polarization etc. to be measured in typical experiments. User controls over ranking often have low usage rates, but when used they do correlate well with quality and item-level surveys. There was no strong evidence on the impact of transparency/explainability on retention. There is reason to believe that generative AI could be used to create better quality signals and enable new kinds of user controls. |
| title | What We Know About Using Non-Engagement Signals in Content Ranking |
| topic | Social and Information Networks H.3.3; H.4.3 |
| url | https://arxiv.org/abs/2402.06831 |