Cascade! Human in the loop shortcomings can increase the risk of failures in recommender systems
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914055716864000 |
|---|---|
| author | Kennedy, Wm. Matthew Shukla, Nishanshi Patlak, Cigdem Chambers, Blake Skeadas, Theodora Tuesday Owadara, Kingsley Dhanotiya, Aayush |
| author_facet | Kennedy, Wm. Matthew Shukla, Nishanshi Patlak, Cigdem Chambers, Blake Skeadas, Theodora Tuesday Owadara, Kingsley Dhanotiya, Aayush |
| contents | Recommender systems are among the most commonly deployed systems today. Systems design approaches to AI-powered recommender systems have done well to urge recommender system developers to follow more intentional data collection, curation, and management procedures. So too has the "human-in-the-loop" paradigm been widely adopted, primarily to address the issue of accountability. However, in this paper, we take the position that human oversight in recommender system design also entails novel risks that have yet to be fully described. These risks are "codetermined" by the information context in which such systems are often deployed. Furthermore, new knowledge of the shortcomings of "human-in-the-loop" practices to deliver meaningful oversight of other AI systems suggest that they may also be inadequate for achieving socially responsible recommendations. We review how the limitations of human oversight may increase the chances of a specific kind of failure: a "cascade" or "compound" failure. We then briefly explore how the unique dynamics of three common deployment contexts can make humans in the loop more likely to fail in their oversight duties. We then conclude with two recommendations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_20099 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Cascade! Human in the loop shortcomings can increase the risk of failures in recommender systems Kennedy, Wm. Matthew Shukla, Nishanshi Patlak, Cigdem Chambers, Blake Skeadas, Theodora Tuesday Owadara, Kingsley Dhanotiya, Aayush Information Retrieval Computers and Society Recommender systems are among the most commonly deployed systems today. Systems design approaches to AI-powered recommender systems have done well to urge recommender system developers to follow more intentional data collection, curation, and management procedures. So too has the "human-in-the-loop" paradigm been widely adopted, primarily to address the issue of accountability. However, in this paper, we take the position that human oversight in recommender system design also entails novel risks that have yet to be fully described. These risks are "codetermined" by the information context in which such systems are often deployed. Furthermore, new knowledge of the shortcomings of "human-in-the-loop" practices to deliver meaningful oversight of other AI systems suggest that they may also be inadequate for achieving socially responsible recommendations. We review how the limitations of human oversight may increase the chances of a specific kind of failure: a "cascade" or "compound" failure. We then briefly explore how the unique dynamics of three common deployment contexts can make humans in the loop more likely to fail in their oversight duties. We then conclude with two recommendations. |
| title | Cascade! Human in the loop shortcomings can increase the risk of failures in recommender systems |
| topic | Information Retrieval Computers and Society |
| url | https://arxiv.org/abs/2509.20099 |