Effects of algorithmic flagging on fairness: quasi-experimental evidence from Wikipedia
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2020
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914231168794624 |
|---|---|
| author | TeBlunthuis, Nathan Hill, Benjamin Mako Halfaker, Aaron |
| author_facet | TeBlunthuis, Nathan Hill, Benjamin Mako Halfaker, Aaron |
| contents | Online community moderators often rely on social signals such as whether or not a user has an account or a profile page as clues that users may cause problems. Reliance on these clues can lead to "overprofiling'' bias when moderators focus on these signals but overlook the misbehavior of others. We propose that algorithmic flagging systems deployed to improve the efficiency of moderation work can also make moderation actions more fair to these users by reducing reliance on social signals and making norm violations by everyone else more visible. We analyze moderator behavior in Wikipedia as mediated by RCFilters, a system which displays social signals and algorithmic flags, and estimate the causal effect of being flagged on moderator actions. We show that algorithmically flagged edits are reverted more often, especially those by established editors with positive social signals, and that flagging decreases the likelihood that moderation actions will be undone. Our results suggest that algorithmic flagging systems can lead to increased fairness in some contexts but that the relationship is complex and contingent. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2006_03121 |
| institution | arXiv |
| publishDate | 2020 |
| record_format | arxiv |
| spellingShingle | Effects of algorithmic flagging on fairness: quasi-experimental evidence from Wikipedia TeBlunthuis, Nathan Hill, Benjamin Mako Halfaker, Aaron Computers and Society Human-Computer Interaction Machine Learning Social and Information Networks K.4.3 Online community moderators often rely on social signals such as whether or not a user has an account or a profile page as clues that users may cause problems. Reliance on these clues can lead to "overprofiling'' bias when moderators focus on these signals but overlook the misbehavior of others. We propose that algorithmic flagging systems deployed to improve the efficiency of moderation work can also make moderation actions more fair to these users by reducing reliance on social signals and making norm violations by everyone else more visible. We analyze moderator behavior in Wikipedia as mediated by RCFilters, a system which displays social signals and algorithmic flags, and estimate the causal effect of being flagged on moderator actions. We show that algorithmically flagged edits are reverted more often, especially those by established editors with positive social signals, and that flagging decreases the likelihood that moderation actions will be undone. Our results suggest that algorithmic flagging systems can lead to increased fairness in some contexts but that the relationship is complex and contingent. |
| title | Effects of algorithmic flagging on fairness: quasi-experimental evidence from Wikipedia |
| topic | Computers and Society Human-Computer Interaction Machine Learning Social and Information Networks K.4.3 |
| url | https://arxiv.org/abs/2006.03121 |