Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2509.20001 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909804349358080 |
|---|---|
| author | Naderi, Babak Cutler, Ross |
| author_facet | Naderi, Babak Cutler, Ross |
| contents | Subjective video quality assessment (VQA) is the gold standard for measuring end-user experience across communication, streaming, and UGC pipelines. Beyond high-validity lab studies, crowdsourcing offers accurate, reliable, faster, and cheaper evaluation-but suffers from unreliable submissions by workers who ignore instructions or game rewards. Recent tests reveal sophisticated exploits of video metadata and rising use of remote-desktop (RD) connections, both of which bias results. We propose objective and subjective detectors for RD users and compare two mainstream crowdsourcing platforms on their susceptibility and mitigation under realistic test conditions and task designs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_20001 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Ensuring Reliable Participation in Subjective Video Quality Tests Across Platforms Naderi, Babak Cutler, Ross Image and Video Processing Computer Vision and Pattern Recognition Multimedia Subjective video quality assessment (VQA) is the gold standard for measuring end-user experience across communication, streaming, and UGC pipelines. Beyond high-validity lab studies, crowdsourcing offers accurate, reliable, faster, and cheaper evaluation-but suffers from unreliable submissions by workers who ignore instructions or game rewards. Recent tests reveal sophisticated exploits of video metadata and rising use of remote-desktop (RD) connections, both of which bias results. We propose objective and subjective detectors for RD users and compare two mainstream crowdsourcing platforms on their susceptibility and mitigation under realistic test conditions and task designs. |
| title | Ensuring Reliable Participation in Subjective Video Quality Tests Across Platforms |
| topic | Image and Video Processing Computer Vision and Pattern Recognition Multimedia |
| url | https://arxiv.org/abs/2509.20001 |