Predicting and Explaining Mobile UI Tappability with Vision Modeling and Saliency Analysis
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2022
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912501447262208 |
|---|---|
| author | Schoop, Eldon Zhou, Xin Li, Gang Chen, Zhourong Hartmann, Björn Li, Yang |
| author_facet | Schoop, Eldon Zhou, Xin Li, Gang Chen, Zhourong Hartmann, Björn Li, Yang |
| contents | We use a deep learning based approach to predict whether a selected element in a mobile UI screenshot will be perceived by users as tappable, based on pixels only instead of view hierarchies required by previous work. To help designers better understand model predictions and to provide more actionable design feedback than predictions alone, we additionally use ML interpretability techniques to help explain the output of our model. We use XRAI to highlight areas in the input screenshot that most strongly influence the tappability prediction for the selected region, and use k-Nearest Neighbors to present the most similar mobile UIs from the dataset with opposing influences on tappability perception. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2204_02448 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Predicting and Explaining Mobile UI Tappability with Vision Modeling and Saliency Analysis Schoop, Eldon Zhou, Xin Li, Gang Chen, Zhourong Hartmann, Björn Li, Yang Human-Computer Interaction Artificial Intelligence Computer Vision and Pattern Recognition Machine Learning We use a deep learning based approach to predict whether a selected element in a mobile UI screenshot will be perceived by users as tappable, based on pixels only instead of view hierarchies required by previous work. To help designers better understand model predictions and to provide more actionable design feedback than predictions alone, we additionally use ML interpretability techniques to help explain the output of our model. We use XRAI to highlight areas in the input screenshot that most strongly influence the tappability prediction for the selected region, and use k-Nearest Neighbors to present the most similar mobile UIs from the dataset with opposing influences on tappability perception. |
| title | Predicting and Explaining Mobile UI Tappability with Vision Modeling and Saliency Analysis |
| topic | Human-Computer Interaction Artificial Intelligence Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2204.02448 |