EyeFormer: Predicting Personalized Scanpaths with Transformer-Guided Reinforcement Learning
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917646579007488 |
|---|---|
| author | Jiang, Yue Guo, Zixin Tavakoli, Hamed Rezazadegan Leiva, Luis A. Oulasvirta, Antti |
| author_facet | Jiang, Yue Guo, Zixin Tavakoli, Hamed Rezazadegan Leiva, Luis A. Oulasvirta, Antti |
| contents | From a visual perception perspective, modern graphical user interfaces (GUIs) comprise a complex graphics-rich two-dimensional visuospatial arrangement of text, images, and interactive objects such as buttons and menus. While existing models can accurately predict regions and objects that are likely to attract attention ``on average'', so far there is no scanpath model capable of predicting scanpaths for an individual. To close this gap, we introduce EyeFormer, which leverages a Transformer architecture as a policy network to guide a deep reinforcement learning algorithm that controls gaze locations. Our model has the unique capability of producing personalized predictions when given a few user scanpath samples. It can predict full scanpath information, including fixation positions and duration, across individuals and various stimulus types. Additionally, we demonstrate applications in GUI layout optimization driven by our model. Our software and models will be publicly available. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_10163 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | EyeFormer: Predicting Personalized Scanpaths with Transformer-Guided Reinforcement Learning Jiang, Yue Guo, Zixin Tavakoli, Hamed Rezazadegan Leiva, Luis A. Oulasvirta, Antti Computer Vision and Pattern Recognition Artificial Intelligence Human-Computer Interaction Machine Learning From a visual perception perspective, modern graphical user interfaces (GUIs) comprise a complex graphics-rich two-dimensional visuospatial arrangement of text, images, and interactive objects such as buttons and menus. While existing models can accurately predict regions and objects that are likely to attract attention ``on average'', so far there is no scanpath model capable of predicting scanpaths for an individual. To close this gap, we introduce EyeFormer, which leverages a Transformer architecture as a policy network to guide a deep reinforcement learning algorithm that controls gaze locations. Our model has the unique capability of producing personalized predictions when given a few user scanpath samples. It can predict full scanpath information, including fixation positions and duration, across individuals and various stimulus types. Additionally, we demonstrate applications in GUI layout optimization driven by our model. Our software and models will be publicly available. |
| title | EyeFormer: Predicting Personalized Scanpaths with Transformer-Guided Reinforcement Learning |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Human-Computer Interaction Machine Learning |
| url | https://arxiv.org/abs/2404.10163 |