Tokenization of Gaze Data
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913764260970496 |
|---|---|
| author | Rolff, Tim Karimian, Jurik Hypki, Niklas Schmidt, Susanne Lappe, Markus Steinicke, Frank |
| author_facet | Rolff, Tim Karimian, Jurik Hypki, Niklas Schmidt, Susanne Lappe, Markus Steinicke, Frank |
| contents | A considerable part of the performance of today's large language models (LLM's) and multimodal large language models (MLLM's) depends on their tokenization strategies. While tokenizers are extensively researched for textual and visual input, there is no research on tokenization strategies for gaze data due to its nature. However, a corresponding tokenization strategy would allow using the vision capabilities of pre-trained MLLM's for gaze data, for example, through fine-tuning.
In this paper, we aim to close this research gap by analyzing five different tokenizers for gaze data on three different datasets for the forecasting and generation of gaze data through LLMs (cf.~\cref{fig:teaser}). We evaluate the tokenizers regarding their reconstruction and compression abilities. Further, we train an LLM for each tokenization strategy, measuring its generative and predictive performance. Overall, we found that a quantile tokenizer outperforms all others in predicting the gaze positions and k-means is best when predicting gaze velocities. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_22145 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Tokenization of Gaze Data Rolff, Tim Karimian, Jurik Hypki, Niklas Schmidt, Susanne Lappe, Markus Steinicke, Frank Machine Learning Computation and Language Computer Vision and Pattern Recognition Human-Computer Interaction A considerable part of the performance of today's large language models (LLM's) and multimodal large language models (MLLM's) depends on their tokenization strategies. While tokenizers are extensively researched for textual and visual input, there is no research on tokenization strategies for gaze data due to its nature. However, a corresponding tokenization strategy would allow using the vision capabilities of pre-trained MLLM's for gaze data, for example, through fine-tuning. In this paper, we aim to close this research gap by analyzing five different tokenizers for gaze data on three different datasets for the forecasting and generation of gaze data through LLMs (cf.~\cref{fig:teaser}). We evaluate the tokenizers regarding their reconstruction and compression abilities. Further, we train an LLM for each tokenization strategy, measuring its generative and predictive performance. Overall, we found that a quantile tokenizer outperforms all others in predicting the gaze positions and k-means is best when predicting gaze velocities. |
| title | Tokenization of Gaze Data |
| topic | Machine Learning Computation and Language Computer Vision and Pattern Recognition Human-Computer Interaction |
| url | https://arxiv.org/abs/2503.22145 |