N-gram-like Language Models Predict Reading Time Best
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915851244929024 |
|---|---|
| author | Michaelov, James A. Levy, Roger P. |
| author_facet | Michaelov, James A. Levy, Roger P. |
| contents | Recent work has found that contemporary language models such as transformers can become so good at next-word prediction that the probabilities they calculate become worse for predicting reading time. In this paper, we propose that this can be explained by reading time being sensitive to simple n-gram statistics rather than the more complex statistics learned by state-of-the-art transformer language models. We demonstrate that the neural language models whose predictions are most correlated with n-gram probability are also those that calculate probabilities that are the most correlated with eye-tracking-based metrics of reading time on naturalistic text. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_09872 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | N-gram-like Language Models Predict Reading Time Best Michaelov, James A. Levy, Roger P. Computation and Language Recent work has found that contemporary language models such as transformers can become so good at next-word prediction that the probabilities they calculate become worse for predicting reading time. In this paper, we propose that this can be explained by reading time being sensitive to simple n-gram statistics rather than the more complex statistics learned by state-of-the-art transformer language models. We demonstrate that the neural language models whose predictions are most correlated with n-gram probability are also those that calculate probabilities that are the most correlated with eye-tracking-based metrics of reading time on naturalistic text. |
| title | N-gram-like Language Models Predict Reading Time Best |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2603.09872 |