Dispersion Measures as Predictors of Lexical Decision Time, Word Familiarity, and Lexical Complexity
Fuente:
arXiv
Guardado en:
| Autores principales: | , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913644363644928 |
|---|---|
| author | Nohejl, Adam Watanabe, Taro |
| author_facet | Nohejl, Adam Watanabe, Taro |
| contents | Various measures of dispersion have been proposed to paint a fuller picture of a word's distribution in a corpus, but only little has been done to validate them externally. We evaluate a wide range of dispersion measures as predictors of lexical decision time, word familiarity, and lexical complexity in five diverse languages. We find that the logarithm of range is not only a better predictor than log-frequency across all tasks and languages, but that it is also the most powerful additional variable to log-frequency, consistently outperforming the more complex dispersion measures. We discuss the effects of corpus part granularity and logarithmic transformation, shedding light on contradictory results of previous studies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_06536 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Dispersion Measures as Predictors of Lexical Decision Time, Word Familiarity, and Lexical Complexity Nohejl, Adam Watanabe, Taro Computation and Language Various measures of dispersion have been proposed to paint a fuller picture of a word's distribution in a corpus, but only little has been done to validate them externally. We evaluate a wide range of dispersion measures as predictors of lexical decision time, word familiarity, and lexical complexity in five diverse languages. We find that the logarithm of range is not only a better predictor than log-frequency across all tasks and languages, but that it is also the most powerful additional variable to log-frequency, consistently outperforming the more complex dispersion measures. We discuss the effects of corpus part granularity and logarithmic transformation, shedding light on contradictory results of previous studies. |
| title | Dispersion Measures as Predictors of Lexical Decision Time, Word Familiarity, and Lexical Complexity |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2501.06536 |