Readability Reconsidered: A Cross-Dataset Analysis of Reference-Free Metrics
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866908599245078528 |
|---|---|
| author | Belem, Catarina G Glenn, Parker Samuel, Alfy Kumar, Anoop Liu, Daben |
| author_facet | Belem, Catarina G Glenn, Parker Samuel, Alfy Kumar, Anoop Liu, Daben |
| contents | Automatic readability assessment plays a key role in ensuring effective and accessible written communication. Despite significant progress, the field is hindered by inconsistent definitions of readability and measurements that rely on surface-level text properties. In this work, we investigate the factors shaping human perceptions of readability through the analysis of 897 judgments, finding that, beyond surface-level cues, information content and topic strongly shape text comprehensibility. Furthermore, we evaluate 15 popular readability metrics across five English datasets, contrasting them with six more nuanced, model-based metrics. Our results show that four model-based metrics consistently place among the top four in rank correlations with human judgments, while the best performing traditional metric achieves an average rank of 8.6. These findings highlight a mismatch between current readability metrics and human perceptions, pointing to model-based approaches as a more promising direction. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_15345 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Readability Reconsidered: A Cross-Dataset Analysis of Reference-Free Metrics Belem, Catarina G Glenn, Parker Samuel, Alfy Kumar, Anoop Liu, Daben Computation and Language Artificial Intelligence Automatic readability assessment plays a key role in ensuring effective and accessible written communication. Despite significant progress, the field is hindered by inconsistent definitions of readability and measurements that rely on surface-level text properties. In this work, we investigate the factors shaping human perceptions of readability through the analysis of 897 judgments, finding that, beyond surface-level cues, information content and topic strongly shape text comprehensibility. Furthermore, we evaluate 15 popular readability metrics across five English datasets, contrasting them with six more nuanced, model-based metrics. Our results show that four model-based metrics consistently place among the top four in rank correlations with human judgments, while the best performing traditional metric achieves an average rank of 8.6. These findings highlight a mismatch between current readability metrics and human perceptions, pointing to model-based approaches as a more promising direction. |
| title | Readability Reconsidered: A Cross-Dataset Analysis of Reference-Free Metrics |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2510.15345 |