Your Model is Overconfident, and Other Lies We Tell Ourselves
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915179597398016 |
|---|---|
| author | Mickus, Timothee Sinha, Aman Vázquez, Raúl |
| author_facet | Mickus, Timothee Sinha, Aman Vázquez, Raúl |
| contents | The difficulty intrinsic to a given example, rooted in its inherent ambiguity, is a key yet often overlooked factor in evaluating neural NLP models. We investigate the interplay and divergence among various metrics for assessing intrinsic difficulty, including annotator dissensus, training dynamics, and model confidence. Through a comprehensive analysis using 29 models on three datasets, we reveal that while correlations exist among these metrics, their relationships are neither linear nor monotonic. By disentangling these dimensions of uncertainty, we aim to refine our understanding of data complexity and its implications for evaluating and improving NLP models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_01235 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Your Model is Overconfident, and Other Lies We Tell Ourselves Mickus, Timothee Sinha, Aman Vázquez, Raúl Computation and Language The difficulty intrinsic to a given example, rooted in its inherent ambiguity, is a key yet often overlooked factor in evaluating neural NLP models. We investigate the interplay and divergence among various metrics for assessing intrinsic difficulty, including annotator dissensus, training dynamics, and model confidence. Through a comprehensive analysis using 29 models on three datasets, we reveal that while correlations exist among these metrics, their relationships are neither linear nor monotonic. By disentangling these dimensions of uncertainty, we aim to refine our understanding of data complexity and its implications for evaluating and improving NLP models. |
| title | Your Model is Overconfident, and Other Lies We Tell Ourselves |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2503.01235 |