Probability Distribution Collapse: A Critical Bottleneck to Compact Unsupervised Neural Grammar Induction
Fuente:
arXiv
Guardado en:
| Autores principales: | , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914055906656256 |
|---|---|
| author | Park, Jinwook Kim, Kangil |
| author_facet | Park, Jinwook Kim, Kangil |
| contents | Unsupervised neural grammar induction aims to learn interpretable hierarchical structures from language data. However, existing models face an expressiveness bottleneck, often resulting in unnecessarily large yet underperforming grammars. We identify a core issue, $\textit{probability distribution collapse}$, as the underlying cause of this limitation. We analyze when and how the collapse emerges across key components of neural parameterization and introduce a targeted solution, $\textit{collapse-relaxing neural parameterization}$, to mitigate it. Our approach substantially improves parsing performance while enabling the use of significantly more compact grammars across a wide range of languages, as demonstrated through extensive empirical analysis. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_20734 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Probability Distribution Collapse: A Critical Bottleneck to Compact Unsupervised Neural Grammar Induction Park, Jinwook Kim, Kangil Computation and Language Unsupervised neural grammar induction aims to learn interpretable hierarchical structures from language data. However, existing models face an expressiveness bottleneck, often resulting in unnecessarily large yet underperforming grammars. We identify a core issue, $\textit{probability distribution collapse}$, as the underlying cause of this limitation. We analyze when and how the collapse emerges across key components of neural parameterization and introduce a targeted solution, $\textit{collapse-relaxing neural parameterization}$, to mitigate it. Our approach substantially improves parsing performance while enabling the use of significantly more compact grammars across a wide range of languages, as demonstrated through extensive empirical analysis. |
| title | Probability Distribution Collapse: A Critical Bottleneck to Compact Unsupervised Neural Grammar Induction |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2509.20734 |