Linguistic Frameworks Go Toe-to-Toe at Neuro-Symbolic Language Modeling
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2021
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918425803096064 |
|---|---|
| author | Prange, Jakob Schneider, Nathan Kong, Lingpeng |
| author_facet | Prange, Jakob Schneider, Nathan Kong, Lingpeng |
| contents | We examine the extent to which, in principle, linguistic graph representations can complement and improve neural language modeling. With an ensemble setup consisting of a pretrained Transformer and ground-truth graphs from one of 7 different formalisms, we find that, overall, semantic constituency structures are most useful to language modeling performance -- outpacing syntactic constituency structures as well as syntactic and semantic dependency structures. Further, effects vary greatly depending on part-of-speech class. In sum, our findings point to promising tendencies in neuro-symbolic language modeling and invite future research quantifying the design choices made by different formalisms. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2112_07874 |
| institution | arXiv |
| publishDate | 2021 |
| record_format | arxiv |
| spellingShingle | Linguistic Frameworks Go Toe-to-Toe at Neuro-Symbolic Language Modeling Prange, Jakob Schneider, Nathan Kong, Lingpeng Computation and Language Artificial Intelligence We examine the extent to which, in principle, linguistic graph representations can complement and improve neural language modeling. With an ensemble setup consisting of a pretrained Transformer and ground-truth graphs from one of 7 different formalisms, we find that, overall, semantic constituency structures are most useful to language modeling performance -- outpacing syntactic constituency structures as well as syntactic and semantic dependency structures. Further, effects vary greatly depending on part-of-speech class. In sum, our findings point to promising tendencies in neuro-symbolic language modeling and invite future research quantifying the design choices made by different formalisms. |
| title | Linguistic Frameworks Go Toe-to-Toe at Neuro-Symbolic Language Modeling |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2112.07874 |