Improving Corpus Annotation Quality Using Word Embedding Models
Fuente:
Redalyc
Saved in:
| Main Author: | |
|---|---|
| Format: | Artículo científico |
| Language: | en |
| Published: |
Instituto Politécnico Nacional
2016
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1876459475081101312 |
|---|---|
| author | Attila Novák |
| author_facet | Attila Novák |
| contents | Improving Corpus Annotation Quality Using Word Embedding Models Attila Novák Computación word embeddings corpus annotation lexical resources Corpus linguistics Web-crawled corpora contain a significant amount of noise. Automatic corpus annotation tools introduce even more noise performing erroneous language identification or encoding detection, introducing tokenization and lemmatization errors and adding erroneous tags or analyses to the original words. Our goal with the methods presented in this article was to use word embedding models to reveal such errors and to provide correction procedures. The evaluation focuses on analyzing and validating noun compounds identifying bogus compound analyses, recognizing and concatenating fragmented words, detecting erroneously encoded text, restoring accents and handling the combination of these errors in a Hungarian web-crawled corpus. 2016 artículo científico 1870-9044 https://www.redalyc.org/articulo.oa?id=402646943006 en http://www.redalyc.org/revista.oa?id=4026 Polibits application/pdf Instituto Politécnico Nacional Polibits (México) Vol.53 |
| format | Artículo científico |
| id | redalyc_402646943006 |
| institution | Redalyc |
| language | en |
| publishDate | 2016 |
| publisher | Instituto Politécnico Nacional |
| spellingShingle | Improving Corpus Annotation Quality Using Word Embedding Models Attila Novák Computación word embeddings corpus annotation lexical resources Corpus linguistics Improving Corpus Annotation Quality Using Word Embedding Models Attila Novák Computación word embeddings corpus annotation lexical resources Corpus linguistics Web-crawled corpora contain a significant amount of noise. Automatic corpus annotation tools introduce even more noise performing erroneous language identification or encoding detection, introducing tokenization and lemmatization errors and adding erroneous tags or analyses to the original words. Our goal with the methods presented in this article was to use word embedding models to reveal such errors and to provide correction procedures. The evaluation focuses on analyzing and validating noun compounds identifying bogus compound analyses, recognizing and concatenating fragmented words, detecting erroneously encoded text, restoring accents and handling the combination of these errors in a Hungarian web-crawled corpus. 2016 artículo científico 1870-9044 https://www.redalyc.org/articulo.oa?id=402646943006 en http://www.redalyc.org/revista.oa?id=4026 Polibits application/pdf Instituto Politécnico Nacional Polibits (México) Vol.53 |
| title | Improving Corpus Annotation Quality Using Word Embedding Models |
| topic | Computación word embeddings corpus annotation lexical resources Corpus linguistics |
| url | https://www.redalyc.org/articulo.oa?id=402646943006 |