Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2302.09207 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916217839681536 |
|---|---|
| author | Bursztein, Elie Zhang, Marina Vallis, Owen Jia, Xinyu Kurakin, Alexey |
| author_facet | Bursztein, Elie Zhang, Marina Vallis, Owen Jia, Xinyu Kurakin, Alexey |
| contents | This paper describes RETVec, an efficient, resilient, and multilingual text vectorizer designed for neural-based text processing. RETVec combines a novel character encoding with an optional small embedding model to embed words into a 256-dimensional vector space. The RETVec embedding model is pre-trained using pair-wise metric learning to be robust against typos and character-level adversarial attacks. In this paper, we evaluate and compare RETVec to state-of-the-art vectorizers and word embeddings on popular model architectures and datasets. These comparisons demonstrate that RETVec leads to competitive, multilingual models that are significantly more resilient to typos and adversarial text attacks. RETVec is available under the Apache 2 license at https://github.com/google-research/retvec. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2302_09207 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | RETVec: Resilient and Efficient Text Vectorizer Bursztein, Elie Zhang, Marina Vallis, Owen Jia, Xinyu Kurakin, Alexey Computation and Language Artificial Intelligence This paper describes RETVec, an efficient, resilient, and multilingual text vectorizer designed for neural-based text processing. RETVec combines a novel character encoding with an optional small embedding model to embed words into a 256-dimensional vector space. The RETVec embedding model is pre-trained using pair-wise metric learning to be robust against typos and character-level adversarial attacks. In this paper, we evaluate and compare RETVec to state-of-the-art vectorizers and word embeddings on popular model architectures and datasets. These comparisons demonstrate that RETVec leads to competitive, multilingual models that are significantly more resilient to typos and adversarial text attacks. RETVec is available under the Apache 2 license at https://github.com/google-research/retvec. |
| title | RETVec: Resilient and Efficient Text Vectorizer |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2302.09207 |