RETVec: Resilient and Efficient Text Vectorizer
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916217839681536 |
|---|---|
| author | Bursztein, Elie Zhang, Marina Vallis, Owen Jia, Xinyu Kurakin, Alexey |
| author_facet | Bursztein, Elie Zhang, Marina Vallis, Owen Jia, Xinyu Kurakin, Alexey |
| contents | This paper describes RETVec, an efficient, resilient, and multilingual text vectorizer designed for neural-based text processing. RETVec combines a novel character encoding with an optional small embedding model to embed words into a 256-dimensional vector space. The RETVec embedding model is pre-trained using pair-wise metric learning to be robust against typos and character-level adversarial attacks. In this paper, we evaluate and compare RETVec to state-of-the-art vectorizers and word embeddings on popular model architectures and datasets. These comparisons demonstrate that RETVec leads to competitive, multilingual models that are significantly more resilient to typos and adversarial text attacks. RETVec is available under the Apache 2 license at https://github.com/google-research/retvec. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2302_09207 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | RETVec: Resilient and Efficient Text Vectorizer Bursztein, Elie Zhang, Marina Vallis, Owen Jia, Xinyu Kurakin, Alexey Computation and Language Artificial Intelligence This paper describes RETVec, an efficient, resilient, and multilingual text vectorizer designed for neural-based text processing. RETVec combines a novel character encoding with an optional small embedding model to embed words into a 256-dimensional vector space. The RETVec embedding model is pre-trained using pair-wise metric learning to be robust against typos and character-level adversarial attacks. In this paper, we evaluate and compare RETVec to state-of-the-art vectorizers and word embeddings on popular model architectures and datasets. These comparisons demonstrate that RETVec leads to competitive, multilingual models that are significantly more resilient to typos and adversarial text attacks. RETVec is available under the Apache 2 license at https://github.com/google-research/retvec. |
| title | RETVec: Resilient and Efficient Text Vectorizer |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2302.09207 |