Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2022
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2212.05098 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917740134006784 |
|---|---|
| author | Clausecker, Robert Lemire, Daniel |
| author_facet | Clausecker, Robert Lemire, Daniel |
| contents | Intel includes in its recent processors a powerful set of instructions capable of processing 512-bit registers with a single instruction (AVX-512). Some of these instructions have no equivalent in earlier instruction sets. We leverage these instructions to efficiently transcode strings between the most common formats: UTF-8 and UTF-16. With our novel algorithms, we are often twice as fast as the previous best solutions. For example, we transcode Chinese text from UTF-8 to UTF-16 at more than 5 GiB/s using fewer than 2 CPU instructions per character. To ensure reproducibility, we make our software freely available as an open source library. Our library is part of the popular Node.js JavaScript runtime. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2212_05098 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Transcoding Unicode Characters with AVX-512 Instructions Clausecker, Robert Lemire, Daniel Data Structures and Algorithms Intel includes in its recent processors a powerful set of instructions capable of processing 512-bit registers with a single instruction (AVX-512). Some of these instructions have no equivalent in earlier instruction sets. We leverage these instructions to efficiently transcode strings between the most common formats: UTF-8 and UTF-16. With our novel algorithms, we are often twice as fast as the previous best solutions. For example, we transcode Chinese text from UTF-8 to UTF-16 at more than 5 GiB/s using fewer than 2 CPU instructions per character. To ensure reproducibility, we make our software freely available as an open source library. Our library is part of the popular Node.js JavaScript runtime. |
| title | Transcoding Unicode Characters with AVX-512 Instructions |
| topic | Data Structures and Algorithms |
| url | https://arxiv.org/abs/2212.05098 |