Processing Natural Language on Embedded Devices: How Well Do Transformer Models Perform?
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2023
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913256456585216 |
|---|---|
| author | Sarkar, Souvika Babar, Mohammad Fakhruddin Hassan, Md Mahadi Hasan, Monowar Santu, Shubhra Kanti Karmaker |
| author_facet | Sarkar, Souvika Babar, Mohammad Fakhruddin Hassan, Md Mahadi Hasan, Monowar Santu, Shubhra Kanti Karmaker |
| contents | This paper presents a performance study of transformer language models under different hardware configurations and accuracy requirements and derives empirical observations about these resource/accuracy trade-offs. In particular, we study how the most commonly used BERT-based language models (viz., BERT, RoBERTa, DistilBERT, and TinyBERT) perform on embedded systems. We tested them on four off-the-shelf embedded platforms (Raspberry Pi, Jetson, UP2, and UDOO) with 2 GB and 4 GB memory (i.e., a total of eight hardware configurations) and four datasets (i.e., HuRIC, GoEmotion, CoNLL, WNUT17) running various NLP tasks. Our study finds that executing complex NLP tasks (such as "sentiment" classification) on embedded systems is feasible even without any GPUs (e.g., Raspberry Pi with 2 GB of RAM). Our findings can help designers understand the deployability and performance of transformer language models, especially those based on BERT architectures. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2304_11520 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Processing Natural Language on Embedded Devices: How Well Do Transformer Models Perform? Sarkar, Souvika Babar, Mohammad Fakhruddin Hassan, Md Mahadi Hasan, Monowar Santu, Shubhra Kanti Karmaker Computation and Language Systems and Control This paper presents a performance study of transformer language models under different hardware configurations and accuracy requirements and derives empirical observations about these resource/accuracy trade-offs. In particular, we study how the most commonly used BERT-based language models (viz., BERT, RoBERTa, DistilBERT, and TinyBERT) perform on embedded systems. We tested them on four off-the-shelf embedded platforms (Raspberry Pi, Jetson, UP2, and UDOO) with 2 GB and 4 GB memory (i.e., a total of eight hardware configurations) and four datasets (i.e., HuRIC, GoEmotion, CoNLL, WNUT17) running various NLP tasks. Our study finds that executing complex NLP tasks (such as "sentiment" classification) on embedded systems is feasible even without any GPUs (e.g., Raspberry Pi with 2 GB of RAM). Our findings can help designers understand the deployability and performance of transformer language models, especially those based on BERT architectures. |
| title | Processing Natural Language on Embedded Devices: How Well Do Transformer Models Perform? |
| topic | Computation and Language Systems and Control |
| url | https://arxiv.org/abs/2304.11520 |