PyMarian: Fast Neural Machine Translation and Evaluation in Python
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914920498462720 |
|---|---|
| author | Gowda, Thamme Grundkiewicz, Roman Rippeth, Elijah Post, Matt Junczys-Dowmunt, Marcin |
| author_facet | Gowda, Thamme Grundkiewicz, Roman Rippeth, Elijah Post, Matt Junczys-Dowmunt, Marcin |
| contents | The deep learning language of choice these days is Python; measured by factors such as available libraries and technical support, it is hard to beat. At the same time, software written in lower-level programming languages like C++ retain advantages in speed. We describe a Python interface to Marian NMT, a C++-based training and inference toolkit for sequence-to-sequence models, focusing on machine translation. This interface enables models trained with Marian to be connected to the rich, wide range of tools available in Python. A highlight of the interface is the ability to compute state-of-the-art COMET metrics from Python but using Marian's inference engine, with a speedup factor of up to 7.8$\times$ the existing implementations. We also briefly spotlight a number of other integrations, including Jupyter notebooks, connection with prebuilt models, and a web app interface provided with the package. PyMarian is available in PyPI via $\texttt{pip install pymarian}$. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_11853 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | PyMarian: Fast Neural Machine Translation and Evaluation in Python Gowda, Thamme Grundkiewicz, Roman Rippeth, Elijah Post, Matt Junczys-Dowmunt, Marcin Computation and Language The deep learning language of choice these days is Python; measured by factors such as available libraries and technical support, it is hard to beat. At the same time, software written in lower-level programming languages like C++ retain advantages in speed. We describe a Python interface to Marian NMT, a C++-based training and inference toolkit for sequence-to-sequence models, focusing on machine translation. This interface enables models trained with Marian to be connected to the rich, wide range of tools available in Python. A highlight of the interface is the ability to compute state-of-the-art COMET metrics from Python but using Marian's inference engine, with a speedup factor of up to 7.8$\times$ the existing implementations. We also briefly spotlight a number of other integrations, including Jupyter notebooks, connection with prebuilt models, and a web app interface provided with the package. PyMarian is available in PyPI via $\texttt{pip install pymarian}$. |
| title | PyMarian: Fast Neural Machine Translation and Evaluation in Python |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2408.11853 |