Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2512.17065 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918256103653376 |
|---|---|
| author | Patel, Dhruvesh Maram, Durga Prasad Chintha, Sai Sreenivas Rozonoyer, Benjamin McCallum, Andrew |
| author_facet | Patel, Dhruvesh Maram, Durga Prasad Chintha, Sai Sreenivas Rozonoyer, Benjamin McCallum, Andrew |
| contents | In recent years, there has been a resurgence of interest in non-autoregressive text generation in the context of general language modeling. Unlike the well-established autoregressive language modeling paradigm, which has a plethora of standard training and inference libraries, implementations of non-autoregressive language modeling have largely been bespoke making it difficult to perform systematic comparisons of different methods. Moreover, each non-autoregressive language model typically requires it own data collation, loss, and prediction logic, making it challenging to reuse common components. In this work, we present the XLM python package, which is designed to make implementing small non-autoregressive language models faster with a secondary goal of providing a suite of small pre-trained models (through a companion xlm-models package) that can be used by the research community. The code is available at https://github.com/dhruvdcoder/xlm-core. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_17065 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | XLM: A Python package for non-autoregressive language models Patel, Dhruvesh Maram, Durga Prasad Chintha, Sai Sreenivas Rozonoyer, Benjamin McCallum, Andrew Computation and Language In recent years, there has been a resurgence of interest in non-autoregressive text generation in the context of general language modeling. Unlike the well-established autoregressive language modeling paradigm, which has a plethora of standard training and inference libraries, implementations of non-autoregressive language modeling have largely been bespoke making it difficult to perform systematic comparisons of different methods. Moreover, each non-autoregressive language model typically requires it own data collation, loss, and prediction logic, making it challenging to reuse common components. In this work, we present the XLM python package, which is designed to make implementing small non-autoregressive language models faster with a secondary goal of providing a suite of small pre-trained models (through a companion xlm-models package) that can be used by the research community. The code is available at https://github.com/dhruvdcoder/xlm-core. |
| title | XLM: A Python package for non-autoregressive language models |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2512.17065 |