Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.05788 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914079129468928 |
|---|---|
| author | Pavlichenko, Nikita Nazarov, Iurii Dolgov, Ivan Garanina, Ekaterina Ustalov, Dmitry Bondyrev, Ivan Lysaniuk, Kseniia Vu, Evgeniia Chekmenev, Kirill Shtok, Joseph Golubev, Yaroslav Semenkin, Anton Sazanovich, Uladzislau |
| author_facet | Pavlichenko, Nikita Nazarov, Iurii Dolgov, Ivan Garanina, Ekaterina Ustalov, Dmitry Bondyrev, Ivan Lysaniuk, Kseniia Vu, Evgeniia Chekmenev, Kirill Shtok, Joseph Golubev, Yaroslav Semenkin, Anton Sazanovich, Uladzislau |
| contents | We present the Mellum models family, open-weight code completion models designed for interactive use in JetBrains IDEs. Mellums have 4B parameters, adopt a Llama-style architecture, and are pre-trained on ~4T tokens of permissively licensed, multi-language code. Our studies show that (i) careful data curation and staged training significantly improve the model's quality, (ii) editor-critical capabilities such as context packing are necessary for high-quality suggestions, and (iii) a compact, task-focused model can meet the cost and latency constraints of interactive completion.
In the paper, we describe an end-to-end industrial pipeline for producing contextualized in-editor completion: disciplined data governance, multi-stage training that includes fill-in-the-middle and project context via supervised fine-tuning, and alignment via direct preference optimization using feedback from real-world scenarios. Our quality evaluations include both large-scale offline benchmarks and online telemetry from production deployments in JetBrains IDEs. Mellums are released under the Apache-2.0 license on HuggingFace, with a public model card providing a reproducible reference for practitioners. Our experience offers a pragmatic blueprint for taking a focused, open model from a research prototype to at scale production for hundreds of thousands of users. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_05788 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Mellum: Production-Grade in-IDE Contextual Code Completion with Multi-File Project Understanding Pavlichenko, Nikita Nazarov, Iurii Dolgov, Ivan Garanina, Ekaterina Ustalov, Dmitry Bondyrev, Ivan Lysaniuk, Kseniia Vu, Evgeniia Chekmenev, Kirill Shtok, Joseph Golubev, Yaroslav Semenkin, Anton Sazanovich, Uladzislau Software Engineering Artificial Intelligence Machine Learning We present the Mellum models family, open-weight code completion models designed for interactive use in JetBrains IDEs. Mellums have 4B parameters, adopt a Llama-style architecture, and are pre-trained on ~4T tokens of permissively licensed, multi-language code. Our studies show that (i) careful data curation and staged training significantly improve the model's quality, (ii) editor-critical capabilities such as context packing are necessary for high-quality suggestions, and (iii) a compact, task-focused model can meet the cost and latency constraints of interactive completion. In the paper, we describe an end-to-end industrial pipeline for producing contextualized in-editor completion: disciplined data governance, multi-stage training that includes fill-in-the-middle and project context via supervised fine-tuning, and alignment via direct preference optimization using feedback from real-world scenarios. Our quality evaluations include both large-scale offline benchmarks and online telemetry from production deployments in JetBrains IDEs. Mellums are released under the Apache-2.0 license on HuggingFace, with a public model card providing a reproducible reference for practitioners. Our experience offers a pragmatic blueprint for taking a focused, open model from a research prototype to at scale production for hundreds of thousands of users. |
| title | Mellum: Production-Grade in-IDE Contextual Code Completion with Multi-File Project Understanding |
| topic | Software Engineering Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2510.05788 |