Challenge on Optimization of Context Collection for Code Completion
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916990921211904 |
|---|---|
| author | Ustalov, Dmitry Bogomolov, Egor Bezzubov, Alexander Golubev, Yaroslav Glukhov, Evgeniy Levtsov, Georgii Kovalenko, Vladimir |
| author_facet | Ustalov, Dmitry Bogomolov, Egor Bezzubov, Alexander Golubev, Yaroslav Glukhov, Evgeniy Levtsov, Georgii Kovalenko, Vladimir |
| contents | The rapid advancement of workflows and methods for software engineering using AI emphasizes the need for a systematic evaluation and analysis of their ability to leverage information from entire projects, particularly in large code bases. In this challenge on optimization of context collection for code completion, organized by JetBrains in collaboration with Mistral AI as part of the ASE 2025 conference, participants developed efficient mechanisms for collecting context from source code repositories to improve fill-in-the-middle code completions for Python and Kotlin. We constructed a large dataset of real-world code in these two programming languages using permissively licensed open-source projects. The submissions were evaluated based on their ability to maximize completion quality for multiple state-of-the-art neural models using the chrF metric. During the public phase of the competition, nineteen teams submitted solutions to the Python track and eight teams submitted solutions to the Kotlin track. In the private phase, six teams competed, of which five submitted papers to the workshop. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_04349 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Challenge on Optimization of Context Collection for Code Completion Ustalov, Dmitry Bogomolov, Egor Bezzubov, Alexander Golubev, Yaroslav Glukhov, Evgeniy Levtsov, Georgii Kovalenko, Vladimir Software Engineering Artificial Intelligence Machine Learning The rapid advancement of workflows and methods for software engineering using AI emphasizes the need for a systematic evaluation and analysis of their ability to leverage information from entire projects, particularly in large code bases. In this challenge on optimization of context collection for code completion, organized by JetBrains in collaboration with Mistral AI as part of the ASE 2025 conference, participants developed efficient mechanisms for collecting context from source code repositories to improve fill-in-the-middle code completions for Python and Kotlin. We constructed a large dataset of real-world code in these two programming languages using permissively licensed open-source projects. The submissions were evaluated based on their ability to maximize completion quality for multiple state-of-the-art neural models using the chrF metric. During the public phase of the competition, nineteen teams submitted solutions to the Python track and eight teams submitted solutions to the Kotlin track. In the private phase, six teams competed, of which five submitted papers to the workshop. |
| title | Challenge on Optimization of Context Collection for Code Completion |
| topic | Software Engineering Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2510.04349 |