Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2601.03708 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912865730953216 |
|---|---|
| author | Zou, Qingyun Cui, Jiahao Chen, Nuo He, Bingsheng Wong, Weng-Fai |
| author_facet | Zou, Qingyun Cui, Jiahao Chen, Nuo He, Bingsheng Wong, Weng-Fai |
| contents | Large language models (LLMs) have achieved strong performance on code completion tasks in general-purpose programming languages. However, existing repository-level code completion benchmarks focus almost exclusively on software code and largely overlook hardware description languages. In this work, we present \textbf{MHRC-Bench}, consisting of \textbf{MHRC-Bench-Train} and \textbf{MHRC-Bench-Eval}, the first benchmark designed for multilingual hardware code completion at the repository level. Our benchmark targets completion tasks and covers three major hardware design coding styles. Each completion target is annotated with code-structure-level and hardware-oriented semantic labels derived from concrete syntax tree analysis. We conduct a comprehensive evaluation of models on MHRC-Bench-Eval. Comprehensive evaluation results and analysis demonstrate the effectiveness of MHRC-Bench. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_03708 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | MHRC-Bench: A Multilingual Hardware Repository-Level Code Completion benchmark Zou, Qingyun Cui, Jiahao Chen, Nuo He, Bingsheng Wong, Weng-Fai Programming Languages Artificial Intelligence Large language models (LLMs) have achieved strong performance on code completion tasks in general-purpose programming languages. However, existing repository-level code completion benchmarks focus almost exclusively on software code and largely overlook hardware description languages. In this work, we present \textbf{MHRC-Bench}, consisting of \textbf{MHRC-Bench-Train} and \textbf{MHRC-Bench-Eval}, the first benchmark designed for multilingual hardware code completion at the repository level. Our benchmark targets completion tasks and covers three major hardware design coding styles. Each completion target is annotated with code-structure-level and hardware-oriented semantic labels derived from concrete syntax tree analysis. We conduct a comprehensive evaluation of models on MHRC-Bench-Eval. Comprehensive evaluation results and analysis demonstrate the effectiveness of MHRC-Bench. |
| title | MHRC-Bench: A Multilingual Hardware Repository-Level Code Completion benchmark |
| topic | Programming Languages Artificial Intelligence |
| url | https://arxiv.org/abs/2601.03708 |