Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2509.24828 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908565941256192 |
|---|---|
| author | Heisler, Joshua Reisinger, Johannes Fischer, Andreas |
| author_facet | Heisler, Joshua Reisinger, Johannes Fischer, Andreas |
| contents | SAP has released its own proprietary generative model SAP Joule, intended for various generative tasks, including serving as a code assistant for software engineers. While Joule is yet not focused on SAP-specific ABAP code generation, it can be used for other common languages, including Javascript. This paper compares SAP Joules Javascript coding capabilities against a total of 29 other models using the HumanEval-X Javascript benchmark. SAP Joule achieves a strict accuracy of 80.49% as the fifth best model in our evaluation. To the best of our knowledge, this is the first comparative evaluation of SAP Joule code generation capabilities. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_24828 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Evaluating SAP Joule for Code Generation Heisler, Joshua Reisinger, Johannes Fischer, Andreas Software Engineering Artificial Intelligence SAP has released its own proprietary generative model SAP Joule, intended for various generative tasks, including serving as a code assistant for software engineers. While Joule is yet not focused on SAP-specific ABAP code generation, it can be used for other common languages, including Javascript. This paper compares SAP Joules Javascript coding capabilities against a total of 29 other models using the HumanEval-X Javascript benchmark. SAP Joule achieves a strict accuracy of 80.49% as the fifth best model in our evaluation. To the best of our knowledge, this is the first comparative evaluation of SAP Joule code generation capabilities. |
| title | Evaluating SAP Joule for Code Generation |
| topic | Software Engineering Artificial Intelligence |
| url | https://arxiv.org/abs/2509.24828 |