CodeSift: An LLM-Based Reference-Less Framework for Automatic Code Validation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916373252276224 |
|---|---|
| author | Aggarwal, Pooja Chatterjee, Oishik Dai, Ting Mohapatra, Prateeti Paulovicks, Brent Blancett, Brad De Magalhaes, Arthur |
| author_facet | Aggarwal, Pooja Chatterjee, Oishik Dai, Ting Mohapatra, Prateeti Paulovicks, Brent Blancett, Brad De Magalhaes, Arthur |
| contents | The advent of large language models (LLMs) has greatly facilitated code generation, but ensuring the functional correctness of generated code remains a challenge. Traditional validation methods are often time-consuming, error-prone, and impractical for large volumes of code. We introduce CodeSift, a novel framework that leverages LLMs as the first-line filter of code validation without the need for execution, reference code, or human feedback, thereby reducing the validation effort. We assess the effectiveness of our method across three diverse datasets encompassing two programming languages. Our results indicate that CodeSift outperforms state-of-the-art code evaluation methods. Internal testing conducted with subject matter experts reveals that the output generated by CodeSift is in line with human preference, reinforcing its effectiveness as a dependable automated code validation tool. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_15630 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | CodeSift: An LLM-Based Reference-Less Framework for Automatic Code Validation Aggarwal, Pooja Chatterjee, Oishik Dai, Ting Mohapatra, Prateeti Paulovicks, Brent Blancett, Brad De Magalhaes, Arthur Software Engineering Artificial Intelligence The advent of large language models (LLMs) has greatly facilitated code generation, but ensuring the functional correctness of generated code remains a challenge. Traditional validation methods are often time-consuming, error-prone, and impractical for large volumes of code. We introduce CodeSift, a novel framework that leverages LLMs as the first-line filter of code validation without the need for execution, reference code, or human feedback, thereby reducing the validation effort. We assess the effectiveness of our method across three diverse datasets encompassing two programming languages. Our results indicate that CodeSift outperforms state-of-the-art code evaluation methods. Internal testing conducted with subject matter experts reveals that the output generated by CodeSift is in line with human preference, reinforcing its effectiveness as a dependable automated code validation tool. |
| title | CodeSift: An LLM-Based Reference-Less Framework for Automatic Code Validation |
| topic | Software Engineering Artificial Intelligence |
| url | https://arxiv.org/abs/2408.15630 |