Cover Image

Is Your AI-Generated Code Really Safe? Evaluating Large Language Models on Secure Code Generation with CodeSecEval

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Jiexin, Luo, Xitong, Cao, Liuwen, He, Hongkui, Huang, Hailin, Xie, Jiayuan, Jatowt, Adam, Cai, Yi
Format:	Preprint
Published:	2024
Subjects:	Software Engineering Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

SecCodePRM: A Process Reward Model for Code Security
by: Yu, Weichen, et al.
Published: (2026)

RealSec-bench: A Benchmark for Evaluating Secure Code Generation in Real-World Repositories
by: Wang, Yanlin, et al.
Published: (2026)

Does Teaming-Up LLMs Improve Secure Code Generation? A Comprehensive Evaluation with Multi-LLMSecCodeEval
by: Sabir, Bushra, et al.
Published: (2026)

Codexity: Secure AI-assisted Code Generation
by: Kim, Sung Yong, et al.
Published: (2024)

ArkEval: Benchmarking and Evaluating Automated CodeRepair for ArkTS
by: Xie, Bang, et al.
Published: (2026)

ComplexCodeEval: A Benchmark for Evaluating Large Code Models on More Complex Code
by: Feng, Jia, et al.
Published: (2024)

PromSec: Prompt Optimization for Secure Generation of Functional Source Code with Large Language Models (LLMs)
by: Nazzal, Mahmoud, et al.
Published: (2024)

ProSec: Fortifying Code LLMs with Proactive Security Alignment
by: Xu, Xiangzhe, et al.
Published: (2024)

CodeScore: Evaluating Code Generation by Learning Code Execution
by: Dong, Yihong, et al.
Published: (2023)

SR-Eval: Evaluating LLMs on Code Generation under Stepwise Requirement Refinement
by: Zhan, Zexun, et al.
Published: (2025)

Rethinking the Evaluation of Secure Code Generation
by: Dai, Shih-Chieh, et al.
Published: (2025)

Do AI Agents Really Improve Code Readability?
by: Horikawa, Kyogo, et al.
Published: (2026)

Is Vibe Coding Safe? Benchmarking Vulnerability of Agent-Generated Code in Real-World Tasks
by: Zhao, Songwen, et al.
Published: (2025)

ScenEval: A Benchmark for Scenario-Based Evaluation of Code Generation
by: Paul, Debalina Ghosh, et al.
Published: (2024)

CoderEval: A Benchmark of Pragmatic Code Generation with Generative Pre-trained Models
by: Yu, Hao, et al.
Published: (2023)

DevEval: Evaluating Code Generation in Practical Software Projects
by: Li, Jia, et al.
Published: (2024)

SolContractEval: A Benchmark for Evaluating Contract-Level Solidity Code Generation
by: Ye, Zhifan, et al.
Published: (2025)

RedCode: Risky Code Execution and Generation Benchmark for Code Agents
by: Guo, Chengquan, et al.
Published: (2024)

AI Code Generators for Security: Friend or Foe?
by: Natella, Roberto, et al.
Published: (2024)

AI Code in the Wild: Measuring Security Risks and Ecosystem Shifts of AI-Generated Code in Modern Software
by: Wang, Bin, et al.
Published: (2025)

LiCoEval: Evaluating LLMs on License Compliance in Code Generation
by: Xu, Weiwei, et al.
Published: (2024)

Evaluation of Code LLMs on Geospatial Code Generation
by: Gramacki, Piotr, et al.
Published: (2024)

On Evaluating the Efficiency of Source Code Generated by LLMs
by: Niu, Changan, et al.
Published: (2024)

SolEval: Benchmarking Large Language Models for Repository-level Solidity Code Generation
by: Peng, Zhiyuan, et al.
Published: (2025)

Fortifying LLM-Based Code Generation with Graph-Based Reasoning on Secure Coding Practices
by: Patir, Rupam, et al.
Published: (2025)

SecCodeBench-V2 Technical Report
by: Chen, Longfei, et al.
Published: (2026)

CFCEval: Evaluating Security Aspects in Code Generated by Large Language Models
by: Cheng, Cheng, et al.
Published: (2025)

DevEval: A Manually-Annotated Code Generation Benchmark Aligned with Real-World Code Repositories
by: Li, Jia, et al.
Published: (2024)

SALLM: Security Assessment of Generated Code
by: Siddiq, Mohammed Latif, et al.
Published: (2023)

A.S.E: A Repository-Level Benchmark for Evaluating Security in AI-Generated Code
by: Lian, Keke, et al.
Published: (2025)

An Empirical Security Evaluation of LLM-Generated Cryptographic Rust Code
by: Elsayed, Mohamed, et al.
Published: (2026)

CodeArena: A Collective Evaluation Platform for LLM Code Generation
by: Du, Mingzhe, et al.
Published: (2025)

Contextualized Code Pretraining for Code Generation
by: Liu, Chen, et al.
Published: (2026)

UniCode: Augmenting Evaluation for Code Reasoning
by: Zheng, Xinyue, et al.
Published: (2025)

ClassEval-Pro: A Cross-Domain Benchmark for Class-Level Code Generation
by: Chen, Yeheng, et al.
Published: (2026)

CodeJudge-Eval: Can Large Language Models be Good Judges in Code Understanding?
by: Zhao, Yuwei, et al.
Published: (2024)

ClarEval: A Benchmark for Evaluating Clarification Skills of Code Agents under Ambiguous Instructions
by: Li, Jialin, et al.
Published: (2026)

ProjectEval: A Benchmark for Programming Agents Automated Evaluation on Project-Level Code Generation
by: Liu, Kaiyuan, et al.
Published: (2025)

Assessing the Quality and Security of AI-Generated Code: A Quantitative Analysis
by: Sabra, Abbas, et al.
Published: (2025)

Code Fingerprints: Disentangled Attribution of LLM-Generated Code
by: Guo, Jiaxun, et al.
Published: (2026)