Cover Image

Can You Really Trust Code Copilots? Evaluating Large Language Models from a Code Security Perspective

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Mou, Yutao, Deng, Xiao, Luo, Yuxiao, Zhang, Shikun, Ye, Wei
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Decoupling Safety into Orthogonal Subspace: Cost-Efficient and Performance-Preserving Alignment for Large Language Models
by: Mou, Yutao, et al.
Published: (2025)

SaRO: Enhancing LLM Safety through Reasoning-based Alignment
by: Mou, Yutao, et al.
Published: (2025)

SG-Bench: Evaluating LLM Safety Generalization Across Diverse Tasks and Prompt Types
by: Mou, Yutao, et al.
Published: (2024)

Is Your AI-Generated Code Really Safe? Evaluating Large Language Models on Secure Code Generation with CodeSecEval
by: Wang, Jiexin, et al.
Published: (2024)

ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedback
by: Mou, Yutao, et al.
Published: (2026)

CodeJudge-Eval: Can Large Language Models be Good Judges in Code Understanding?
by: Zhao, Yuwei, et al.
Published: (2024)

Is Compression Really Linear with Code Intelligence?
by: Xuyang, Shijie, et al.
Published: (2025)

Can Large Language Models abstract Medical Coded Language?
by: Lee, Simon A., et al.
Published: (2024)

DynaCode: A Dynamic Complexity-Aware Code Benchmark for Evaluating Large Language Models in Code Generation
by: Hu, Wenhao, et al.
Published: (2025)

SIMCOPILOT: Evaluating Large Language Models for Copilot-Style Code Generation
by: Jiang, Mingchao, et al.
Published: (2025)

CodeJudge: Evaluating Code Generation with Large Language Models
by: Tong, Weixi, et al.
Published: (2024)

CodeMind: Evaluating Large Language Models for Code Reasoning
by: Liu, Changshu, et al.
Published: (2024)

DolphCoder: Echo-Locating Code Large Language Models with Diverse and Multi-Objective Instruction Tuning
by: Wang, Yejie, et al.
Published: (2024)

DA-Code: Agent Data Science Code Generation Benchmark for Large Language Models
by: Huang, Yiming, et al.
Published: (2024)

CodeCriticBench: A Holistic Code Critique Benchmark for Large Language Models
by: Zhang, Alexander, et al.
Published: (2025)

Refining Corpora from a Model Calibration Perspective for Chinese Spelling Correction
by: Yu, Dingyao, et al.
Published: (2024)

CodeSimpleQA: Scaling Factuality in Code Large Language Models
by: Yang, Jian, et al.
Published: (2025)

LLM4Decompile: Decompiling Binary Code with Large Language Models
by: Tan, Hanzhuo, et al.
Published: (2024)

Latent Preference Coding: Aligning Large Language Models via Discrete Latent Codes
by: Gong, Zhuocheng, et al.
Published: (2025)

MPL: Multiple Programming Languages with Large Language Models for Information Extraction
by: Li, Bo, et al.
Published: (2025)

CodeMixBench: Evaluating Large Language Models on Code Generation with Code-Mixed Prompts
by: Sheokand, Manik, et al.
Published: (2025)

Dialogue You Can Trust: Human and AI Perspectives on Generated Conversations
by: Ebubechukwu, Ike, et al.
Published: (2024)

Code Comparison Tuning for Code Large Language Models
by: Jiang, Yufan, et al.
Published: (2024)

AutoRed: A Free-form Adversarial Prompt Generation Framework for Automated Red Teaming
by: Diao, Muxi, et al.
Published: (2025)

AutoCodeBench: Large Language Models are Automatic Code Benchmark Generators
by: Chou, Jason, et al.
Published: (2025)

CodeApex: A Bilingual Programming Evaluation Benchmark for Large Language Models
by: Fu, Lingyue, et al.
Published: (2023)

SV-TrustEval-C: Evaluating Structure and Semantic Reasoning in Large Language Models for Source Code Vulnerability Analysis
by: Li, Yansong, et al.
Published: (2025)

Hal-Eval: A Universal and Fine-grained Hallucination Evaluation Framework for Large Vision Language Models
by: Jiang, Chaoya, et al.
Published: (2024)

Improving Natural Language Capability of Code Large Language Model
by: Li, Wei, et al.
Published: (2024)

Coding Triangle: How Does Large Language Model Understand Code?
by: Zhang, Taolin, et al.
Published: (2025)

FreeEval: A Modular Framework for Trustworthy and Efficient Evaluation of Large Language Models
by: Yu, Zhuohao, et al.
Published: (2024)

AutoSurvey: Large Language Models Can Automatically Write Surveys
by: Wang, Yidong, et al.
Published: (2024)

Plot2Code: A Comprehensive Benchmark for Evaluating Multi-modal Large Language Models in Code Generation from Scientific Plots
by: Wu, Chengyue, et al.
Published: (2024)

Evaluating Intermediate Reasoning of Code-Assisted Large Language Models for Mathematics
by: Al-Khalili, Zena, et al.
Published: (2025)

Compositional Hardness of Code in Large Language Models -- A Probabilistic Perspective
by: Wolf, Yotam, et al.
Published: (2024)

LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
by: Jain, Naman, et al.
Published: (2024)

HumanEval Pro and MBPP Pro: Evaluating Large Language Models on Self-invoking Code Generation
by: Yu, Zhaojian, et al.
Published: (2024)

OverleafCopilot: Empowering Academic Writing in Overleaf with Large Language Models
by: Wen, Haomin, et al.
Published: (2024)

SecureReviewer: Enhancing Large Language Models for Secure Code Review through Secure-aware Fine-tuning
by: Liu, Fang, et al.
Published: (2025)

CodeChameleon: Personalized Encryption Framework for Jailbreaking Large Language Models
by: Lv, Huijie, et al.
Published: (2024)