Cover Image

Running in CIRCLE? A Simple Benchmark for LLM Code Interpreter Security

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Chua, Gabriel
Format:	Preprint
Published:	2025
Subjects:	Cryptography and Security Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

SafeGenBench: A Benchmark Framework for Security Vulnerability Detection in LLM-Generated Code
by: Li, Xinghang, et al.
Published: (2025)

SecRepoBench: Benchmarking Code Agents for Secure Code Completion in Real-World Repositories
by: Shen, Chihao, et al.
Published: (2025)

HardSecBench: Benchmarking the Security Awareness of LLMs for Hardware Code Generation
by: Chen, Qirui, et al.
Published: (2026)

Benchmarking Practices in LLM-driven Offensive Security: Testbeds, Metrics, and Experiment Design
by: Happe, Andreas, et al.
Published: (2025)

RAS-Eval: A Comprehensive Benchmark for Security Evaluation of LLM Agents in Real-World Environments
by: Fu, Yuchuan, et al.
Published: (2025)

Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents
by: Zhang, Hanrong, et al.
Published: (2024)

DUALGUAGE: Automated Joint Security-Functionality Benchmarking for Secure Code Generation
by: Pathak, Abhijeet, et al.
Published: (2025)

Towards Effective Offensive Security LLM Agents: Hyperparameter Tuning, LLM as a Judge, and a Lightweight CTF Benchmark
by: Shao, Minghao, et al.
Published: (2025)

Large Language Model (LLM) for Software Security: Code Analysis, Malware Analysis, Reverse Engineering
by: Jelodar, Hamed, et al.
Published: (2025)

LLM-CSEC: Empirical Evaluation of Security in C/C++ Code Generated by Large Language Models
by: Shahid, Muhammad Usman, et al.
Published: (2025)

When Developer Aid Becomes Security Debt: A Systematic Analysis of Insecure Behaviors in LLM Coding Agents
by: Kozak, Matous, et al.
Published: (2025)

Security of LLM-generated Code: A Comparative Analysis
by: Morkonda, Srivathsan G, et al.
Published: (2026)

MCP Security Bench (MSB): Benchmarking Attacks Against Model Context Protocol in LLM Agents
by: Zhang, Dongsen, et al.
Published: (2025)

Towards Secure Logging: Characterizing and Benchmarking Logging Code Security Issues with LLMs
by: Yuan, He Yang, et al.
Published: (2026)

A New Era in LLM Security: Exploring Security Concerns in Real-World LLM-based Systems
by: Wu, Fangzhou, et al.
Published: (2024)

A Mixture of Linear Corrections Generates Secure Code
by: Yu, Weichen, et al.
Published: (2025)

SeCodePLT: A Unified Platform for Evaluating the Security of Code GenAI
by: Nie, Yuzhou, et al.
Published: (2024)

On the (In)Security of LLM App Stores
by: Hou, Xinyi, et al.
Published: (2024)

Benchmarking LLM-Based Static Analysis for Secure Smart Contract Development: Reliability, Limitations, and Potential Hybrid Solutions
by: Susan, Stefan-Claudiu, et al.
Published: (2026)

A Framework for Formalizing LLM Agent Security
by: Siu, Vincent, et al.
Published: (2026)

Enhancing Reliability in LLM-Based Secure Code Generation
by: Kharma, Mohammed F., et al.
Published: (2026)

Benchmark of Benchmarks: Unpacking Influence and Code Repository Quality in LLM Safety Benchmarks
by: Chu, Junjie, et al.
Published: (2026)

Ocassionally Secure: A Comparative Analysis of Code Generation Assistants
by: Elgedawy, Ran, et al.
Published: (2024)

SecPI: Secure Code Generation with Reasoning Models via Security Reasoning Internalization
by: Wang, Hao, et al.
Published: (2026)

Information Security Based on LLM Approaches: A Review
by: Gong, Chang, et al.
Published: (2025)

CodeBC: A More Secure Large Language Model for Smart Contract Code Generation in Blockchain
by: Wang, Lingxiang, et al.
Published: (2025)

Fortifying LLM-Based Code Generation with Graph-Based Reasoning on Secure Coding Practices
by: Patir, Rupam, et al.
Published: (2025)

Breaking the Code: Security Assessment of AI Code Agents Through Systematic Jailbreaking Attacks
by: Saha, Shoumik, et al.
Published: (2025)

VibeGuard: A Security Gate Framework for AI-Generated Code
by: Xie, Ying
Published: (2026)

Marking Code Without Breaking It: Code Watermarking for Detecting LLM-Generated Code
by: Kim, Jungin, et al.
Published: (2025)

Benchmarking Prompt Engineering Techniques for Secure Code Generation with GPT Models
by: Bruni, Marc, et al.
Published: (2025)

Artificial-Intelligence Generated Code Considered Harmful: A Road Map for Secure and High-Quality Code Generation
by: Chong, Chun Jie, et al.
Published: (2024)

SkillTester: Benchmarking Utility and Security of Agent Skills
by: Wang, Leye, et al.
Published: (2026)

Security in LLM-as-a-Judge: A Comprehensive SoK
by: Masoud, Aiman Al, et al.
Published: (2026)

LLM Agents Should Employ Security Principles
by: Zhang, Kaiyuan, et al.
Published: (2025)

aiXamine: Simplified LLM Safety and Security
by: Deniz, Fatih, et al.
Published: (2025)

ESAA-Security: An Event-Sourced, Verifiable Architecture for Agent-Assisted Security Audits of AI-Generated Code
by: Filho, Elzo Brito dos Santos
Published: (2026)

Demo: SGCode: A Flexible Prompt-Optimizing System for Secure Generation of Code
by: Ton, Khiem, et al.
Published: (2024)

Measuring and Exploiting Contextual Bias in LLM-Assisted Security Code Review
by: Mitropoulos, Dimitris, et al.
Published: (2026)

An Empirical Evaluation of LLM-Generated Code Security Across Prompting Methods
by: Kharma, Mohammed, et al.
Published: (2026)