Cover Image

Benchmarking Open-Source Safety Guard Models: A Comprehensive Evaluation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Harsh, Reetu Raj, Sarmah, Bhaskarjit, Pasquali, Stefano
Format:	Preprint
Published:	2026
Subjects:	Computation and Language Artificial Intelligence Software Engineering
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

FinReflectKG -- MultiHop: Financial QA Benchmark for Reasoning with Knowledge Graph Evidence
by: Arun, Abhinav, et al.
Published: (2025)

BenchGuard: Who Guards the Benchmarks? Automated Auditing of LLM Agent Benchmarks
by: Tu, Xinming, et al.
Published: (2026)

A Code Comprehension Benchmark for Large Language Models for Code
by: Havare, Jayant, et al.
Published: (2025)

Uncovering Representation Bias for Investment Decisions in Open-Source Large Language Models
by: Dimino, Fabrizio, et al.
Published: (2025)

Is Open Source the Future of AI? A Data-Driven Approach
by: Vake, Domen, et al.
Published: (2025)

BuildBench: Benchmarking LLM Agents on Compiling Real-World Open-Source Software
by: Zhang, Zehua, et al.
Published: (2025)

GR-NLP-TOOLKIT: An Open-Source NLP Toolkit for Modern Greek
by: Loukas, Lefteris, et al.
Published: (2024)

Guarded Repair for Harm-Aware Post-hoc Replacement of LLM Mathematical Reasoning
by: Xia, Haizhou
Published: (2026)

SWE-MERA: A Dynamic Benchmark for Agenticly Evaluating Large Language Models on Software Engineering Tasks
by: Adamenko, Pavel, et al.
Published: (2025)

SWE-Bench++: A Framework for the Scalable Generation of Software Engineering Benchmarks from Open-Source Repositories
by: Wang, Lilin, et al.
Published: (2025)

OmniCode: A Benchmark for Evaluating Software Engineering Agents
by: Sonwane, Atharv, et al.
Published: (2026)

VHDL-Eval: A Framework for Evaluating Large Language Models in VHDL Code Generation
by: Vijayaraghavan, Prashanth, et al.
Published: (2024)

FinReflectKG -- EvalBench: Benchmarking Financial KG with Multi-Dimensional Evaluation
by: Dimino, Fabrizio, et al.
Published: (2025)

BenchBrowser: Retrieving Evidence for Evaluating Benchmark Validity
by: Diddee, Harshita, et al.
Published: (2026)

Granite Code Models: A Family of Open Foundation Models for Code Intelligence
by: Mishra, Mayank, et al.
Published: (2024)

UA-Code-Bench: A Competitive Programming Benchmark for Evaluating LLM Code Generation in Ukrainian
by: Syromiatnikov, Mykyta, et al.
Published: (2025)

Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safety
by: Lee, Seongmin, et al.
Published: (2025)

PRISM: A Design Framework for Open-Source Foundation Model Safety
by: Neumann, Terrence, et al.
Published: (2024)

Towards Automated Smart Contract Generation: Evaluation, Benchmarking, and Retrieval-Augmented Repair
by: Chen, Zaoyu, et al.
Published: (2025)

Collu-Bench: A Benchmark for Predicting Language Model Hallucinations in Code
by: Jiang, Nan, et al.
Published: (2024)

A Case Study of Web App Coding with OpenAI Reasoning Models
by: Cui, Yi
Published: (2024)

SemCoder: Training Code Language Models with Comprehensive Semantics Reasoning
by: Ding, Yangruibo, et al.
Published: (2024)

Source Code Foundation Models are Transferable Binary Analysis Knowledge Bases
by: Su, Zian, et al.
Published: (2024)

Narrowing the Gap: Supervised Fine-Tuning of Open-Source LLMs as a Viable Alternative to Proprietary Models for Pedagogical Tools
by: Solano, Lorenzo Lee, et al.
Published: (2025)

Illocutionary Explanation Planning for Source-Faithful Explanations in Retrieval-Augmented Language Models
by: Sovrano, Francesco, et al.
Published: (2026)

Large Language Models (LLMs) for Source Code Analysis: applications, models and datasets
by: Jelodar, Hamed, et al.
Published: (2025)

CodeScope: An Execution-based Multilingual Multitask Multidimensional Benchmark for Evaluating LLMs on Code Understanding and Generation
by: Yan, Weixiang, et al.
Published: (2023)

Risk-Adjusted Harm Scoring for Automated Red Teaming for LLMs in Financial Services
by: Dimino, Fabrizio, et al.
Published: (2026)

OpenHands: An Open Platform for AI Software Developers as Generalist Agents
by: Wang, Xingyao, et al.
Published: (2024)

Beyond Correctness: Benchmarking Multi-dimensional Code Generation for Large Language Models
by: Zheng, Jiasheng, et al.
Published: (2024)

AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents
by: Trivedi, Harsh, et al.
Published: (2024)

Dynamic Benchmarking of Reasoning Capabilities in Code Large Language Models Under Data Contamination
by: Chen, Simin, et al.
Published: (2025)

A Comparative Study of DSPy Teleprompter Algorithms for Aligning Large Language Models Evaluation Metrics to Human Evaluation
by: Sarmah, Bhaskarjit, et al.
Published: (2024)

Comprehensive Analysis of Transparency and Accessibility of ChatGPT, DeepSeek, And other SoTA Large Language Models
by: Sapkota, Ranjan, et al.
Published: (2025)

IndustryCode: A Benchmark for Industry Code Generation
by: Zeng, Puyu, et al.
Published: (2026)

TRUSTVIS: A Multi-Dimensional Trustworthiness Evaluation Framework for Large Language Models
by: Sun, Ruoyu, et al.
Published: (2025)

E2Edev: Benchmarking Large Language Models in End-to-End Software Development Task
by: Liu, Jingyao, et al.
Published: (2025)

Multi-SWE-bench: A Multilingual Benchmark for Issue Resolving
by: Zan, Daoguang, et al.
Published: (2025)

PrimeGuard: Safe and Helpful LLMs through Tuning-Free Routing
by: Manczak, Blazej, et al.
Published: (2024)

ICE-Score: Instructing Large Language Models to Evaluate Code
by: Zhuo, Terry Yue
Published: (2023)