Cover Image

Supporting Artifact Evaluation with LLMs: A Study with Published Security Research Papers

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Heye, David, Kindermann, Karl, Decker, Robin, Lohmöller, Johannes, Belova, Anastasiia, Geisler, Sandra, Wehrle, Klaus, Pennekamp, Jan
Format:	Preprint
Published:	2026
Subjects:	Cryptography and Security Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

PASTA-4-PHT: A Pipeline for Automated Security and Technical Audits for the Personal Health Train
by: Welten, Sascha, et al.
Published: (2024)

Hidden Secrets in the arXiv: Discovering, Analyzing, and Preventing Unintentional Information Disclosure in Source Files of Scientific Preprints
by: Pennekamp, Jan, et al.
Published: (2026)

MAC Aggregation over Lossy Channels in DTLS 1.3
by: Wagner, Eric, et al.
Published: (2025)

On the Security of Research Artifacts
by: Rani, Nanda, et al.
Published: (2026)

CAIBA: Multicast Source Authentication for CAN Through Reactive Bit Flipping
by: Wagner, Eric, et al.
Published: (2025)

Deployment Challenges of Industrial Intrusion Detection Systems
by: Wolsing, Konrad, et al.
Published: (2024)

Exploring Advanced Methodologies in Security Evaluation for LLMs
by: Huang, Jun, et al.
Published: (2024)

An Empirical Evaluation of LLMs for Solving Offensive Security Challenges
by: Shao, Minghao, et al.
Published: (2024)

Hooked: A Real-World Study on QR Code Phishing
by: Geisler, Marvin, et al.
Published: (2024)

Revisiting the Robust Alignment of Circuit Breakers
by: Schwinn, Leo, et al.
Published: (2024)

An Interdisciplinary Survey on Information Flows in Supply Chains
by: Pennekamp, Jan, et al.
Published: (2023)

CoFacS -- Simulating a Complete Factory to Study the Security of Interconnected Production
by: Lenz, Stefan, et al.
Published: (2025)

LLMs, You Can Evaluate It! Design of Multi-perspective Report Evaluation for Security Operation Centers
by: Okada, Hiroyuki, et al.
Published: (2026)

Before You Hand Over the Wheel: Evaluating LLMs for Security Incident Analysis
by: Jajodia, Sourov, et al.
Published: (2026)

Writing a Good Security Paper for ISSCC (2025)
by: Banerjee, Utsav, et al.
Published: (2025)

CTI-REALM: Benchmark to Evaluate Agent Performance on Security Detection Rule Generation Capabilities
by: Chakraborty, Arjun, et al.
Published: (2026)

Unconsidered Installations: Discovering IoT Deployments in the IPv6 Internet
by: Dahlmanns, Markus, et al.
Published: (2024)

A Secure, Confidential, and Verifiable Decision Support System
by: Marangone, Edoardo, et al.
Published: (2025)

LLMs Cannot Reliably Identify and Reason About Security Vulnerabilities (Yet?): A Comprehensive Evaluation, Framework, and Benchmarks
by: Ullah, Saad, et al.
Published: (2023)

Toward an Intent-Based and Ontology-Driven Autonomic Security Response in Security Orchestration Automation and Response
by: Huang, Zequan, et al.
Published: (2025)

A Survey on Security with Quantum Computing
by: Sangala, Manik Kumar, et al.
Published: (2026)

An Approach for Safe and Secure Software Protection Supported by Symbolic Execution
by: Dorfmeister, Daniel, et al.
Published: (2026)

Prompt Injection Evaluations: Refusal Boundary Instability and Artifact-Dependent Compliance in GPT-4-Series Models
by: Heverin, Thomas
Published: (2026)

Using LLMs for Tabletop Exercises within the Security Domain
by: Hays, Sam, et al.
Published: (2024)

Sentry: Authenticating Machine Learning Artifacts on the Fly
by: Gan, Andrew, et al.
Published: (2025)

Can Developers rely on LLMs for Secure IaC Development?
by: Firouzi, Ehsan, et al.
Published: (2026)

LLM-Safety Evaluations Lack Robustness
by: Beyer, Tim, et al.
Published: (2025)

Inference Attacks on Encrypted Online Voting via Traffic Analysis
by: Belousova, Anastasiia, et al.
Published: (2025)

From LLMs to Agents: A Comparative Evaluation of LLMs and LLM-based Agents in Security Patch Detection
by: Han, Junxiao, et al.
Published: (2025)

Generative Model Watermarking Suppressing High-Frequency Artifacts
by: Zhang, Li, et al.
Published: (2023)

Evaluating Software Supply Chain Security in Research Software
by: Hegewald, Richard, et al.
Published: (2025)

GuardPhish: Securing Open-Source LLMs from Phishing Abuse
by: Mishra, Rina, et al.
Published: (2026)

Chasing Shadows: Pitfalls in LLM Security Research
by: Evertz, Jonathan, et al.
Published: (2025)

LLMs in the SOC: An Empirical Study of Human-AI Collaboration in Security Operations Centres
by: Singh, Ronal, et al.
Published: (2025)

Using LLMs to Automate Threat Intelligence Analysis Workflows in Security Operation Centers
by: Tseng, PeiYu, et al.
Published: (2024)

Can We Trust Large Language Models Generated Code? A Framework for In-Context Learning, Security Patterns, and Code Evaluations Across Diverse LLMs
by: Mohsin, Ahmad, et al.
Published: (2024)

Supporting Secured Integration of Microarchitectural Defenses
by: Ramkrishnan, Kartik, et al.
Published: (2026)

A Systematic Evaluation of Parameter-Efficient Fine-Tuning Methods for the Security of Code LLMs
by: Lee, Kiho, et al.
Published: (2025)

A Systematic Security Analysis for Path-based Traceability Systems in RFID-Enabled Supply Chains
by: Heikamp, Fokke, et al.
Published: (2026)

Persistent Human Feedback, LLMs, and Static Analyzers for Secure Code Generation and Vulnerability Detection
by: Firouzi, Ehsan, et al.
Published: (2026)