Cover Image

Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Dong, Yihong, Jiang, Xue, Liu, Huanyu, Jin, Zhi, Gu, Bin, Yang, Mengfei, Li, Ge
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence Cryptography and Security Machine Learning Software Engineering
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Automated Trustworthiness Oracle Generation for Machine Learning Text Classifiers
by: Tung, Lam Nguyen, et al.
Published: (2024)

Evaluation of the Programming Skills of Large Language Models
by: Heitz, Luc Bryan, et al.
Published: (2024)

VulInstruct: Teaching LLMs Root-Cause Reasoning for Vulnerability Detection via Security Specifications
by: Zhu, Hao, et al.
Published: (2025)

Concerned with Data Contamination? Assessing Countermeasures in Code Language Model
by: Cao, Jialun, et al.
Published: (2024)

Evaluating and Mitigating Linguistic Discrimination in Large Language Models
by: Dong, Guoliang, et al.
Published: (2024)

Large Language Models Cannot Reliably Detect Vulnerabilities in JavaScript: The First Systematic Benchmark and Evaluation
by: Fei, Qingyuan, et al.
Published: (2025)

VulnRepairEval: An Exploit-Based Evaluation Framework for Assessing Large Language Model Vulnerability Repair Capabilities
by: Wang, Weizhe, et al.
Published: (2025)

Understanding the Effectiveness of Large Language Models in Detecting Security Vulnerabilities
by: Khare, Avishree, et al.
Published: (2023)

PoLLMgraph: Unraveling Hallucinations in Large Language Models via State Transition Dynamics
by: Zhu, Derui, et al.
Published: (2024)

Evaluating Large Language Models for Line-Level Vulnerability Localization
by: Zhang, Jian, et al.
Published: (2024)

AutoTestForge: A Multidimensional Automated Testing Framework for Natural Language Processing Models
by: Xing, Hengrui, et al.
Published: (2025)

From Solitary Directives to Interactive Encouragement! LLM Secure Code Generation by Natural Language Prompting
by: Liu, Shigang, et al.
Published: (2024)

Decoding Secret Memorization in Code LLMs Through Token-Level Characterization
by: Nie, Yuqing, et al.
Published: (2024)

Belobog: Move Language Fuzzing Framework For Real-World Smart Contracts
by: Kong, Ziqiao, et al.
Published: (2025)

Scrub It Out! Erasing Sensitive Memorization in Code Language Models via Machine Unlearning
by: Chu, Zhaoyang, et al.
Published: (2025)

TAIBOM: Bringing Trustworthiness to AI-Enabled Systems
by: Safronov, Vadim, et al.
Published: (2025)

Evaluating Large Language Models in detecting Secrets in Android Apps
by: Alecci, Marco, et al.
Published: (2025)

Engineering Trustworthy Machine-Learning Operations with Zero-Knowledge Proofs
by: Scaramuzza, Filippo, et al.
Published: (2025)

A Broad Comparative Evaluation of Software Debloating Tools
by: Brown, Michael D., et al.
Published: (2023)

Harnessing Large Language Models for Seed Generation in Greybox Fuzzing
by: Shi, Wenxuan, et al.
Published: (2024)

The Secrets Must Not Flow: Scaling Security Verification to Large Codebases (extended version)
by: Arquint, Linard, et al.
Published: (2025)

Efficient Detection of Toxic Prompts in Large Language Models
by: Liu, Yi, et al.
Published: (2024)

Reflection-Driven Control for Trustworthy Code Agents
by: Wang, Bin, et al.
Published: (2025)

RepoMark: A Data-Usage Auditing Framework for Code Large Language Models
by: Qu, Wenjie, et al.
Published: (2025)

YASA: Scalable Multi-Language Taint Analysis on the Unified AST at Ant Group
by: Wang, Yayi, et al.
Published: (2026)

An Empirical Security Evaluation of LLM-Generated Cryptographic Rust Code
by: Elsayed, Mohamed, et al.
Published: (2026)

Argus: Reorchestrating Static Analysis via a Multi-Agent Ensemble for Full-Chain Security Vulnerability Detection
by: Liang, Zi, et al.
Published: (2026)

OpenTracer: A Dynamic Transaction Trace Analyzer for Smart Contract Invariant Generation and Beyond
by: Chen, Zhiyang, et al.
Published: (2024)

GenSIaC: Toward Security-Aware Infrastructure-as-Code Generation with Large Language Models
by: Li, Yikun, et al.
Published: (2025)

Unknown Attack Detection in IoT Networks using Large Language Models: A Robust, Data-efficient Approach
by: Ali, Shan, et al.
Published: (2026)

Exploring Data-Efficient Adaptation of Large Language Models for Code Generation
by: Jiang, Xue, et al.
Published: (2024)

SmartInv: Multimodal Learning for Smart Contract Invariant Inference
by: Wang, Sally Junsong, et al.
Published: (2024)

Tactics, Techniques, and Procedures (TTPs) in Interpreted Malware: A Zero-Shot Generation with Large Language Models
by: Zhang, Ying, et al.
Published: (2024)

Automated Generation of Accurate Privacy Captions From Android Source Code Using Large Language Models
by: Jain, Vijayanta, et al.
Published: (2026)

PACE: Improving Prompt with Actor-Critic Editing for Large Language Model
by: Dong, Yihong, et al.
Published: (2023)

AC4: Algebraic Computation Checker for Circuit Constraints in ZKPs
by: Yang, Qizhe, et al.
Published: (2024)

Detecting Malicious Intents in Smart Contracts with Pre-trained Programming Language Models
by: Huang, Youwei, et al.
Published: (2025)

Accelerating Automatic Program Repair with Dual Retrieval-Augmented Fine-Tuning and Patch Generation on Large Language Models
by: Guo, Hanyang, et al.
Published: (2025)

Probing Privacy Leaks in LLM-based Code Generation via Test Generation
by: Ge, Yifei, et al.
Published: (2026)

DLAP: A Deep Learning Augmented Large Language Model Prompting Framework for Software Vulnerability Detection
by: Yang, Yanjing, et al.
Published: (2024)