Image de couverture de livre

Cheating Automatic LLM Benchmarks: Null Models Achieve High Win Rates

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Zheng, Xiaosen, Pang, Tianyu, Du, Chao, Liu, Qian, Jiang, Jing, Lin, Min
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language Artificial Intelligence Cryptography and Security Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Improved Few-Shot Jailbreaking Can Circumvent Aligned Language Models and Their Defenses
par: Zheng, Xiaosen, et autres
Publié: (2024)

Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast
par: Gu, Xiangming, et autres
Publié: (2024)

Improving Your Model Ranking on Chatbot Arena by Vote Rigging
par: Min, Rui, et autres
Publié: (2025)

Denial-of-Service Poisoning Attacks against Large Language Models
par: Gao, Kuofeng, et autres
Publié: (2024)

Benchmarking Large Multimodal Models against Common Corruptions
par: Zhang, Jiawei, et autres
Publié: (2024)

Improved Techniques for Optimization-Based Jailbreaking on Large Language Models
par: Jia, Xiaojun, et autres
Publié: (2024)

Test-Time Backdoor Attacks on Multimodal Large Language Models
par: Lu, Dong, et autres
Publié: (2024)

GenBreak: Red Teaming Text-to-Image Generators Using Large Language Models
par: Wang, Zilong, et autres
Publié: (2025)

How Much Do Large Language Model Cheat on Evaluation? Benchmarking Overestimation under the One-Time-Pad-Based Framework
par: Liang, Zi, et autres
Publié: (2025)

Compiling Activation Steering into Weights via Null-Space Constraints for Stealthy Backdoors
par: Yin, Rui, et autres
Publié: (2026)

Lifelong Safety Alignment for Language Models
par: Wang, Haoyu, et autres
Publié: (2025)

Meta-Unlearning on Diffusion Models: Preventing Relearning Unlearned Concepts
par: Gao, Hongcheng, et autres
Publié: (2024)

Imperceptible Jailbreaking against Large Language Models
par: Gao, Kuofeng, et autres
Publié: (2025)

CVE-LLM : Ontology-Assisted Automatic Vulnerability Evaluation Using Large Language Models
par: Ghosh, Rikhiya, et autres
Publié: (2025)

LITMUS: Benchmarking Behavioral Jailbreaks of LLM Agents in Real OS Environments
par: Zhang, Chiyu, et autres
Publié: (2026)

Task-Agnostic Detector for Insertion-Based Backdoor Attacks
par: Lyu, Weimin, et autres
Publié: (2024)

CI-Work: Benchmarking Contextual Integrity in Enterprise LLM Agents
par: Fu, Wenjie, et autres
Publié: (2026)

IPIGuard: A Novel Tool Dependency Graph-Based Defense Against Indirect Prompt Injection in LLM Agents
par: An, Hengyu, et autres
Publié: (2025)

Raccoon: Prompt Extraction Benchmark of LLM-Integrated Applications
par: Wang, Junlin, et autres
Publié: (2024)

Factuality Beyond Coherence: Evaluating LLM Watermarking Methods for Medical Texts
par: Hastuti, Rochana Prih, et autres
Publié: (2025)

NSmark: Null Space Based Black-box Watermarking Defense Framework for Language Models
par: Zhao, Haodong, et autres
Publié: (2024)

No Free Lunch in LLM Watermarking: Trade-offs in Watermarking Design Choices
par: Pang, Qi, et autres
Publié: (2024)

HoneyWin: High-Interaction Windows Honeypot in Enterprise Environment
par: Aung, Yan Lin, et autres
Publié: (2025)

PRISM: Privacy-Aware Routing for Adaptive Cloud-Edge LLM Inference via Semantic Sketch Collaboration
par: Zhan, Junfei, et autres
Publié: (2025)

Sparse Tokens Suffice: Jailbreaking Audio Language Models via Token-Aware Gradient Optimization
par: Fang, Zheng, et autres
Publié: (2026)

Talk is (Not) Cheap: A Taxonomy and Benchmark Coverage Audit for LLM Attacks
par: Iyer, Karthik Raghu, et autres
Publié: (2026)

ML-Bench&Guard: Policy-Grounded Multilingual Safety Benchmark and Guardrail for Large Language Models
par: Zhao, Yunhan, et autres
Publié: (2026)

T2ISafety: Benchmark for Assessing Fairness, Toxicity, and Privacy in Image Generation
par: Li, Lijun, et autres
Publié: (2025)

CheatAgent: Attacking LLM-Empowered Recommender Systems via LLM Agent
par: Ning, Liang-bo, et autres
Publié: (2025)

Watermarking LLM Agent Trajectories
par: Meng, Wenlong, et autres
Publié: (2026)

Why LLM Safety Guardrails Collapse After Fine-tuning: A Similarity Analysis Between Alignment and Fine-tuning Datasets
par: Hsiung, Lei, et autres
Publié: (2025)

Confidential Prompting: Privacy-preserving LLM Inference on Cloud
par: Li, Caihua, et autres
Publié: (2024)

StructuralSleight: Automated Jailbreak Attacks on Large Language Models Utilizing Uncommon Text-Organization Structures
par: Li, Bangxin, et autres
Publié: (2024)

XGuardian: Towards Explainable and Generalized AI Anti-Cheat on FPS Games
par: Zhang, Jiayi, et autres
Publié: (2026)

Out of the Cage: How Stochastic Parrots Win in Cyber Security Environments
par: Rigaki, Maria, et autres
Publié: (2023)

SciSafeEval: A Comprehensive Benchmark for Safety Alignment of Large Language Models in Scientific Tasks
par: Li, Tianhao, et autres
Publié: (2024)

Are All Prompt Components Value-Neutral? Understanding the Heterogeneous Adversarial Robustness of Dissected Prompt in Large Language Models
par: Zheng, Yujia, et autres
Publié: (2025)

Auto-RT: Automatic Jailbreak Strategy Exploration for Red-Teaming Large Language Models
par: Liu, Yanjiang, et autres
Publié: (2025)

PrivLM-Bench: A Multi-level Privacy Evaluation Benchmark for Language Models
par: Li, Haoran, et autres
Publié: (2023)

Prediction Inconsistency Helps Achieve Generalizable Detection of Adversarial Examples
par: Han, Sicong, et autres
Publié: (2025)