Cover Image

Counterfactual Evaluation for Blind Attack Detection in LLM-based Evaluation Systems

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liu, Lijia, Kondo, Takumi, Atarashi, Kyohei, Takeuchi, Koh, Li, Jiyi, Saito, Shigeru, Kashima, Hisashi
Format:	Preprint
Published:	2025
Subjects:	Cryptography and Security Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Emulating Retrieval Augmented Generation via Prompt Engineering for Enhanced Long Context Comprehension in LLMs
by: Park, Joon, et al.
Published: (2025)

AHP-Powered LLM Reasoning for Multi-Criteria Evaluation of Open-Ended Responses
by: Lu, Xiaotian, et al.
Published: (2024)

PandaGuard: Systematic Evaluation of LLM Safety against Jailbreaking Attacks
by: Shen, Guobin, et al.
Published: (2025)

Exploring Causes and Mitigation of Hallucinations in Large Vision Language Models
by: Sun, Yaqi, et al.
Published: (2025)

On the Hidden Costs of Counterfactual Knowledge Training in LLM Unlearning
by: Ye, Xiaotian, et al.
Published: (2026)

Security Attacks on LLM-based Code Completion Tools
by: Cheng, Wen, et al.
Published: (2024)

Attack and Defense of Deep Learning Models in the Field of Web Attack Detection
by: Shi, Lijia, et al.
Published: (2024)

Personalized Attacks of Social Engineering in Multi-turn Conversations: LLM Agents for Simulation and Detection
by: Kumarage, Tharindu, et al.
Published: (2025)

Defending LLM Watermarking Against Spoofing Attacks with Contrastive Representation Learning
by: An, Li, et al.
Published: (2025)

Can Federated Learning Safeguard Private Data in LLM Training? Vulnerabilities, Attacks, and Defense Evaluation
by: Guo, Wenkai, et al.
Published: (2025)

Blind Spots in the Guard: How Domain-Camouflaged Injection Attacks Evade Detection in Multi-Agent LLM Systems
by: Pai, Aaditya
Published: (2026)

Robust Anomaly Detection Under Normality Distribution Shift in Dynamic Graphs
by: Xu, Xiaoyang, et al.
Published: (2025)

Adversarial Attacks on LLM-as-a-Judge Systems: Insights from Prompt Injections
by: Maloyan, Narek, et al.
Published: (2025)

Trust No Tool: Evaluating and Defending LLM Agents under Untrusted Tool Feedback
by: Yan, Lecheng, et al.
Published: (2026)

Model-Agnostic Lifelong LLM Safety via Externalized Attack-Defense Co-Evolution
by: Zhang, Xiaozhe, et al.
Published: (2026)

Evaluating Cross-Modal Reasoning Ability and Problem Characteristics with Multimodal Item Response Theory
by: Uebayashi, Shunki, et al.
Published: (2026)

Prompt Optimization and Evaluation for LLM Automated Red Teaming
by: Freenor, Michael, et al.
Published: (2025)

GuidedBench: Measuring and Mitigating the Evaluation Discrepancies of In-the-wild LLM Jailbreak Methods
by: Huang, Ruixuan, et al.
Published: (2025)

AdaPPA: Adaptive Position Pre-Fill Jailbreak Attack Approach Targeting LLMs
by: Lv, Lijia, et al.
Published: (2024)

Rethinking Backdoor Detection Evaluation for Language Models
by: Yan, Jun, et al.
Published: (2024)

Iron Sharpens Iron: Defending Against Attacks in Machine-Generated Text Detection with Adversarial Training
by: Li, Yuanfan, et al.
Published: (2025)

Factuality Beyond Coherence: Evaluating LLM Watermarking Methods for Medical Texts
by: Hastuti, Rochana Prih, et al.
Published: (2025)

Blind Baselines Beat Membership Inference Attacks for Foundation Models
by: Das, Debeshee, et al.
Published: (2024)

Evaluation of LLM Chatbots for OSINT-based Cyber Threat Awareness
by: Shafee, Samaneh, et al.
Published: (2024)

ProvX: Generating Counterfactual-Driven Attack Explanations for Provenance-Based Detection
by: Wu, Weiheng, et al.
Published: (2025)

BiAxisAudit: A Novel Framework to Evaluate LLM Bias Across Prompt Sensitivity and Response-Layer Divergence
by: Gan, Jialing, et al.
Published: (2026)

IP Leakage Attacks Targeting LLM-Based Multi-Agent Systems
by: Wang, Liwen, et al.
Published: (2025)

CVE-LLM : Ontology-Assisted Automatic Vulnerability Evaluation Using Large Language Models
by: Ghosh, Rikhiya, et al.
Published: (2025)

RLCracker: Evaluating the Worst-Case Vulnerability of LLM Watermarks with Adaptive RL Attacks
by: Huang, Hanbo, et al.
Published: (2025)

Mitigating Fine-tuning based Jailbreak Attack with Backdoor Enhanced Safety Alignment
by: Wang, Jiongxiao, et al.
Published: (2024)

SecurityLingua: Efficient Defense of LLM Jailbreak Attacks via Security-Aware Prompt Compression
by: Li, Yucheng, et al.
Published: (2025)

Talk is (Not) Cheap: A Taxonomy and Benchmark Coverage Audit for LLM Attacks
by: Iyer, Karthik Raghu, et al.
Published: (2026)

AgentShield: Deception-based Compromise Detection for Tool-using LLM Agents
by: Rassul, Yassin H., et al.
Published: (2026)

PsySafe: A Comprehensive Framework for Psychological-based Attack, Defense, and Evaluation of Multi-agent System Safety
by: Zhang, Zaibin, et al.
Published: (2024)

FlexLLM: Exploring LLM Customization for Moving Target Defense on Black-Box LLMs Against Jailbreak Attacks
by: Chen, Bocheng, et al.
Published: (2024)

Self-Evaluation as a Defense Against Adversarial Attacks on LLMs
by: Brown, Hannah, et al.
Published: (2024)

RedTWIZ: Diverse LLM Red Teaming via Adaptive Attack Planning
by: Horal, Artur, et al.
Published: (2025)

Self-Disguise Attack: Induce the LLM to disguise itself for AIGT detection evasion
by: Zhou, Yinghan, et al.
Published: (2025)

LoopLLM: Transferable Energy-Latency Attacks in LLMs via Repetitive Generation
by: Li, Xingyu, et al.
Published: (2025)

MGTEVAL: An Interactive Platform for Systemtic Evaluation of Machine-Generated Text Detectors
by: Li, Yuanfan, et al.
Published: (2026)