On the (In)feasibility of ML Backdoor Detection as an Hypothesis Testing Problem
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909120882278400 |
|---|---|
| author | Pichler, Georg Romanelli, Marco Manivannan, Divya Prakash Krishnamurthy, Prashanth Khorrami, Farshad Garg, Siddharth |
| author_facet | Pichler, Georg Romanelli, Marco Manivannan, Divya Prakash Krishnamurthy, Prashanth Khorrami, Farshad Garg, Siddharth |
| contents | We introduce a formal statistical definition for the problem of backdoor detection in machine learning systems and use it to analyze the feasibility of such problems, providing evidence for the utility and applicability of our definition. The main contributions of this work are an impossibility result and an achievability result for backdoor detection. We show a no-free-lunch theorem, proving that universal (adversary-unaware) backdoor detection is impossible, except for very small alphabet sizes. Thus, we argue, that backdoor detection methods need to be either explicitly, or implicitly adversary-aware. However, our work does not imply that backdoor detection cannot work in specific scenarios, as evidenced by successful backdoor detection methods in the scientific literature. Furthermore, we connect our definition to the probably approximately correct (PAC) learnability of the out-of-distribution detection problem. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_16926 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | On the (In)feasibility of ML Backdoor Detection as an Hypothesis Testing Problem Pichler, Georg Romanelli, Marco Manivannan, Divya Prakash Krishnamurthy, Prashanth Khorrami, Farshad Garg, Siddharth Cryptography and Security Artificial Intelligence Machine Learning We introduce a formal statistical definition for the problem of backdoor detection in machine learning systems and use it to analyze the feasibility of such problems, providing evidence for the utility and applicability of our definition. The main contributions of this work are an impossibility result and an achievability result for backdoor detection. We show a no-free-lunch theorem, proving that universal (adversary-unaware) backdoor detection is impossible, except for very small alphabet sizes. Thus, we argue, that backdoor detection methods need to be either explicitly, or implicitly adversary-aware. However, our work does not imply that backdoor detection cannot work in specific scenarios, as evidenced by successful backdoor detection methods in the scientific literature. Furthermore, we connect our definition to the probably approximately correct (PAC) learnability of the out-of-distribution detection problem. |
| title | On the (In)feasibility of ML Backdoor Detection as an Hypothesis Testing Problem |
| topic | Cryptography and Security Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2402.16926 |