Recent research has shown that every ML model is vulnerable to multiple security and privacy attacks
Machine learning has made tremendous progress during the past decade and is being adopted in various critical real-world applications. However, recent research has shown that every ML model is vulnerable to multiple security and privacy attacks. In particular, backdoor attacks against ML models have recently raised awareness. A successful backdoor attack can cause severe consequences, such as allowing an adversary to bypass critical authentication systems.
The security of machine learning is becoming increasingly critical as ML models find their way into a growing number of applications. The new study focuses on the security threats of delegating the training and development of machine learning models to third parties and service providers.
What are Backdoor Attacks?
Backdoor attacks insert hidden associations or triggers to the machine learning models to override correct inferences such as classification and make the system perform maliciously according to the attacker-chosen target while behaving normally in the absence of the trigger. As a new and rapidly evolving realistic attack, it could result in dire consequences, especially considering that the backdoor attack surfaces are broad.
Machine learning models are increasingly deployed to make decisions on our behalf on various (mission-critical) tasks such as computer vision, disease diagnosis, financial fraud detection, defending against malware and cyber-attacks, access control, and surveillance. However, there are realistic security threats against a deployed ML system. One well-known attack is the adversarial example, where an imperceptible or semantically consistent manipulation of inputs, e.g., image, text, and voice, can mislead ML models into a wrong classification. To make matters worse, the adversarial example is not the only threat. As written by Ian Goodfellow and Nicolas in 2017 and many other kinds of attacks are possible, such as attacks based on surreptitiously modifying the training data to cause the model to learn to behave the way the attacker wishes it to behave.” The recent whirlwind backdoor attacks against deep learning models (deep neural networks (DNNs)), exactly fit such insidious adversarial purposes.
Machine Learning Backdoor
Current backdooring techniques rely on adding static triggers (with fixed patterns and locations) on ML model inputs which are prone to detection by the current backdoor detection mechanisms. In a research paper of Cornell University, Ahmed Salem, Rui Wen, Michael Backes, Shiqing Ma, and Yang Zhang propose the first class of dynamic backdooring techniques against deep neural networks (DNN), namely Random Backdoor, Backdoor Generating Network (BaN), and conditional Backdoor Generating Network (c-BaN). Triggers generated by their techniques can have random patterns and locations, which reduce the efficacy of the current backdoor detection mechanisms. In particular, BaN and c-BaN based on a novel generative network are the first two schemes that algorithmically generate triggers. Moreover, c-BaN is the first conditional backdooring technique that given a target label, can generate a target-specific trigger. Both BaN and c-BaN are essentially a general framework that renders the adversary, the flexibility for further customizing backdoor attacks.
The techniques are extensively evaluated by them on three benchmark datasets: MNIST, CelebA, and CIFAR-10. Their techniques achieve almost perfect attack performance on backdoored data with a negligible utility loss. They further show that their techniques can bypass current state-of-the-art defense mechanisms against backdoor attacks, including ABS, Februus, MNTD, Neural Cleanse, and STRIP.
Most ML backdooring techniques come with a performance tradeoff on the model’s main task. If the model’s performance on the main task degrades too much, the victim will either become suspicious or refrain from using it because it doesn’t meet the required performance.
Source: analyticsinsight.net
