What should we learn from the One Pixel Attack?
The paper that shook costly Deep Learning Image Classifiers
If you have been alive recently, you are likely aware of the Deep Learning Hype. People are convinced that with enough Computing Power and Data, DNNs will solve any problem. By this point, DNNs have established themselves as reliable Image Classifiers. This is why the paper “One Pixel Attack for Fooling Deep Neural Networks” by Jiawei Su et al should not be ignored. By changing only one pixel, they are able to reliably fool expensive DNNs across different architectures. The image on the left shows the results using various metrics on the different metrics. As we can see, the results are exceptional. Even the perturbation protocol used is extremely inexpensive. If you want to learn more about the technique, check out this fantastic article: Why you should be using Differential Evolution for your optimization problems. It breaks down the technique used to select the pixel to be used. To summarise though: DE is cheap, can be applied to almost any problem, and can be customized easily to match any configuration.
In this article, I will be breaking down the major takeaways from this groundbreaking paper. We will discuss the implementation, possible extensions, and results. We will also go into interesting techniques used in the evaluation. As always, the annotated paper is attached at the end. Be sure to check it out for more details and a super thorough breakdown. If this article was useful/interesting for you, please be sure to clap and share it. Additionally, any feedback will be appreciated.
A quick note on this paper matters
Some of you might be wondering why this paper is significant. After all, what can we do by changing 1 Pixel? Aside from being used as a potential benchmark for future image classification models, the One Pixel attack has profound implications for our privacy. This attack could be used as a way to stop facial recognition software from gathering information without our consent. Since the changes are imperceptible to human eyes, any images could still be used by humans in times of need, but mass automated surveillance could be prevented. By understanding the nuances of this attack, how it can be improved, and why it works a certain way, we can improve our results and gain a better understanding of the learning processes behind complex Neural Networks.
Experiment Setup
The experiment design was elegant and something we can learn from. The experiment used multiple datasets including Kaggle CIFAR 10, Original CIFAR 10, and the ImageNet dataset All the datasets used are industry standard in Image Related works. Furthermore, the use of multiple datasets highlights the generality of the results.
The images were taken from multiple different classes (what the image represents). The model was trained to recognize and label an image into one of the classes. Once the image classification was trained, they attacked the model. The attacks were both targeted and non-targeted. Targeted attacks attempted to make the model misclassify the image as a specific different class, while non targeted attacks just tried to fool the classifier.
To understand this take for example a fictional simple dataset containing 3 types of images cats, dogs, and rats. Take a model trained on our dataset. A targeted one-pixel attack one a cat pic might try to fool the model that the image is a rat. While a non-targeted attack will just try to fool the model that the image is not a cat.
How did they evaluate the results of changing a pixel?
With a lot of Machine Learning Research, it is important to look into the metrics used to evaluate the results. By understanding the metrics, we can evaluate a piece of research and its utility for potential solutions. We can look at the possible downsides of using the method, and the net scope. The research team used 4 different metrics for evaluating the results.
- Success Rate: Simply put this was the percentage of images of one class that could be changed to another. This is the simplest one to understand and its use is fairly obvious.
- Confidence: The average confidence given by the target system when misclassifying adversarial images. This is calculated as “Accumulates the values of probability label of the target class for each successful perturbation, then divided by the total number of successful perturbations.” The important thing is that this only looks at the successful cases.
- Number of Target Classes: Counts the number of natural images that successfully perturb to a certain number of target classes. “In particular, by counting the number of images that can not be perturbed to any other classes, the effectiveness of non-targeted attack can be evaluated.”
- Number of Original-Target Class Pairs: Counts the number of times each original-destination class pair was attacked. This puts the other metrics into perspective.
Interesting Results/Takeaways
This paper carries some interesting implications and learnings that one should not ignore. What I found interesting was:
Effective ML research can be conducted on lower resources
It feels like ML Research (especially on Images) is becoming a game of computing power. Teams use thousands of computing hours and datasets of a colossal magnitude. New protocols are often extremely expensive. Which is why techniques like Differential Evolution and RandAugment always excite me. These paper costs peanuts. Take a look at the table below to understand.
The costs of evaluation are super low. Most strong computers could run them without any problems. The fact that this paper was able to trick the costly DNN image classifiers with such low costs speaks volumes of the need for more diversity in terms of research directions and ideas.
We need to understand Neural Nets Better
This isn’t a new take. But this paper truly showcased the need to understand the black box learning process of DNNs. While most of the morphs were symmetric (if class A can be morphed to class B, then vice versa.), the paper revealed that there some classes that were easier to morph into others. For example with the AllConv Network, images of dogs could be misclassified as other things (including a truck?a frog?). This is obviously a curious result that warrants further investigation. By understanding the why behind this (or at least the how), we could potentially unlock the potential of ML.
Extensions
This paper has some interesting directions that might be explored. The paper does bring up several of them, but what struck me was the low quality of images. Using HD images would make the research more directly applicable to the real world. It would also validate the results for a world with an HD camera in every pocket.
Paper
As promised here is the paper with annotations from me. Feel free to look over this in leisure to go over anything that I didn’t include. Be sure to comment anything that you find interesting
Reach out to me
Thank you for reading this. I am dropping all my relevant social media below. Follow any (or all) to see my content across different platforms. I like to use the strengths of different platforms. Leave any feedback you might have, as it really helps a growing content creator like myself. If you found this useful, please share the article.
I’ve shortened the URLs using this great service. They do great work, so show them some love. This is not sponsored, but it’s always good to promote useful work.
Check out my other articles on Medium. : https://rb.gy/zn1aiu
My YouTube. It’s a work in progress haha: https://rb.gy/88iwdd
Reach out to me on LinkedIn. Let’s connect: https://rb.gy/m5ok2y
My Twitter: https://twitter.com/Machine01776819
My Substack: https://devanshacc.substack.com/
If you would like to work with me email me: devanshverma425@gmail.com
Live conversations at twitch here: https://rb.gy/zlhk9y
To get updates on my content- Instagram: https://rb.gy/gmvuy9
Get a free stock on Robinhood: https://join.robinhood.com/fnud75
