I tried Google’s new Python fuzzer: Atheris
Fuzzing has been one of the most popular methods of automating bug finding in recent time. While fuzzers like American Fuzzy Lop, Honggfuzz etc. focus mainly on C/C++ code, there wasn’t a breakthrough for Python until now. Recently, Google opensourced their Python fuzzer Atheris!
You can read their blogpost here: Announcing Atheris Python Fuzzer
Before I dive in, let me quickly explain couple of things about fuzzing. Every time I tell my friends that I work on fuzzing/with fuzzers, 8 out of 10 times I get the reply “oh yeah, I know fuzzy logic”. If you are one of those people, please spend few minutes of your time to read some of the definitions in the section below before moving on, else you may skip to the next section.
Fuzzing/Fuzzer: What and types?
Fuzzing or fuzz testing is an effective way of finding bugs or vulnerabilities in the software. The program which is used to fuzz is called a fuzzer and the program being fuzzed is our target. A fuzzer typically starts feeding the target program with random inputs while observing its behaviour. Whenever the target crashes, the fuzzer reports the input which caused the crash to the user as a bug.
There are different kinds of fuzzing:
Black-box fuzzing: Fuzzing where there is no information about the code being fuzzed.
White-box fuzzing: Fuzzers here know the complete structure of the target and mainly follow symbolic execution.
Grey-box fuzzing: Fuzzers know partial information about the target and mainly measure the code coverage.
The interesting part of grey-box fuzzing is, whenever the fuzzer finds a new input which generates unique code coverage, it saves this input to further mutate it in next iterations. This is called as coverage-guided fuzzing.
Atheris: The Python fuzzer
Atheris is one of the first coverage-guided Python fuzzers. It means, the fuzzer measures the code coverage and observers the target program during execution and notes down inputs which result in unique execution behaviour.
Atheris is a high performace fuzzer that supports both native and pure-Python fuzzing. But the feature I cannot wait to explore is its differential fuzzing! Here as the name indicates, the fuzzer tries to find the difference in behaviour of two different libraries intended to do the same thing!!
(I will write a separate post about differential fuzzing.)
Installation
I tried Atheris on my Linux machine and installing is simple.
pip install atherisYup that’s it. If you are on MacOS (will try soon), you need to install fully fledged Clang and not the crippled one Apple ships as Artheris leverages the libFuzzer for code coverage and input generation.
The installation instructions on the Atheris github is simple and easy to follow.
Let’s start fuzzing
There are few examples given on the Atheris github page and blogposts. Let’s try it out first!
Example program 1:
import atheris
import sys
def TestOneInput(data):
if data == b"bad":
raise RuntimeError("Badness!")
atheris.Setup(sys.argv, TestOneInput)
atheris.Fuzz()Save the above program on your computer. For example, I saved it as example1.py
In the above example, we are fuzzing the function “TestOneInput” which takes an argument. If the argument is “bad”, the program raises a RuntimeError!
Running the fuzzer is simple. Just execute the code!
python ./example1.pyIf you have done everything right so far, you will see that the fuzzer within seconds will find the input causing the RuntimeError and writes it to a file.
Upon doing hexdump on the generated file, you will see the word “bad”
Example program 2:
While the last one was fairly simple program, to test the code coverage capabilitiy of the fuzzer, let’s try fuzzing with some nested if-statements. The idea is that the fuzzer must navigate through the if-statements to find the bug. For this I use the following program:
import sys
import atherisdef TestOneInput(data): # Our entry pointif len(data) != 8:
returnif chr(data[0]) == "d":
if chr(data[1]) == "e":
if chr(data[2]) == "a":
if chr(data[3]) == "d":
if chr(data[4]) == "b":
if chr(data[5]) == "e":
if chr(data[6]) == "e":
if chr(data[7]) == "f":
raise RuntimeError("Error input found!")atheris.Setup(sys.argv, TestOneInput)
atheris.Fuzz()
Save the above code as example2.py
In the above program, the fuzzer has to navigate through the nested if-statements and find the word “deadbeef” to find the bug. With each character checked, the code coverage capability of the fuzzer will be tested.
When I run the program, Atheris find the bug without a hassel!
In the summary printed, we can even notice the type of mutation used to navigate through the branches. The hexdump on the crash file shows the input we were expecting :)
Conclusion
Atheris supports fuzzing Python code and native extensions in Python 2.7 and Python 3.3+. However Python 3.8+ is recommended for better coverage performance.
With fuzzing becoming a mainstream testing technique and lots of projects being written in Python, Atheris is definitely a great addition to the existing list of coverage based fuzzers. Further, Google has already announced that Atheris will be part of OSS-Fuzz soon!
Happy fuzzing!
Cheers!
