7.1 Fuzz Testing Exercise

1. Introduction

The goal of this exercise is to give you experience with the basics of fuzz random testing. Fuzzing, as this kind of testing is often called, is a simple type of testing. Its goal is to cause failures in the form of crashes or hangs. These failures typically indicate that you have owned bits in the program that you were not intended to own.

More specifically, your goals are:

  1. Learn about the fuzz generator tool and how it works.
  2. Use the fuzz generator to try to cause failures in an example program.
  3. Find the cause of the failures (i.e., debug the program being tested) and fix them.
  4. Re-run the tests to verify that your fixes are sound.

You will use the classic fuzz generator to test a program written in C.

The basic steps are to:

  1. Build the fuzz generator and the test program.
  2. Run the fuzz generator on the test program.
  3. When you find a crash, debug the cause of the crash, and fix it.
  4. Re-run the program to see if your fix actually works.
You will repeat steps 2-4 to find multiple problems in the code.

The test program is named report; it supports queries on a database of flight log entries.

You run report by typing a command of the form:

report logfile1 logfile2 ...

The input log files have one entry per line. Two sample log files are provided, named log1 and log2. report starts by processing the log files and then reads query commands from standard input. You can see a summary of the commands for report by typing 

> help

2. Exercise Instructions

For this assignment you will use the VM image provided for the course.

2.1 Compiling Fuzz and Report

Note that if at some time you need the original files, you can find the tar file using the file path /p/mist/public/html/fuzz-639.tar on any of the CompSci Linux systems or download from here.

In the virtual machine image you will focus on two subdirectories:

Subdirectory    Purpose
fuzz:    Contains the fuzz generator tool, with its Makefile
report:    Contains the log query test program files with its Makefile

To start, you should cd into each of these two directories and run make. Both fuzz and report should build with no errors (though there may be one or two warnings). 

cd /home/user/Desktop/EXERCISES/fuzzing_exercises/log_exercise/fuzz
make
cd /home/user/Desktop/EXERCISES/fuzzing_exercises/log_exercise/report
make

2.2 Try out Each Program

Since fuzz and report are not in the same directory, you'll have to explicitly name the path to the executable file.

For fuzz, you can look inside the source (.c) file in the comments to see a descriptions of the options supported by the program. Try out several of these to see how it works.

For report, you can start the program using one or both of the sample log files, such as: 

report log1 log2
Then type give the command "help". Try out several of these to see how it works.

Some sample commands to try are: 

> select dte sel act hdd f/t lng
> range dte start 1/1/1989
> list

2.3 Start Fuzzing the Test Program

There are two ways to run report with fuzz input. The basic way is to pipe the output of the fuzz generator directly into the test program with a command like:

fuzz 100000 | report log1
This is a good way to try things out but makes it difficult to reproduce your results since fuzz uses a random seed to decide what characters to generate.

There are couple of way to make the test data streams reproducible. The easiest way is to run fuzz separately and have it store the random streams in a file, such as this example that stores the random characters into file "f1": 

fuzz -o f1 100000

Then you can repeated run report like this:

report log1 log2 < f1

A second way to do this is to choose a random seed when you run fuzz, which will produce the same output stream each time: 

fuzz -s 12345 100000 | report log1
You will run fuzz and report until report crashes or hangs.

Once that happens, you should debug the program to determine the cause of the crash, and then fix the bug. Of course, you need to re-run the same test to make sure that the crash no longer happens.

After you fix the first problem, repeat this whole test/debug/fix cycle. You should be able to find two (2) different problems in report.

2.4 Write a Report of Each Crash

You will write a report for each crash, including:
  1. An excerpt of the code that was vulnerable (identifying the file and line numbers).
  2. Output of the program showing the crash behavior.
  3. A description of how you fixed the bug that allowed the crash. (Show the new code.)

3. Extra Challenge

In the regular assignment, you fuzzed the standard input to the report program. In the extra challenge, you will fuzz the input from the log files.

Since the input files are specified as parameters to the report command, you cannot pipe the output of fuzz directly into the program. So, you will have to store the output of the fuzz program to a file and then use that file as a parameter when running report, such as: 

fuzz -o f1 100000
report f1
As with the regular assignment, you will try to crash or hang the report program. If you are able to crash the program, then you will debug the cause of the crash, develop a remediation for the crash, check to see if this remediation works and then write a report on this new crash.