In this document, we describe how to reproduce the figures in paper Reducing File System Latencies with Chopper. More importantly, the document provides some basic code that you can modifiy to analyze new results from Chopper.

I try to make it easy for you to run the code. The minimum requirement is that you have R and several packages installed (describe in this page). Then you can simply copy and paste the code to R and you will see the figures. This may take some time as it has to download data files from GitHub. If you want it to be faster, you can also download Chopper at https://github.com/junhe/chopper and produce the figures locally (instructions are also available in this page).

Please note that some factor names in source code are different to the paper. And I am too lazy to keep them consistent. But you should map them fairely easy. I provide a mapping table here.

Paper Code
DiskSize disk.size
UsedRatio disk.used
FreeSpaceLayout layoutnumber
CPUCount num.cores
FileSize file.size
ChunkCount num.chunk
InternalDensity Fullness
ChunkOrder chunk.order
Fsync fsync
Sync sync
FileCount num.files
DirectorySpan dir.span

Install R

We wrote the analysis with R. You can download and install R from http://www.r-project.org/ .

If you use Ubuntu, you can install it by:

sudo apt-get install r-base

Install Required Packages for Analysis

Run the following script in R to install necessary packages:

install.packages(c("grid", "ggplot2", "combinat", "gridExtra", "plyr", "reshape2", "R.utils", "devtools"), dependencies = TRUE)

If the one above does not work, try the following. I almost ruined my R environment when trying to verify the command. I am not gonna try it again… You get the idea - install these packages listed.

install.packages(c("grid", "ggplot2", "combinat", "gridExtra", "plyr", "reshape2", "R.utils", "devtools"))

Produce Figures

We now describe how to reproduce the figures in paper Reducing File System Tail Latencies with Chopper.

Figure 1

You can copy and paste the following code to R to reproduce the figure without downloading the analysis code.

library(devtools)
source_url('https://raw.githubusercontent.com/junhe/chopper/master/vis-analysis/fig-1.layoutnumber.R')

If you have downloaded Chopper to your local file system, you can also copy and paste the following code to R to get the figure.

USE.REMOTE.CHOPPER = FALSE
setwd('/Users/junhe/workdir/chopper/vis-analysis') # Replace the path with yours
source('fig-1.layoutnumber.R')

Figure 2

Figure 2 is diagram showing components of Chopper. So not here.

Figure 3

Copy and paste the following code to R to reproduce figure 3 without local Chopper folder.

library(devtools)
USE.REMOTE.CHOPPER = TRUE
source_url('https://raw.githubusercontent.com/junhe/chopper/master/vis-analysis/common.r')
source_url('https://raw.githubusercontent.com/junhe/chopper/master/vis-analysis/fig-3.cdfs-ext4-xfs.r')

If you have downloaded Chopper to your local file system, you can also copy and paste the following code to R to get the figure.

USE.REMOTE.CHOPPER = FALSE
setwd('/Users/junhe/workdir/chopper/vis-analysis') # Replace the path with yours
source('./common.r')
source('./fig-3.cdfs-ext4-xfs.r')

Figure 4

Copy and paste the following code to R to reproduce the figure without local Chopper folder.

library(devtools)
USE.REMOTE.CHOPPER = TRUE
source_url('https://raw.githubusercontent.com/junhe/chopper/master/vis-analysis/common.r')
source_url('https://raw.githubusercontent.com/junhe/chopper/master/vis-analysis/fig-4.variance.r')

If you have downloaded Chopper to your local file system, you can also copy and paste the following code to R to get the figure.

USE.REMOTE.CHOPPER = FALSE
setwd('/Users/junhe/workdir/chopper/vis-analysis') # Replace the path with yours
source('./common.r')
source('./fig-4.variance.r')

Figure 5

Copy and paste the following code to R to reproduce the figure without local Chopper folder.

library(devtools)
USE.REMOTE.CHOPPER = TRUE
source_url('https://raw.githubusercontent.com/junhe/chopper/master/vis-analysis/common.r')
source_url('https://raw.githubusercontent.com/junhe/chopper/master/vis-analysis/fig-5.tail-distribution.r')