Oct 10: We discovered that we can't always just copy programs from the device to the emulator. Sometimes packages rely on other libraries. We figured out a way to remount the primary yaffs2 partition of the emulator as RW, allowing us to copy over library files. Unfortunately, this hasn't resolved our issue.

Oct 12: Requested MySQL database from CSL. More work on client that sends proc data to server. Started looking into how to schedule background processes on the phones.

Oct 15: CSL setup the MySQL database, and we got a basic database started. We setup a php page on our CS space to get POSTs from Androids and do the inserts, and the Android application can send data to the page.

Oct 27: We figured out how to create Android services. We also figured out how to make an application receive a message that is broadcast upon startup so that the application can start our service. Unfortunately, although the application works, it apparently hangs for a while, as we get a “force quit” or “wait” message upon startup. Clicking “wait” allows the application to work. Update: The application not responding message appears to not occur on the actual device.

Oct 29: We significantly modified our two logging programs/services (one records data and the other uploads data to our server). The redesign design is simpler, will fix a concurrency issue, and should be faster, but it's not completely working yet.

We added more tables to our database where we will dump more procfs data.

We looked into getting more interesting information from Android in addition to basic procfs stuff and found a BatteryStatsService class (frameworks/base/services/java/com/android/server/am/BatteryStatsService.java) that could potentially be extremely useful by providing battery usage information.

We did more work on our program that automatically runs a bunch of applications on the Android, but it's still buggy. We observed that some of the applications we were trying to start were already running when the experiments began (they begin at startup), so killing those first helped. We plotted the memory usage of some of the applications we were able to collect data for and noticed some interesting trend. Primarily, VM usage far exceeds what is ever kept in memory. We have yet to explain this.

Oct 30: We met to further troubleshoot our logging/posting services, but haven't made further progress.

Nov 5: We finally found and fixed several bugs in our code (mostly formatting issues) that had been causing problems for quite some time. We also combined our logging and posting programs into a single program. This makes the code simpler and eliminates the need to worry about concurrency issues. We discovered that some of the issues we are still having are due to Android emulator procfs content being slightly different than the formatting on actual devices.

Nov 6: We finally figured out how to properly create services that don't result in the “force quit”/“wait” dialog. onCreate needs to return quickly instead of looping indefinitely, so we set up a timer to cause work to be done at regular intervals instead of doing everything in onCreate.

Nov 11: We brainstormed ideas on the direction for project. Raja Ram put together a presentation explaining the paper (insert paper name here).

Nov 13: We came to the conclusion that the best direction for our project was to create a system where applications are graded based on their network performance, battery consumption, memory usage, etc.

Nov 14: We realized that our insert.php script has a security flaw. Any CS student can explore the html directory and read the file, thus obtaining our password. We explored different AFS settings to hide the password, but were unsuccessful.

Nov 15: We came up with an idea for fixing our security problem: keep half the password in the .php script and the other half on the phones. Then, the only way someone could gain unauthorized access to our DB would be if they obtained our logging application, reverse engineered it, and had access to the CS AFS filesystem.

Nov 16: Met to discuss what metrics we would use to grade applications. Decided to start on memory and cpu usage. Looked at how the top source code measures cpu usage.

Nov 18: We found an application that collected battery information, downloaded the source code, and compiled it as a android project. Also looked at the Android top implementation.

Nov 19: Created service that gathers cpu and memory usage. Found ActivityManager class which lists processes that are currently running, which provides pid, name, importance (foreground/background), and lru (low lru means it has been used more recently). Will use this information when polling procfs. Also noticed that applications running in the background will get killed if free memory approaches a certain threshold.

NOTE: We did a ton of work after this too, but forgot about the wiki for a while. Here's some of the highlights that we can remember:

We wrote an AppGrader program that calculates the CPU-Percent-Seconds and Mem-Page-Seconds based on procfs information. 1 CPU-Percent-Second means that the application did the equivalent of using the CPU at 100% capacity for 1 second (for example, perhaps the application ran for 2 seconds at 50% capacity). Mem-Page-Seconds is similar.

We used the information from ActivityManager to create a “utility function” that estimates how much value an application is providing to the user. Basically, we can't just grade based on CPU-Percent-Seconds because we don't want to give an application a worse grade just because the user used that application more. Thus, we divide CPU-Percent-Seconds and Mem-Page-Seconds by the amount of service the application provides to the user. Our utility function estimates the amount of service by looking at how much time the application spent in the foreground (we assume the user is actively using the application during this time). We also wanted to give applications some credit for the time they spend running in the background, but intuitively they shouldn't get as much credit as when they are actively being used in the foreground. Thus, our AppGrader gives applications one fourth credit for background time.

Just like our previous data collection programs, our new AppGrader posts the data to our database. After creating the appropriate database tables, we put our applications on several phones.

During this time, we also attempted to collect network information from the procfs, but were unsuccessful, as procfs only records system wide stats. We also attempted to collect data on I/O to the solid state drive, but were unsuccessful as the android kernel was apparently compiled without CONFIG_TASKSTATS flag set, so the io pseudo file is not available to us on Android phones like it is in Linux. We found one UNIX application, pidinfo, that get displays per process file I/O information, but upon inspecting the tools source code, we learned that pidinfo also relies on the io pseudo file in procfs.

We spent a great deal of time trying to recompile the kernel with the CONFIG_TASKSTATS so we could get the I/O info. We feel like we're close to figuring it out, but the phones hang when we flash them with the new kernel.

In an effort to evaluate our grading program, several of us used the same applications for a set period of time and we recorded the grades. Then we used the applications again while running the “counter program” in the background. We found some correlation between the speed at which the counter runs and the grades assigned by our program.