Main / P2
Project 2 - Motion Capture Display and Editing
On this page... (hide)
For this project, you'll build a program read in and display motion capture data, and do some basic motion editing.
There are two reasons for this project:
The basic things your program must do:
For an interactive viewer, your program must (at least):
More details follow...
You will work in assigned groups of 3. It is up to you how to divide the labor. You could even decide to have everyone write their own program and just turn in the one that turns out the best (this is not a recommended strategy).
Be warned: I cannot provide you with any help for tools other than C++/FlTk. If you choose another development environment, you're on your own. Also, it is your responsibility to install any tools you need so you can run your program on the 1358 computers. If you get hung up because of issues in the tools you've chosen, expect little sympathy.
Another warning: there is a chance that we will mix up the groups for the 3rd project, so other people may be looking at your code.
Your program must read "Biovision hierarchy files."
Motion capture file formats are documented in a set of notes by Jeff Thingvold (now at Motion Analysis) (here). These notes are over 10 years old, but they are still useful - people still refer to them. The BVH file format documentation is somewhat vague, but its a vague format. The "original, official" specification was some hand sketched notes by some people at BioVision (I have a copy of those notes, and they aren't too helpful).
In the directory p:/graphics/public/Data/Motion, you will find lots of BVH files to experiment with. You will see all sorts of different skeletal configurations, joint types, and other variations of the file formats.
In the ideal world, you would be able to read any BVH file we threw at your program. More realistically, we would like your program to understand some subset of them. The larger the subset, the better (and this will be rewarded in grades). Some ground ground rules:
Some reasonable simplifying assumptions you might make include: supporting only certain types of euler angles, or only supporting a fixed topology. (the latter is a severe limitation, and I do not recommend it).
There are sample files designed to stress test a parser in:
Your program should be able to read in multiple motions. Its nice if you can display them at the same time (switchable).
You must provide an interactive viewer for displaying the motions that you read in. Your viewer should use OpenGL to display things, and provide the ability to play the motion at "frame rate" as well as the ability to "scrub" (interactively move through frames). You must provide some interactive camera controls so that the user can control the view.
The nicer that you draw things, the better. Drawing lines between the joints is the easiest, but drawing some "bones" (like ellipsoids) looks a lot better. Skinning looks the best, but that's a really advanced feature.
Drawing a groundplane and hack (drop) shadows are an easy way to make things look a lot better.
You program must be able to place the camera in a position where it can see the whole motion (this requires you to figure out the spatial extents of the motion). When a new motion is read in, the camera should be positioned so that the whole motion is visible.
The user must be able to interactively manipulate the camera.
You must put in a timer to control framerate. Your program must offer the option of displaying animation at 30fps. You might want to allow other options (like just blasting the animation as fast as possible, or allowing for slow motion viewing).
Bonus Feature: A tracking camera that follows the character as it moves. This take a little thought to do well - it can't be too bouncy, or spin too fast, ...
You should do some things to help make the motion easier to see. The user should be able to switch these on and off.
Bonus Feature: Try being creative in given tools to help the user visualize the motion. Consider how to draw "strobes" (multiple frames at once).
These are the building blocks of more advanced techniques. By themselves they aren't too interesting, but they will be useful later.
Your program must be able to concatenate sub-clips of the motions loaded in together. For example, you should be able to do something like "add frames 30-60 from motion A to the end of motion B". For now, the specification of what motions to concatenate can be manual - and it doesn't have to have a good user interface. And you can assume that the characters have the same topology/morphology (best if you provide an error if they don't). If the limb lengths aren't the same, you should just use the skeleton from the motion that you're appending on to. You can consider retargeting later.
You must do at least the most basic version of this, better if you can do the harder versions:
You'll probably want 2 and 3 for project 3, so its better to do it sooner rather than later.
Bonus Features: you could support various experiments with number 2 and 3 to see how the various methods compare. For example, rather than using the exact alignment in number 2, you could adjust the angle a little bit (how much cand you do this before its noticable?). Or, are blend transitions better than cut/filter transitions?
Basic blending - where you take a "linear" (linearly in quotes, since you might want to average rotations in a better way - like using slerp for quaternions) combination of each pose - was used a little bit in the previous section, but your program should be able to do it more generally. You should also be able to have the blend weights change (like go from 0 to 100% of motion B in order to make a transition).
You should be able to do something like "blend of frames 10-40 of motion A with frames 50-80 of motion B with weights 30/70 and put the result at the end of motion C". Just like with concatenation, there are issues of positional alignment, ...
An easy addition: stretch the length of one motion so that it has the same length as the other. (timewarping is a bigger bonus thing). One thing you can do with this: blend a pose with a motion, so you can blend a sitting down pose with walking motion to get, well, something funny.
For now, you just need to support simple blending. Looking ahead, you might want to handle blending where the motions are aligned better.
A simple alignment (recommended): align the first frames of the two motions to be blended (for example, aligning both to the end of the motion that the result is being concatenated onto). You might also consider incrementally blending the global position orientation. (but that's a more advanced feature).
Bonus Features: Fancier blending (implementing more of registration curves), or coming up with better transition mechanisms.
While "big things" are better left for project 3, here are some that aren't too unreasonable:
Path Editing: You can provide the user with a way to edit the path of the motion (e.g. if the character was walking in a straight line, you can bend it and have it walk along a curve).
Path Analysis: A lot of social scientists are interested in what are the natural paths people take when they talk around. If you just look at the root trajectory, you see the oscillations side-to-side for each stride, so its hard to really assess the curvature. You can try various things for this.
Footplant identification: you can determine when the feet are supposed to be planted on the floor. Its easy to get it right 95% of the time. Tricky to get it 99%. I don't think its possible to be right 100% of the time. Footplant solving (fixing errors in the footplants) is probably too much for this project.
You can talk to me about any of these, or even other ideas.
On April 17th you'll turn everything in:
Each group must have a wiki blog. You should post to it regularly. However, you must at a minimum make the following postings. (be sure to use the word "MILESTONE" in the posting title)
Note: if you aren't meeting the milestones (e.g. deciding what language platform by 3/30, or showing motion by 4/10), you will have a really hard time getting the project done.
The handin for project 2 will consist of 3 parts:
1 is scheduled. Officially, 2 and 3 are due at the same time. However, I will accept late things (within reason - up until Tuesday, April 21st) without penalty. One person in your group should send me email when the wiki page and the zip file are ready (telling me where they are).
In your wiki page writeup, please include:
Either as part of your writeup, or as a seperate wiki page, you also need to provide an assessment of every other group's project (you'll see them during the demo session). The big question for you: does it look like this project will be a good platform for doing Project 3? (implementing a more advanced motion editing/synthesis technique)
Note: we'll be moving on to project 3 soon, so you might want to get this done quickly.