Eva Schiffer

Computer Puppetry, and Importance-Based Approach Hyun Joon Shin, et al., ACM ToG 2001.

Summary:

This paper discusses an automated, real-time approach to retargeting puppet positions and joint angles to match motion capture data from a puppeteer. The authors carefully balanced their system to handle humanoid puppets that are superficially similar to their puppeteers extremely efficiently, making real-time motion capture driven animation possible.

Problem:

How does one fit a puppet's motion to motion capture data in real time?
How does one preserve environment interactions when fitting a puppet to motion capture data?
How does one determine which parts of a puppeteer's motion and position are important at any given time?


Method:

The author's of this paper took a very practical approach to figuring out what was to be considered important at any given time. Because they wanted their puppets to interact with the environment, they computed the importance of a given end-effector by determining how close it was to the surrounding environment. The closer the end-effector to the environment, the more important its position was considered. Because they blended the affect of the end-effector's positions and the joint angles leading up to that position, their importance measure allowed puppeteers to target specific parts of the environment while still having the puppet position itself in a similar manner to the puppeteer where it's end effectors were not near the environment. The authors also broke the task of finding the puppet's actual position into three separate problems. First they estimated the position of the root. Then they tried to figure out the posture of the main body. In these two steps they kept in mind the target points of the end-effectors and tweaked the root and body posture appropriately. Their final step was to determine the posture of the limbs. Because they used an inverse kinematics solver with their importance data as a guide, they only rarely had to resort to a heavy duty, computationally expensive full numerical solver. 

Key Ideas:

Motion capture based puppeteering can be done in real-time.

Importance can be simply determined by environmental closeness if environmental interaction is considered a very important aspect of the character.

Carefully tweaking existing IK solvers to include importance data can lessen the need for full numerical solvers.

Contributions:

The choice of an importance based heuristic.

The tweaking of an IK solver to allow real-time computations.

The use of Kalman filters to smooth noisy motion capture data.

Questions:

I would have liked to see their system in motion on the television shows that they mentioned, so it would be easier to evaluate their resulting data.
I was also curious how the authors got all of the 3D environment data to figure out what the puppet was interacting with. From the image of the children's TV show, it seemed likely that the entire background was computer generated, but I was unsure if this was actually how the authors went about getting this information.