Marker data recovered from optical systems often contain gaps in the sampled postion signals corresponding to each marker. Traditionally, these gaps are filled in using a combination of automatic interpolation, and manual editin.g
The main goal of this project was to explore various methods of recovering the missing data using fully automatic means.
The data used for the project consisted of marker data from an optical system. The data provided had been processed so that correspondences were provided. Unfortunately the correspondences were not wholey accurate.
Interpolation methods are usefull when the missing data has useful recordered values about the gap which can be used for interpolation. If the missing data occurs at either end of the recorded sequence, interpolation fails, as illustrated in the figure below.
| |
|
Despite this shortcoming, interpolation techniques are worth considering for two reasons. First, they are easy to implement, and second, they are generally faster than more accurate methods. This makes them suitable for real-time systems.
Point interpolation processes each marker signal independently and interpolates marker position to fill in missing data. While this method works well for small gaps where little movement occurs, it does not work well for large gaps, or gaps where the markers motion is due to complex articulated motion.
The figure above shows the poor placement of a knee marker using linear interpolation of the markers position over a gap of over 700 frames.
Offset interpolation uses all available links of the articulated figure to generate the positions of missing markers. Each link connected to the marker is expressed as an offset from the opposite marker. The interpolation then occurs over the offsets, instead of the raw position. By interpolating in a space local to the marker, as opposed to the global space, the interploation works much better over longer gaps with more complex motions.
 |
|
| Point Interpolation | Offset Interpolation |
The two animations above illustrate the two how well each interpolation technique "fills in" missing data. The point interpolation clearly fails at several points. The offset interpolation also appears to fail when interpolating the knee, however this is due to the fact that the knee had been mislabeled and is in the wrong location.
However, both motions fail to recover a reasonable motion for the pirouette.
 |
| Bad Knee Correspondence |
We can measure how well the interpolation predicts the actual location of the missing markers by taking a known marker signal and comparing it to the signal that interpolation predicts.
The graph above shows the
In general, the offset interpolation performed better than point interpolation. However, both forms of interpolation had troubles with long gaps. The graph below illustrates the accumulated error as the gaps grew in frame number. Notice that both perform better with the upper back than the knee. This is because the upper back is much more stable during the running motion.
The problem of filling in missing marker data can be solved fairly reliably over short gaps, or medium gaps where there is little complex motion. As the gaps increase in length, or the motion increases complexity, these methods fail, just as sampling theory tells us they should.
Given this, interpolating the joint offsets provides better results than simply interpolating the markers themselves. This is because more information is used to reconstruct the path of the missing marker.
While optimization methods my give "better" results, at least according to the objective function, they are still going to suffer the same problems that large gaps cause. High frequency motion will be very difficult to recover correctly.