Introduction

One problem with the feature-based morphing technique of Beier and Neely is that all features had to be specified by the animator. However, in highly correlated images, such as faces, finding features should be able to be found automatically. The goal of my project was to find these corresponding features.

Technique

I begin by making several assumptions, which may be summarized as "the scene is very well behaved". I assume:
  1. the faces are in the middle third of the images
  2. the faces are aligned
  3. the faces have similar features (eyes are open, mouth is closed, neither subject is wearing glasses, etc.)
  4. the faces are facing forward
  5. the faces are similarly sized
  6. the background is a fairly constant color
  7. the scene is well-lit
My goal was to implement my technique for these assumptions, and if that worked, to slowly pare the assumptions in the order given.

My technique begins by first converting the image to grayscale, and then using the Sobel operator for edge detection. Once I have a set of edges, I calculate the edge magnitudes and generate row-wise and column-wise histograms of the magnitude (see below).

I then use the histograms to find the center of my face. I find the location of highest symmetry by finding the least SSD.

From there, I then planned to examine the column-wise histogram (see below) to find the outer edges of the heads: the sides of the head will be located along the maxs on either side of the line of symmetry. The top of the head would be found in a similar manner using the row-wise histogram. Then, edge following would be used to draw the lines for the outside of the head. Once these edges had been found, I would use higher-level knowledge of the shape of a face to narrow my search space for the other features. Again, the edges would be used to find the best features.

Results

I first implemented the morphing techniques, trying to provide a decent user interface. Below is a set of faces with lines, an intermediate image with lines, and the final morph. There are 36 control lines and 33 intermediate images. The morph took approximately 9 minutes to generate on a 300 Mhz Pentium II, and significantly longer to find control lines.
I did not quite get as far as I had hoped to for this project. Below is my the beginnings of my technique, showing the edge magnitudes, the histograms, and the center line I found.

FUTURE WORK

I think that my results thus far show that my project might have a chance at working had I fully implemented it. It may be possible to use Seitz and Dyer's view morphing techniques to align a face. I could then find features as I would have before, and inverse project the lines so that they match the original features. I could use the same technique to interpolate the lines.

Bibliography

  • Beier T. and Neely, S. "Feature-based Image Metamorphosis". Proc. SIGGRAPH 1992, 35 - 42
  • Seitz, S. M. and Dyer, C. R. "View Morphing". Proc. SIGGRAPH 1996, 21 - 30