View Morphing

This is my final project for Computer Animation course, which implements the View Morphing by Steven M. Seitz etc [2]. You can see the full report here (in Chinese).

Background

Image morphing techniques can generate plausible transitions between images, but they may yield unnatural distortions when images are of different camera pose or different scenes transforms. The View Morphing technique is an extension to image morphing which also accounts for changes of viewpoint and scene transformations. It produces convincing transition results between images and can synthesize image under new viewpoint with only two images as input.

Algorithm

View Morphing has three steps.

Computing H0 and H1

We first compute Fundamental Matrix F as described in [1], then we get H0 and H1 as follows:

Image Morphing

Like the original paper, I use the Feature-line based image morphing method [4] to morph the images. The algorithm works by first specifying some feature-line pairs on two images (e.g. connect two feature points to get a feature line), then warping and blending the images which are interpolated under the control of those feature lines. The morphing algorithm is GPU-friendly, so I implement it on GPU using BSGP language [7].

Computing Hs

After morphing the prewarped images, we need to postwarp them back to the original image space. This perspective matrix can be computed following [3], that is to ask user to specify 4 feature-point pairs, so that we have 8 linear equations, and we add another constraint (such as forcing the last element in matrix to be 1), then we can solve the linear system to get the matrix.

Program Interface

I organize the program as a step-by-step wizard.

Step 1

User specifies two images, the second image can be a mirrored copy of the first one.

Step 2

Edit the corresponding feature lines. Whenever user adds a line on a image, a corresponding line will automatically be added on the other image.

Step 3

This step has no user interaction, it merely shows the prewarped results. The corresponding points in two images should appear on same lines.

Step 4

Edit the I0.5 rectangle, which is used to guide the postwarp.

Step 5

In this final step, user can drag the slider to view the morphing results or export them as a video.

Results

I compare view morphing's results with those of standard image morphing[4].

The last frame of Image Morphing (I0 morphed towards I1)
The last frame of View Morphing (I0 morphed towards I1)

The view morphing result clearly has less distortions.

Videos

Image Morphing
View Morphing
Image Morphing
View Morphing

The difference may seem subtle, but by careful examination, you can see that the view morphing result yields a more realistic head-turning motion, while the image morphing result exhibits some squeezing and stretching effects of morphing.

Source Code

On my github page.

Reference

  1. R. I. Hartley, In defense of the 8-point algorithm, Fifth International Conference on Computer Vision, IEEE Computer Society Press, Cambridge Massachusetts, pp. 1064-1070, 1995.
  2. S. M. Seitz and C. R. Dyer, View Morphing, Proc. SIGGRAPH 96, 1996, pp. 21-30.
  3. S. M. Seitz, Image-Based Transformation of Viewpoint and Scene Appearance, Ph.D.
  4. Dissertation, Computer Sciences Department Technical Report 1354, University of Wisconsin, Madison, October 1997.
  5. T. Beier and S. Neely, Feature-Based Image Metamorphosis, Computer Graphics, 26(2), 1992, pp. 35-42.
  6. http://people.csail.mit.edu/hasinoff/viewmorph/
  7. http://www.cs.cmu.edu/afs/andrew/scs/cs/15-463/f07/proj_final/www/amichals/
  8. http://houqiming.net/

Last updated 10/13/2014