Face Morphing
CS180 Project 3
Defining Correspondences
After scaling and cropping the two images to be the same size, the first step for face morphing is to define correspondence points on the images. I selected 51 correspondence points for each image, making sure to match up the facial features and body positions . Then, I used scipy.spatial.Delaunay to connect the points on each image into non-overlapping triangles.
Computing the "Mid-way Face"
Before figuring out how to generate the morph sequence, I first had to figure out how to create the "mid-way face" for Chris and me. This is pretty much just warping my face and Chris's face to the average shape of both our faces and then averaging the colors. For the sake of simplicity, let's call the image of my face "A", the the image of Chris's face "B".
I first computed where the average correspondence points would be: avg_corres_pts = warp_frac * A_corres_pts + (1 - warp_frac) * B_corres_pts, where warp_frac is 1/2. Then, I used the same Delaunay triangulation and to connect the average points into triangles.
For each triangle in the avg-image, I calculated the inverse transformation matrix T such that T * avg_triangle_i = A_triangle_i. Then, I get all the points in avg_triangle_i using sk.draw.polygon. Multiplying T with these points gets all the points in A_triangle_i. However, because these image coordinates have decimals, and there cannot be a fraction of a pixel, I had to interpolate the colors from the surrounding pixels using scipy.interpolate.RegularGridInterpolator. Now that I have the pixel values from A_triangle_i, I can repeat the same steps to get the pixel values from B_triangle_i.
Finally, I can fill in the average-image triangle with dissolve_frac * (interpolated colors from A_triangle_i) + (1-dissolve_frac) * (interpolated colors from B_triangle_i), where dissolve_frac is 1/2. Doing this for all the triangles gets the entire average image!
The Morph Sequence
Morphing one face to another face is just computing multiple "mid-way" faces, decrementing the warp_frac and dissolve_frac linearly from 1 to 0. At every step, the generated image will have more of Chris Hemsworth's image's shape and color and less of mine.
The resulting gif has 55 "mid-way" images, playing at 12 fps.
The "Mean Face" of a Population
Using morphing, I computed the mean face of the 200 unsmiling faces from the FEI database, which is a database of Brazilian faces. I did this by first computing the average of the correspondence points to get the average shape (summing each corresponding point and diving by the number of images).
FEI face triangulated with correspondence points
Note: some of the correspondence points were out of bounds, so I had to np.clip the values so they were within the height and width of the image.
Average of all the correspondence points
Then I warped each image in the dataset to the average shape. The final image was created by averaging the colors of all the warped images.
The "Mean Face" of a Population
I then warped my face shape to the average FEI face shape and warped the average FEI face shape to my face shape. warp_frac was set to 0 (shape warped completely to target) and dissolve_frac was set to 1 (color comes 100% from origin).
Caricatures: Extrapolating from the mean
To create caricatures, I extrapolated from the population mean I found previously. The caricature shape is alpha * (avg_population_correspondence_pts - my_face_correspondence_pts) + my_face_correspondence_pts, where alpha less 0 or greater than 1. I then warped my face to this caricature shape. If alpha > 0, the output img emphasizes the population's features (I would look more Brazilian), but if alpha < 0, the output img emphasizes my features that stand out from the population.
Morphing Ages and Morphing from Student to Student
I morphed images of myself from elementery school, to middle school, to high school, to college, and to elementery school again. A bunch of students and I also created a music video of our faces morphing into each other. This was done using the same technique as morphing two images. We just had to be careful with labeling the correspondence points in the same order for each image in the morph sequence.
