How to Facial Track An Indie Shot

Fixed it in post! Added digital glasses for actress Lindsay Ogus

In our recent short film, our actress Lindsay Ogus was suppose to be wearing glasses, but you know what happens when you actually start filming, the director (me) forgets about it. Thanks to today’s visual effects, I was able to fix this issue and add glasses into the shot in post. The set-up was done entirely in 2D which makes things more simple.

The technique I used came from a mixture of free tutorials from Video Copilot. You will need a video editing software, a great open source one I recommend is Davinci Resolve by Black Magic. Here, I’m using Sony Vegas. You will also need Adobe Photoshop and After effects. You will need to be quite familiar with the software’s and techniques as this is an intermediate tutorial. It is also good to know how to track with Mocha by Imagineer Systems. To learn about the basic aspects of After Effects, be sure to check out Andrew Kramer’s free Basic Training video. Without further ado, check out the tutorials below:

Fig 1. Simple setup with 2 lights. 3 cameras were rolling but only 1 was used for the final cut

The Setup:

The camera was shot on a mono pod, coincidently, our actress’s did not turn her head or move too much. This is important later on as it helped make the tracking easier. The background was dark and there was a single light shining down over the top of her head. She said her lines into the camera as if it was a webcam. The entire length of the monologue was close to 2 minutes. (Fig 1.)

The Breakdown:

To add glasses onto her head, there are 2 techniques you can do with this type of shot. If the camera or actress was moving more in the shot 3D tracking would be the best option. However it’s more technically involved and is not really necessary in our example. Another method is to track in 2D, which is easier to comprehend and you don’t need any software’s beyond the ones mentioned above. I’ll outline the two methods below.

3D Tracking: Track her head with a 3D camera tracker which comes with After Effects. The tool is called “track camera”. Once you have collected the tracking data, you will need to load it into a 3d software like 3D Studio Max, Maya, or a great open source one named Blender. The software will create a camera move that matches the original camera. You may need to manually track to make more adjustments. You then need to download the model, a pair of 3D glasses or build your own. A few free 3D models can be found at Turbo Squid. You can also add a light to simulate the same lighting effects recorded in your shot. Once your model and your tracking data is properly set-up in the software, render and export the shot. Then import it into After Effects and composite.

From here, you can colour correct and make minor adjustments to your model. Your glasses should stick onto the live footage. This is the outline for taking the 3D method, which for me, is too complicated. So I took the more simple, but effective 2D method.

Fig 2. Creating a shape around the nose was the best technique for Mocha

2D Tracking:

1) Track the shot; I used Mocha. This shot was quite difficult to track because there were no tracking markers placed onto her face. Normally tracking markers are placed on the surface of the face so that it is easier for the software’s algorithm to follow the point. The markers are then painted out afterwards. One can use a mask tool in After Effects, and offset the picture slightly in order to remove the marker. Please watch Kramer’s talk on facial marker removal (Fig 3). In Mocha, the tracking shape I found worked best was her nose, and sometimes, forehead (Fig. 2). With Mocha, the larger the shape you use to track, the more accurate the information. Mocha is a 2D planar tracker software, it does not look for contrast in a group of pixels, instead it tracks the shape of an object. Once you have completed the tracking in Mocha, copy and paste the tracking point into a null object in After Effects. Delete the anchor point and test out your tracking data. Here is a video for more information on how to track your footage narrated by Mary Poplin, a Mocha specialist.

2) Search for the image. To do the effect in 2D, find an image with a pair of glasses in the front view. In Google, search for images by selecting the “image” tab, select search tools and under user rights, choose “label for reuse with modification”. Use Adobe Photoshop to remove the background by using the magnetic tool. You will also need to find an image with the arm of the glasses because our actress turns her head slightly which reveals it. So we’ll need to extract the arm from a side image of a pair of glasses as well.

Once you have extracted the front and side images of the glasses, import them into after effects.

Import the front view image of your glasses, and link its position to the null object. You may need to manually adjust some of the points as the image may not stick to the shot 100%.

3) Once the image sticks to the face, you have done a successful track. Pre compose the composition and add a glass layer so that it looks as if there are lenses in the frame. I have also added a shadow and an effect called “noise match” to create a subtle amount noise to match the original footage.

4) Colour correct the pair of glasses so that it matches the original footage and export.

Facial tracking is a new tool in the later versions of After Effects. In our example, we didn’t have any markers on the face because this wasn’t meant to be an visual effects shot in the first place. However markers are recommended when the face and head are moving more dynamically.

Feel free to contact us if you have any editing or production questions. Like always, happy editing!