It is no secret that computer generated imagery has opened horizons in terms of what storytellers can achieve with their narratives. No longer are filmmakers, commercial videographers, and content teams grounded by limitations that practical effects once held over productions, rather there are now two routes one can take when deciding how to shape the narrative when it comes to effects, and often times both these worlds meet.
This blog will focus on how minor digital effects work can enhance a live-action shot by adding another layer to the set design that would not have been possible with practical effects due to limited time and budget.
For this example we will look at a still image from the author’s film, Bleed American. Intermediate Adobe After Effects knowledge is needed for this tutorial.
1. Planning for the effect on set
While on set filming a commercial, industrial, film, or Television show, it is important to understand what you plan to do in post-production with each particular shot and image. Looking down the road will help you accurately frame and prepare for a streamlined post-production process. In this particular situation, there was an opportunity to expand on the narrative in the given scene.
The script called for two characters to walk down an alley in a rough area. While most of the buildings are worn down with weeds growing through the pavement, there was another opportunity to occupy a blank wall of the left foreground building with graffiti or a mural. This would lend to the story, helping to push further the idea of the area being unkempt and decaying.
On set, we made sure to leave a section of the blank wall within the frame, as opposed to cropping it out, knowing the plan was to fill the wall with art in post-production. As mentioned in the introduction, we could have certainly painted a mural practically on set, however this would have meant hiring painters or artists, and allowing at least a day or two for the work to be done, something we did not have the time nor the budget to allow. Once the decision was made on how we wanted to film the scene, we moved into post-production.
2. 3D Map the camera movement
The first step is to map the objects within the frame. Note that this step can be skipped if the camera was locked down on a tripod with no movement when filmed. However, this scene was shot
handheld with the camera mounted on the operators shoulder. This means all the objects are shifting points within the frame, so we need to track those movements.
In Adobe After Effects, apply the “Track Camera” option from the Tracker menu. This will create digital marker points on your footage and will track them frame by frame as the scene progresses.
Once the tracking is completed, it is time to create a solid and a camera. The solid will act as a digital placeholder for our mural image that we will add later. To do this, you will need to draw a lasso around the track points that are nearest the object you wish to place the solid, in this case the wall. Simply click and drag, rolling over the desired areas.
Once we have a few selected, right click and select “Create Solid and Camera”.
If done correctly, a colored block should appear on the wall. Some adjusting in terms of positioning may be necessary to align it perfectly over the section we wish to add the mural. Scale the solid to cover as much of the wall as possible.
3. Replace the Solid with Artwork
With the solid in place, the next step is to replace it with the digital artwork. In this case, we used a royalty free mural from pexels.com. Double click on the layer labeled “Track Solid 1”.
The layer will open in a new viewer window, and the next step is to take the desired artwork (in this case, the 2D mural) and drop it into the comp.
Navigate back to the main layer window, and the solid should now be replaced with your artwork image, looking something like the following:
Additional scaling may be needed of the solid layer in order to fit the artwork on the wall. Keeping the goal in mind, we want this image to look seamless, as if painted directly on the bricks by some passerby.
4. Refine the artwork
Once the solid has been replaced with the art, it is time to refine it to make it feel natural to the scene. One simple way of doing this is to change the “blend mode” from normal to soft light.
This will let the natural brick of the existing wall to show through the image, making the artwork appear more realistic, and faded.
You will notice there are other elements interfering with the mural. The mural is layered over the telephone pole, which is breaking the illusion of the effect. Since this is a 2D image, we will have to create the depth virtually. This means masking out the telephone pole.
5. Mask out the pole
To do this, duplicate the source footage and draw a mask with the pen tool around the telephone pole we wish to duplicate.
Doing this will isolate the pole as its own layer, existing above the footage containing the rest of the scene. To make sure this worked, we can turn off the footage layer of the main scene, and what remains visible should be the duplicated/masked layer of the pole:
You will need to track this mask using a standard “Motion Tracker” from the track menu in order for the mask to match the motion of the main scene, thus ensuring the pole stays aligned appropriately.
Once the tracking is done, we should see the pole is no longer beneath the mural, but rather now sits above it:
6. Color Correct and Color Grade
The last step to selling the effect is to color correct and grade the image. The main goal is to have the effect match the contrast, saturation, tone, and hue of the rest of the scene so that everything feels cohesive. For a more detailed look at color grading, visit our blog on how it affects the mood of a shot here. With color grading complete, we can take a look at the before and after images side by side.
In the end, the goal of blending digital CGI into footage is to enhance the narrative. This tutorial is equally applicable to branded content videos, commercials, and more. Look for ways to incorporate digital components to your shoots to further your storytelling capabilities! To see everything discussed in this tutorial in a timelapse video, take a look at this brief transition clip here: Visual Breakdown.