Earlier this year, Italian director Francesco Calabrese and VFX freelance artist, Fabrizio Bonaga were thinking of ways to collaborate with Amari, an aspiring band from Udine, Italy, with whom they’d become quite friendly with in recent months. Calabrese wanted to do something funny, creative, challenging, and even a little crazy. So the idea was hatched to make a film-style music video based on a series of classic science fiction movies and to realistically insert the band members into as many shots as possible.
With the concept solidified, Calabrese and Bonaga began pouring through as many sci-fi movies as possible, looking for inspiration and ideas on how this project could evolve. They also carefully reviewed and planned out the process and workflow in detail to ensure they had the most cost-effective and simplest approach.
They started the project by stitching together scenes from multiple movies, successfully finding sequences with enough continuity between the films. Calabrese then created a rough cut of what would eventually become the music video, carefully piecing together scenes based on the actors’ movements, ensuring the least amount of rotation and changes in perspective. When you’re about to embark on a major face-replacement project, the less ambiguity in each shot, the better!
“The biggest challenge was replacing the actors’ faces with the band members’ and creating the most realistic, seamless look possible. And that’s tough when you’re working with low-quality DivX as your source,” said Bonaga, who was charged with all the visual effects for the music video. “We knew in the planning stages that tracking these shots with an average point tracker would’ve been impossible. So we decided early on that Imagineer Systems’ Mocha was the only way to go. Even with such low quality source material, Mocha worked with extreme precision in every shot.”
I started the process by making some rough face substitutions; literally roughly dropping in faces throughout the video
“I started the process by making some rough face substitutions; literally roughly dropping in faces throughout the video,” continued Bonaga. “I specifically started the process this way to give me a sense of where my most challenging shots would be. But tracking with Mocha was so solid and accurate, there was really nothing else for me to do - Mocha tracked scale flawlessly. We did such a good job on the first pass, all I needed to do was track position and rotation; I never even needed to track perspective. The second pass was really about color matching. Mocha’s tracking was solid, just as we expected!”
Creating a music video with such sophisticated visual effects doesn’t come without its problems, however. With only one day to shoot the band and capture all the green screen shots needed to begin the face replacement in post, the team was tired. “At the end of a long day of shooting, we were exhausted. When we discovered that a few of the shots were the wrong perspective with the faces in the wrong positions, we decided not to re-shoot, but to take advantage of some of the tools in post to fix the problems. This turned out to be a relatively easy fix. I simply removed the entire original head from the movie, wiped it clean with a plate created in Photoshop, tracked the background with Mocha and replaced the head completely with the band members’ head!”
The second major challenge was color matching. Since many of the movies they selected for the video were older, they had a grainy quality. So not only did Bonaga have to precisely track each face, but he needed to recreate the lighting and grainy look as well. “I’m not an expert in color correction,” continued Bonaga. “But I jumped in with enthusiasm and managed with a very practical approach. The ability to export my Mocha tracking data, and do it in low-rez jpeg compression, I was able to recreate the grainy, squared ‘noise’ that matched the original film perfectly.
Bonaga also credits Francesco Calabrese’s expertise behind the camera as key to making color correction and matching that much easier. “After studying the source material closely, Francesco was able to capture such a close look and feel with the on-stage lighting as he was shooting. This truly made my job easier too!”
One day of shooting, two days of pre-production and approximately 90 hours of post production went into the creation of the Tiger music video for Amari. The result is nothing short of spectacular, with images so clean and accurate, it’s almost impossible to tell that a major face replacement project took place. And the result was exactly what Calabrese, Bonaga and the band members of Amari were striving for.
“We wanted a challenge,” concluded Bonaga. “But we knew we could have never even conceived this project without Mocha!”