Marco Sebastiano Alessi |
December 13, 2023
When you watch a sci-fi movie, there are a few things that you unconsciously expect, including incredible special effects: alien creatures, spaceships, otherworldly landscapes, and so on.
In today’s article, you’ll learn about the matchmoving technique that has revolutionized the film industry and continues to improve year by year. You’ll understand the concept of matchmoving, what a matchmove artist does, the process and types of matchmoving techniques. Is matchmoving the same as motion tracking? I’ll answer this and many more questions about matchmoving in the article below.
Let’s dive in!
What is Matchmoving in VFX?
Matchmoving is a filmmaking technique used to recreate real-life camera movement from live-action footage in a 3D virtual environment to add computer-generated elements to the original footage. Matchmoving aims to make the CG elements appear as if recorded in the real world.
Camera tracking is another term for matchmoving, as it involves tracking the camera motion using a tracking algorithm and tracking markers from features recorded in the footage (like something in the floor or on the wall that can be used as tracking points) or by placing them in the set yourself to create accurate tracking information.
Next, you’ll need to transfer the footage inside 3D computer software with as much camera information as you need, including: movement, position, orientation, perspective, shutter angle, lens distortion, focal length, distance, and information from the physical set, like its dimensions.
With all the camera parameters collected, you can create a virtual camera in the 3D scene that matches the one in the real world. Everything 3D rendered will be viewed from the same camera perspective as the original footage, making the computer-generated models more vivid and realistic in your final composite.
As a matchmoving artist, you must be proficient in tracking and recording camera motion, object tracking, and familiar with filmmaking techniques and the best types of cameras and lenses for the purpose.
What is the Process of Matchmoving?
Matchmove artists can be involved in every step of movie production. In pre-production, you can work with the director and VFX artists to plan the shots and determine if tracking markers must be placed on the set. During production, you can check the set and take measurements, place tracking markers, and get information on the camera motion.
Visiting the Shooting Location
You may have the chance to visit the shooting location. If that’s the case, this will allow you to gather information, take measurements of the set, and talk to the production team about the cameras and lenses they will use.
Identify tracking markers, decide if you need to add markers on stage or actors, and inquire about other objects with which the CG elements and 3D characters will interact.
For example, if the CG characters will be sitting in a chair, you will need to take detailed information on the properties of the chair, size, and shape.
-
Matchmoving in Post-Production
For many independent matchmove artists, the process will begin during post-production. You will receive the live-action video footage and a briefing with a description of the lens, camera, frames, set measurements, and all the information you would collect if you had visited the film set yourself.
Study the Live Action Footage
Analyze the footage you received. Replay it and find the best spots to use as tracking markers. Import the footage to your matchmoving software and start the tracking process. Depending on your software, it may use tracking algorithms to automatically create, detect, and track these markers. When the scene presents motion blur, you must manually add markers for better tracking.
-
Creating the Virtual Camera
Once the program tracks features, it’s time to solve camera motion. With the tracking information and point clouds generated, you can determine the position and movement of the camera in the 3D space and the position of the objects over the scene. With this, plus the camera information you provide from the real world, your matchmoving software will transfer the data to your virtual camera.
-
Adding 3D Elements to the Footage
In bigger production teams, you will send the tracking data to the VFX artist, who will create and add the 3D elements, objects, characters, or set extensions. However, you might be in charge of the whole VFX process in smaller projects. In that case, you must create the 3D models, place them into the virtual footage, and composite them in the scene.
Types of Matchmoving
There are three main types of matchmoving: two-dimensional, three-dimensional, and real-time matchmoving.
-
2D Matchmoving
It only tracks features in a 2D space. It’s most used for stabilization or adding basic effects when replacing an object in the image plane and overlooking the camera movement, perspective, or distortion.
-
3D Matchmoving
3D matchmoving is like 2D matchmoving, except it considers the 3D space, camera movement, perspective, and distortion. This type of matchmoving is used most to build a 3D scenario and add 3D elements and animated CG characters. 3D matchmoving is also essential for real-time matchmoving.
-
Real-Time Matchmoving
Real-time matchmoving allows the director, production staff, and actors to preview how the CG elements will look in the final composite. It also allows the actors to interact with the preview and perform better, removing the usual problems with filming with a green screen. It also helps you to save time in post-production.
Real-time matchmoving is essential for the production process in films that use virtual production.
Is There A Difference Between Motion Tracking and Matchmoving?
You probably realized that motion tracking and matchmoving are often used interchangeably. However, motion tracking refers to the complete process of capturing footage, tracking it, and using that data to match 3D elements with live-action ones. Matchmoving, on the other hand, is a part of the motion tracking process: it's a specific step of tracking the camera's motion, not the objects, and matching the virtual camera with the one outside, allowing you to add the 3D elements into your live-action footage later.
Matchmoving Software
Nowadays, there are tons of matchmoving software options, and you can also combine two or more for specific situations and project needs. Here's a list of the best matchmoving software you can start with.
-
SynthEyes: An affordable standalone application optimized for camera, object, geometry, and planar tracking and solving. It helps with stabilizing shots and inserting any 3D animation into your footage.
-
3DEqualizer: It has advanced camera tracking software to merge live-action footage with visual effects at the highest quality. It is one of the most used matchmoving software by professional film studios in the industry.
-
PFtrack: PFtrack its automated and manual tracking tools for cameras and objects and photogrammetry capabilities enable precise camera placement to recreate 3D environments from static images or video footage.
-
Mocha Pro: It works as a plug-in and standalone application for planar motion tracking. It helps you track data quickly, solve the camera, and export the tracking data directly into your composite software.
-
Nuke: A node-based compositing and visual effects application with an integrated 3D environment that allows quick placement of 2D and 3D elements.
-
Cinema 4D: It’s a 3D computer animation, modeling, simulation, and rendering software with a fast and powerful tool for the most demanding 3D project needs.
Final Thoughts
Once you know how matchmoving works, you start to see and analyze movies with a different perspective, identifying scenes where matchmoving was used and wondering how visual effects were added.
If you’re interested in starting to work in the VFX field, learning the process of matchmoving will give you more opportunities to get started and enjoy the learning curve.
Have fun!
