Quick Start: 3-D Planar Tracking

< Previous | Contents | Manuals Home | Boris FX | Next >

‌Quick Start: 3-D Planar Tracking ‌

SynthEyes offers a very powerful 2- and 3-D planar tracking subsystem with many features, so many that it has its own manual: see Planar Tracking PDF on the SynthEyes Help menu. The 2-D capabilities serve as another flavor of supervised tracker with additional capabilities.

The 3-D planar trackers are especially powerful, such that you can create a full 3- D track from a single 3-D planar tracker, if a suitable flat area exists in the scene. That's what we'll describe in this section. We'll work from a rectangular area, which is best and easiest, though the planar tracking system can work with other shapes too.

Opening SynthEyes, click File/New and open the shot "Tower," a 77-frame 1080p HD shot available in the Sample Files area of the website.

Click the Planar room on the top room tab area, which is where planar trackers are created, tracked, and edited.

Tip : if you don't see the Planar room, you have upgraded and have your own room preferences. Right-click in the room area and select Reset All to Factory. For more info see "I Don't See the Planar Room" in the planar tracking manual.

On the planar panel, click the creation wand at top left. Then click 3 times around the corners of the window group at top left center, starting at top-left, then proceeding to top right and bottom right; the bottom left corner will be last, in a moment. (Right-click to cancel any time in the middle of this process.)

As you go to create the fourth point, at lower left, you'll notice that much more interesting things start to happen: a pyramidal display appears, and the fourth corner no longer follows the cursor and seems to have a mind of its own!

The pyramid is literally that, a 3-D pyramid sitting on top of the rectangle you are creating in 3-D. As you see, many of the corner locations are impossible or non-useful, and the pyramid will flicker around wildly to different locations:

Assuming you've placed the first three points reasonably, when you place the fourth properly, you'll see something that makes sense:

The pyramid is now projecting outwards, perpendicular to the wall. As you've been doing this, you might have noticed the Own FOV (field of view) display. With the corners placed, we now have a reasonable value for the camera field-of-view, from one tracker and one frame. While you're positioning the corner, you might notice the field of view going to wildly implausible values, 179 degrees or 1 degree. That's a normal piece of feedback.

Tip : If your initial corners aren't well placed, you may not be able to place the fourth where it should go. Hold down ALT/command while moving the fourth; SynthEyes will adjust the first three slightly for you.

You'll also notice that the tracker mini-view is showing the interior of the planar tracker, compressed into a nice little box. The 3-D Aspect value gives the aspect ratio of the planar tracker in the 3-D environment ( not the image!).

Now that you've placed the four corners, you can adjust them as needed, either by moving them in the camera view, or by dragging the corners inside the tracker mini- view. You can drag the whole tracker, or move the center of the tracker mini-view.

You can re-aim the tracker by dragging the apex, or change the FOV or aspect ratio with the spinners. There's a lot of functionality waiting for you in the Planar Tracking manual.

For this quick-start, place the cursor over one of the four corners, hold down shift, and scale the tracker up to encompass some more of the surrounding window structure. Release shift as you drag to allow the axes to scale independently. Then drag the whole thing a little to center it up. (Note that as you scaled or moved the planar tracker, the FOV and 3-D aspect don't change. The mouse operations occur in the 3-D environment; they are not 2-D effects.)

With our 3-D planar tracker positioned, it's ready to go, so hit the Play button

at center bottom of the display, or immediately above the channel selector icon , above the tracker mini-view. SynthEyes will track it through the shot. The additional rectangle that appears is the search region.

We now have a 3-D track. Click the lock icon to prevent redundant calculations and inadvertent changes.

You can scrub the shot in the camera view. You can see the stabilized pattern in the tracker mini-view, with some motion due to the non-planar covers above the building's windows, which SynthEyes has ignored.

Change the Layout to Quad, and scrub some more. You'll see the red rectangle moving around—that is your 3-D plane moving in space (relative to the camera).

Change the Layout to Perspective and scrub, there's your 3-D tracker moving.

Click Lock on the mouse bar at top of the Perspective view.

Wait! Now there are two rectangles! One is the 2-D rectangle, it is located correctly in the image. The other is the 3-D rectangle, it does not match because the SynthEyes camera Camera01 (and thus perspective view) is not using the Field of View required by the planar tracker.

Again, the planar tracker requires a specific field of view, from its Own FOV spinner. Any 3-D camera view must have exactly that same FOV, if the 2-D and 3-D rectangles are to be able to match up.

Hold down the control key and click the Planar Options tool gear . More about that in a moment, but for now the Planar Tracking script bar will pop up (also available normally from the Scripts menu).

Click the Save FOV as Known button (to run that script). Presto! Now there is only a single rectangle visible. Where did the 3-D rectangle go? It's right there now, exactly lined up with the 2-D rectangle.

The script copied the Own FOV into the Camera's Known field of view track. You can bounce back and forth between the Lens and Planar panels to verify that. With the right FOV in place, both rectangles match up.

Now that the FOVs match, we're ready for a 3D export. Clicking the After Effects 3-D button will launch the exporter (or via the File/Export menu). You can enter "tower" as the file name, and then control panel for the After Effects exporter will appear. It's a sophisticated thing in its own right, see the section After Effects 3-D Javascript Procedure for details. With the default settings (set your AE version!), the scene will be exported and launch automatically in After Effects.

That will give you a 3-D composition in After Effects with a 3-D camera and a moving Placeholder layer that is exactly your planar tracker; you can drop new imagery or comps into it. Further details are outside the scope of this quick-start.

You can also click AE Corner Pin on the Planar Tracking script bar (or on the File/Export menu). With this export, you will get a 2-D composition inside After Effects, which may be simpler if you're more familiar with AE's 2-D environment and only need to do a simple effect. (This export can export both 2-D and 3-D planar trackers.)

Of course, SynthEyes supports many more applications. Instead of special exports for each application, click the Export Prep button on the script bar (this is the script Planar Export Preparation).

Export Prep can set up the scene as a moving camera with a single stationary 3- D planar tracker, or as a stationary camera with multiple moving 3-D planar trackers. Or you can run it twice for both.

For the quick-start, select Animate Camera and leave the Plane Orientation at Front. Click OK. Change the Layout to Quad Perspective and then scrub through the shot. As you see, the 3-D planar tracker is now stationary, glued to the front view. The camera is now moving. The planar tracker has been augmented with an identically- sized and -placed mesh object. (In moving-object mode, there are additional SynthEyes moving objects, with the mesh parented to the moving object.)

This is a much better configuration for a gravity- and velocity-dependent particle effect, such as an animation of smoke pouring from the window. You can move the scene around using the Whole button on the 3-D panel if desired.

This 3-D planar tracker scene, as augmented by the Export Prep script, can be exported through a wide variety of SynthEyes's existing exporters, such as Cinema 4D, Filmbox, etc.

Of course, this quick-start has been just that, touching on some of the fundamentals. The planar tracking feature set is very deep. Here are some of the issues that require more attention, and that are covered in the planar tracking manual:

 Occlusion, where actors, vehicles, etc pass between the camera and the planar surface being tracked.

 Variable lighting conditions

 Multiple 3-D trackers in the same scene (all the FOVs must match, since the camera can only have one at a time)

You can see a additional controls on the Planar Options panel, found underneath the main planar panel on high-resolution displays, or by clicking the gear icon ( without holding control).

The feature set includes quite a few different planar tracker modes, both 2-D modes and various 3-D modes, including for zoom cameras. You can also supply your own on-set measurements for the rectangle dimensions (notice the 3-D Size spinner?) and known camera FOV.

Keep in mind that 3-D planar trackers are just one more tool in the toolbox, not a panacea or replacement for a solid 3-D track. Planar trackers use much smaller amounts of information from the scene, and consequently have inherently less accuracy. But for a lot of quick effects, they can be a great tool!

Again, for full details see the separate Planar Tracking Manual, which can be found via the Help menu of SynthEyes.

< Previous | Contents | Manuals Home | Boris FX | Next >