3D Scene
Description
The 3D Scene node provides an environment where you can 3D track with SynthEyes or load scenes in Alembic or FBX formats. Select tracked 3D points, solve them into a plane, and place cards in 3D space.
Using the 3D Scene Data port, you can:
• 3D Tracking
• 3D Head Tracking
• Use cards to Insert, Match Move and Stabilize in the Transform node
• Unproject images, paint or composite them, and then reproject the results
• Transform Paint strokes with cards
• Import cards and convert them to layers for use in Roto and Tracker capable nodes
• Stabilize and Unproject directly in the Viewer using cards
• Convert 3D trackers into 2D point trackers
Go to the 3D Scene tutorials to see how it works.
Node Group
3D.
Toolbar
SynthEyes
The 3D Scene node features a built-in SynthEyes camera tracking and solving system.
See the SynthEyes section below for more information.
Head Track ML
Automatically creates a 3D head mesh and tracks facial shape, pose, and expressions over time. When used with the Unwrap and Rewrap nodes, it enables stable, face-aware beauty effects.
See the Head Track ML section below for more information.
Select Points
Selects tracked 3D points to calculate a solved plane to be used as a surface. After a point selection, a purple line appears from each selected point to its closest point on the plane. Longer lines indicate points that fit the plane less accurately.
Card
Adds a card object to the scene which you can use to insert 2D layers or unproject, paint or composite, and then reproject the original motion.
Position (Q)
Moves the selected object along the x, y or z axis.
Scale (E)
Scales the selected object along the x, y or z axis.
Rotate (W)
Rotates the selected object along the x, y or z axis.
Sphere
Renders a wireframe sphere to check the accuracy of the 3D track.
Controls
Load 3D Scene
Imports Alembic and FBX files.
Depending on the selected object, the following parameters are available.
Camera
Mode
Determines the camera projection type.
Perspective
Simulates a real-world camera, where objects diminish with distance and parallel lines converge.
Orthographic
Removes perspective, keeping object scale constant regardless of depth for technical and alignment views.
Lens Squeeze Ratio
Adjusts the horizontal scaling of the image to simulate anamorphic lenses. Values other than 1.0 introduce horizontal stretch or compression, useful for matching or correcting anamorphic footage.
Near Clip
Sets the minimum distance from the camera at which geometry is rendered. Objects closer than this distance are not visible.
Far Clip
Sets the maximum distance from the camera at which geometry is rendered. Objects beyond this distance are not visible.
Vertical Aperture
Defines half of the sensor’s vertical size and influences the vertical field of view in combination with focal length.
Horizontal Aperture
Defines half of the sensor’s horizontal size and controls the horizontal field of view.
Sensor Fit
Specifies how the sensor is fit to the output image when their aspect ratios differ. This determines whether the horizontal or vertical field of view is preserved, or if the image is scaled to fill the frame.
Transform
Position
Moves the selected object along the x, y or z axis.
Scale
Scales the selected object along the x, y or z axis.
Rotate
Rotates the selected object along the x, y or z axis.
Reference Frame
The Reference Frame is used when a transform is requested from the selected object. It ensures that the transform on this frame will be the identity.
Note: The Transform and Reference Frame parameters are available for the Scene, Solved Plane, Card and Points objects.
Card
Dimensions
Sets the width and height of the card in 3D local units.
Color
Sets the color of the selected object.
3D Views
Located in the Viewer, there are a number of 3D Views to visualize the 3D point cloud. See the 3D Views options for more information.
SynthEyes
The 3D Scene node features a built-in SynthEyes camera tracking and solving system.
Workflow
SynthEyes works by tracking the scene and deriving the camera’s movement based on the relative motion of tracked features.
Step 1 - Creating A Camera Solve
Before adjusting settings, click the Solve button and review the initial results. For most shots, default settings provide a solid starting point.
Step 2 - Masking Problematic Objects
To prevent obstructions from interfering with the solve, isolate them using a Roto or Matte Assist ML node. Then, connect the output to the Occlusion Matte input and enable the Matte > Use Occlusion Matte parameter to exclude those areas from processing.
Common Items to Mask
• Large moving objects (e.g., actors, cars)
• Clusters of smaller, high-contrast moving objects
• Distant areas like clouds or skylines that don't contribute to the solution
Step 3 - Solve the Shot
Click the Solve button to begin processing.
Solve Process Stages
• Blipping: Identifies auto features (blips) while ignoring masked areas.
• Solving: Generates a camera solution.
Post-Solve Review
• The Viewer displays the solved camera, features, and ground plane grid.
• The Solve Data column presents the average pixel error for the scene.
• Focal length updates based on solver calculations.
A pixel error below 1.0 is a good solve while above 1.5, you should consider parameter adjustments.
Camera
Solve
Start the solve process using the selected parameter settings. Running it again will clear previous results.
Focal Length
Enter the known focal length here. By default, Unknown is checked, disabling the field. After solving, this field updates to the calculated focal length.
Film Back
Defines the camera’s horizontal and vertical film back (also known as sensor size). Adjusting one value automatically updates the others to match the clip’s dimensions.
Reset All
Removes all solved data from the scene and resets all parameters.
3D Motion
Defines the type of movement within the scene, based on SynthEyes conventions.
Motion Type
Normal Motion
The most common camera movement--typically the default starting point.
Crash Pan
Used for rapid panning shots.
Low Detail
Ideal for scenes with minimal trackable detail, such as green screen backgrounds.
Zoom Lens
Enable when the camera uses lens zoom during the shot.
On Tripod
Indicates a stationary tripod-mounted camera, used for PTZ (pan-tilt-zoom) solutions.
Locked Shot
If the camera remains stationary, enable this option. It’s designed to assist in solving motion for objects when there’s no camera movement.
Corners
Recognizes structural features like buildings, windows, and bricks, aiding in non-organic scene solving.
Features
Defines the automatic tracking features.
Min Trackers/Frame
Specifies the number of auto feature trackers to detect per frame.
Max Tracker Count
Sets the maximum number of auto feature trackers across the entire frame range.
Add More Trackers
Adds additional trackers between existing paths to increase point cloud density.
Small Blip Size
Defines the smallest detectable feature size (in pixels).
Large Blip Size
Defies the largest detectable feature size (in pixels).
Occlusion Source
Excludes an area from being tracked.
Source Alpha
Uses the source alpha.
Occlusion Matte
Uses the occlusion matte input.
Invert
Inverts the matte.
Pre-Processing
Images that contain film grain or video noise, lack contrast or sharpness as well as images that have flicker can be tracked more accurately by applying certain filters during the tracking process. Filters such as Blur, Sharpen, Contrast, Gamma, Denoise and Deflicker will increase tracker accuracy for problematic images.
Blur
Applies a blur to the image features.
Sharpen
Sharpens image features.
Contrast
Increases contrast.
Gamma
Increases brightness by leaving the white and black points the same and only modifies the values in-between.
Denoise
Applies a denoise algorithm to smooth out noise while retaining detail.
Deflicker
Corrects image flicker or brightness variation over time, which has a habit of interfering with tracker accuracy.
Preview
Shows the effect of the Pre-Processing parameters on the entire image in the Viewer.
Solve Data
Displays solve results.
Solve Output
Opens a dialog displaying messages from the SynthEyes process, providing useful debugging insights for solving issues. Also includes a summary of the Camera Solve.
Average Error
HPIX graph displaying the scene’s average error per frame and overall error. See Average Error below for details.
Clean Up Features
Cleans up a solve after it has been completed.
Bad frames
Disable bad frames in solved features equal to or over the HPIX threshold.
Short-range Trackers
Remove features that were solved from ranges shorter than the set frame number.
High-Error Trackers
Remove any features that have too many bad frames. Based on the percentage of frames that are bad and the upper HPIX of a bad frame.
Fix Features
Clean up the solved features based on the settings above.
Refine Solve
Updates a cleaned-up solution without starting over, providing a quick way to minimize pixel drift.
UV Ref Frame
Defines the reference frame for UV coordinates.
SynthEyes Terminology
The 3D camera solver leverages SynthEyes tracking and solving processes, adopting its terminology to describe key functions, for instance, Features, Blips, and Trackers.
Features
A feature is an individual point of interest tracked in the scene, either in 2D (x, y) or as its 3D (x, y, z) equivalent after solving.
Blips
A blip is SynthEyes' term for a point of interest within a single frame. During auto feature generation, SynthEyes analyzes each frame, identifying and labeling blips based on notable features. Within Silhouette, this process runs invisibly as part of the camera solve, but blip size can be adjusted via parameters.
Trackers
For auto features in SynthEyes, tracking occurs by linking blips that correlate across frames, forming feature trackers.
Adding More Trackers
The Add More Trackers field increases point density between existing trackers. By analyzing the paths of current auto features, more features are placed along those tracks. While adding more trackers can improve solve accuracy, it also extends solve time.
Solve Quality & Average Error
When a solve is complete, a graph appears under Solve Data, displaying the Horizontal Pixel Error (HPIX), which represents the average pixel error across frames.
Evaluating Solve Accuracy—Good vs. Bad Error
Aim for an average error below 1 pixel.
• 1-2 pixels isn't terrible, but a value of 0.9 or lower is considered excellent.
• Higher error values indicate a poor solution.
SynthEyes filters out high-error features during cleanup, which helps improve solve quality. But if a shot fails to solve, it’s often because extreme pixel errors caused too many features to be removed. Try adding more trackers and adjust the blip size to give the solver more reliable data.
Head Track ML
Automatically generates a 3D head mesh and tracks facial shape, pose, and expressions over time. Mesh fine-tuning is achieved through trackable, keyframeable deformer anchor points, ensuring precise alignment with facial features. When combined with Paint, Roto, Unwrap, and Rewrap nodes, it enables stable, face-aware beauty effects.
Workflow
1 Create and Track A 3D Head Mesh
• Select the Head Track ML tool.
• Set Track to Heads.
• Click Track Forward or Track Reverse, depending on the desired tracking direction and a 3D head mesh is created and tracked.
Note: Smoothing is enabled by default, as the Head Tracker may produce jittery results without it.
2 Mesh Refinement
• With a head selected in the Object List, choose Add Deformer and then click to add anchor points.
• Adjust the anchor points so the mesh aligns accurately with the facial features. Anchor points can be positioned and keyframed manually, or tracked automatically.
• Be sure to add anchor points in any areas where you’ll be doing roto, paint, or compositing to ensure those regions remain firmly locked in place.
• In the Object List, select the anchor points to be tracked and set Track to Anchors.
• Click Track Forward or Track Reverse, depending on the desired tracking direction to perform an additional refinement pass.
Track
Tracks either head meshes or deformer anchor points.
Heads
Creates 3D head meshes and tracks facial shape, pose, and expressions over time.
Anchors
Tracks deformer anchor points. In the Object List, Tracker direction icons are enabled when one or more anchor points are selected, or when the deformer itself is selected (tracking all anchor points). To delete anchor points, click to select a point--or Shift-drag to select multiple--and press Delete.
Tracker Direction
The tracker can track both forward and backward one frame at a time or for the entire duration.
Track Backward One Frame
Tracks backward one frame at a time.
Track Backward
Tracks backward to the beginning of the Timebar.
Track Forward
Tracks forward to the end of the Timebar.
Track Forward One Frame
Tracks forward one frame at a time.
Smooth
Turns track smoothing on or off. Enabled by default, as the Head Tracker may produce jittery results without smoothing.
Amount
Sets the amount of frames to calculate the smoothing.
Occlusion Source
Excludes an area from being tracked.
Source Alpha
Uses the source alpha.
Occlusion Matte
Uses the occlusion matte input.
Invert
Inverts the matte.
Add Deformer
Adds a deformer to the selected head, allowing you to click directly on the mesh to place anchor points in areas that require manual positioning or more precise stabilization.
Note: To add anchors, the deformer or another anchor needs to be selected in the Object List.
Anchor Point Parameters
Each anchor point has an associated Offset and Weight parameter visible in the Object window.
Offset
Controls how much the mesh is offset relative to the selected anchor point.
Weight
Weights the influence of the Offset.
Accuracy
In Track > Anchors mode, Low, Medium, and High accuracy options provide a trade-off between processing speed and result quality.