3D Scene
The 3D Scene node provides an environment where you can 3D track with SynthEyes or load scenes in Alembic or FBX formats. You can select 3D tracks, solve them into a plane, and place cards in 3D space.
Paint/Compositing With Unproject/Reproject
The Unproject/Reproject workflow consists of unprojecting motion derived from a 3D scene (effectively stabilizing all motion), painting or compositing on the locked down sequence and then reprojecting back to the original motion.
1 Add a source file and connect it to a 3D Scene node.
2 Select the SynthEyes tool to perform a 3D track.
Note: To bring in external 3D tracking data, use SynthEyes standalone, Mocha Pro, or similar software to track your scene. Export the result as an Alembic or FBX file, then click Import 3D Scene in the 3D Scene node to load it.
3 Press Solve.
When tracking is complete, tracked 3D points appear.4 Select Clean Up Features, enable the desired options and press Fix Features.
This cleans up the initial solve.
5 Click on Refine Solve.
Refine Solve updates the cleaned-up solution without starting over, providing a quick way to minimize pixel drift.
6 With the Select Points tool enabled, drag select the tracked 3D points in the area where you want to work.
7 Choose the Card tool and click-drag to create a card. The card appears and the area is automatically solved into a 3D plane.
As you drag through the sequence, the card is transformed by the tracking data.
8 Connect the Source node to a Transform > Unproject node. Then connect the Data output of the 3D Scene node to the Data input on the Unproject node (located at the top right of the node).
9 Single-click in the center of the Unproject node to view and edit it.
In the Surface pop-up, Auto is selected by default. Because the 3D scene contains only one surface, it is automatically selected and the motion is stabilized and unprojected.
10 If the result is stretched horizontally or vertically, change the Fit Mode to a different option.
11 Paint or composite after the Unproject node.
12 Add a Transform > Reproject node after the paint or compositing nodes and connect the Data output of the Unproject node into the Reproject node’s Data input located at the top right of the node.
The painted or composited result is reprojected back to the original motion.
Paint Using Viewer Stabilization & Unproject
The downside of the Unproject/Reproject approach is that the image is filtered twice resulting in some slight image degradation, but nothing a bit of sharpening won’t solve.
A novel twist on the Unproject/Reproject workflow is Silhouette’s Viewer stabilization which non-destructively stabilizes and optionally unprojects the Viewer. The beauty of this approach is that image pixels are not modified during stabilization/unprojection--only the Viewer is stabilized.
1 Follow the instructions in the previous tutorial for creating a 3D Scene node, selecting points and adding a card.
2 Select the Stabilize icon above the Viewer and the Stabilize toolbar appears.
3 Choose the card you just created from the list and select Enable.
The Viewer is now stabilized based on the selected card’s tracking data.
4 Enable Unproject in the Stabilization toolbar.
Unproject remaps the area inside the layer or card so that it becomes flat--making it easier to work with pixels that have a regular repeating pattern or tend to be more horizontal or vertical. It's most useful to help correct for foreshortened projected pixels. For example, imagine you're the camera and you reposition yourself to look head-on into an area of the image that has perspective.
Note: Clicking the Stabilize icon again hides the Stabilize toolbar, but stabilization remains active until disabled.
5 Add a Paint node to the source file.
6 Connect the Data output of the 3D Scene node into the Paint node’s Data input located at the top right of the node.
7 Single-click in the center of the Paint node to view and edit it.
8 In the Paint node, select the Card in the Paint > Transform menu.
9 Paint a stroke, select it and enable Match Move in the Auto Paint window.
10 Auto Paint the stroke throughout the sequence.
Strokes are match moved and auto painted while the Viewer is stabilized and unprojected.
11 To disable Viewer stabilization, click the Stabilize icon, deselect Enable, and the Viewer is returned to its normal state.
12 Click the Stabilize icon again to close its toolbar.
When you play now, the painted strokes will appear normal and match moved to the sequence.
Roto/Paint/Compositing With 3D Head Meshes
Facial beauty work can be streamlined using a 3D Scene node Head Tracker, Unwrap, and Rewrap workflow. The process involves tracking a 3D head mesh and unwrapping its surface into 2D UV space to create a flat, stabilized representation. This view simplifies roto, paint, and beauty work by removing motion. After adjustments are made on the stabilized sequence, the result is rewrapped onto the original facial performance.
3D Head Tracking
1 Add a source file and connect it to a 3D Scene node.
2 Select the Head Track ML tool.
3 Set Track to Heads.
4 Click Track Forward.
Note: Smoothing is enabled by default, as the Head Tracker may produce jittery results without it.
When tracking begins, a 3D head mesh is generated and tracked.
Mesh Refinement
If the mesh is not fitting the face as expected, you can adjust it with anchor points.
1 With a head selected in the Object List, choose Add Deformer and then click the mesh to add anchor points.
2 Adjust the anchor points so the mesh aligns accurately with the facial features. Anchor points can be positioned and keyframed manually, or tracked automatically.
3 Be sure to add anchor points in any areas where you’ll be doing roto, paint, or compositing to ensure those regions remain firmly locked in place.
4 In the Object List, select the anchor points you want to track, or select the Deformer to track all anchor points.
5 Set Track to Anchors.
6 Click Track Forward or Track Reverse, depending on the desired tracking direction to perform an additional refinement pass.
Unwrap/Roto, Paint, Or Composite/Rewrap
1 Connect the Source node to a Transform > Unwrap node. Then connect the Data output of the 3D Scene node to the Data input on the Unwrap node (located at the top right of the node).
2 Single-click in the center of the Unwrap node to view and edit it.
In the Surface pop-up, Auto is selected by default. Because the 3D scene contains only one surface, it is automatically selected and the image is unwrapped, flattened and stabilized.
3 Roto, paint or composite after the Unwrap node.
4 Add a 3D > Rewrap node after the Roto, Paint or compositing nodes and connect the Data output of the 3D Scene node into the Rewrap node’s Data input located at the top right of the node.
Note: When using Paint in the Unwrap/Rewrap workflow, it is best to connect the Paint Only output to Rewrap, as the resulting composite in the next step will appear more natural and seamless.
The Rewrap node reprojects the roto, paint, or composite created in 2D UV space back onto the 3D model, restoring the effects to their correct position on the model’s surface.
The last step is to composite the re-wrapped result over the source.
5 Connect the Rewrap node into the Foreground input of the Composite node and the Source node into the Background input.
3D Tracker / Card Conversion To Point Trackers or Layers
There are various options to convert 3D trackers or cards in any Tracker capable node.
Layer From 3D Points
Generate a tracked layer from a selection of four or more 3D track points. The layers can then be used for Match Moving and Stabilization in any node that supports layers.
1 Connect the Data output from a 3D Scene node to the Data input located at the top-right corner of a Tracker node.
2 With the Tracker tool selected, click on the Point Tracker tab.
3 Select 3D track points:
• Drag select a minimum of four 3D track points.
or
• Select a one or more shapes and the points within the shape are automatically selected.
4 Choose Create > Layer from 3D Points.
A tracker layer is created in the Object List. If shapes were selected, they are placed within the layer.
Convert Cards To Layers
Convert 3D Scene node cards to layers.
1 Connect the Data output from a 3D Scene node to the Data input located at the top-right corner of a Tracker node.
2 Select the Tracker tool.
There are two methods to convert cards to layers: Import from Data Port and Point Tracker > Create menu.
Import from Data Port
1 Choose Import From Data Port in the I/O section.
All cards from the 3D Scene node are imported and converted to layers in the Object List.
Point Tracker > Create Menu
1 Select the Point Tracker tab.
2 Click on a card to select it.
3 Choose Create > Layer from Card.
The selected card from the 3D Scene node is converted to a layer in the Object List.
Convert 3D Trackers To 2D Point Trackers
1 Connect the Data output from a 3D Scene node to the Data input located at the top-right corner of a Tracker node.
2 With the Tracker tool selected, click on the Point Tracker tab.
3 Drag select the 3D track points to be converted to 2D point tracks.
4 Choose Create > Point Trackers from 3D Points.
Point trackers are created and appear in the Object List.