Mask ML
Description
Mask ML is an easy to use point and click, single frame mask generator utilizing machine learning object segmentation. A single click selects a defined segment of the image while successive clicks add adjacent areas to the object. The masks are useful as input to the Matte Assist ML node where the mask is propagated over time or to limit strokes in the Paint node.
A text-prompted object detection workflow is also possible where masks are auto-generated in Mask ML and then propagated in Matte Assist ML--usable via UI or command line. This unlocks a fully automated workflow: prompt, segment, propagate, render. For most shots, it’s hands-off. But when objects deform or occlude heavily, a few manual keyframes may still be needed.
Go to the Mask ML tutorial to see how it works.
Node Group
AI/ML, Key.
Toolbar
When the Mask ML node is selected, six tools are selectable from the Toolbar to the left of the Viewer.
Add
Click within an object to select it. With each click, a green point appears. Add additional points as needed.
Note: When you hover over an area, a preview overlay appears showing the region that will be segmented when you click.
Subtract (Right-click)
Click on any unwanted alpha areas outside of the selected object. With each click, a red point appears. Add additional points as needed.
Note: Ctrl/Cmd-click a point to delete it.
Object
Select Object
Drag out a rectangle to select an entire object.
Select Sparse Objects
Drag out a rectangle to select the individual parts of an object.
Select Dense Objects
Drag out a rectangle to select all segments within it.
Note: Both the Sparse and Dense Objects tools can create duplicates on some images.
Foreground (Shift-F)
Once an initial selection has been made, click the Foreground icon and draw to add to the foreground area--the region to mask out.
Note: If you Shift-click with the left mouse button and then click in a different location, a straight line is drawn. If you keep Shift depressed while clicking, you will create interconnected straight lines.
Background (Shift-B)
Once an initial selection has been made, click the Background icon and draw to remove foreground areas.
Note: To delete a Foreground or Background stroke, simply Shift-Alt-click on it.
Controls
Auto DOD
Automatically creates a DOD (Domain of Definition) around the mask resulting in a more accurate result.
Prompt
Automatically detect arbitrary objects in images using text prompts--such as category names or descriptive phrases--instead of manual selection. Supports automation via scripting.
Prompt
Enter a description of the object you want to detect--such as person, bicycle, or apple. To detect multiple object types, add each prompt separated by a comma.
Threshold
The model returns a bunch of bounding boxes with probability scores with the threshold set at .40--anything lower than that is discarded, anything higher is kept. Lower the value to include more of what you are prompting for.
Mode
Selects either Object or Parts mode.
Object
Select the entire object.
Parts
Selects the individual parts of an object.
Detect Objects
Uses the provided prompts to locate objects in the image, drawing bounding boxes and generating masks.
Note: You can automate object detection, mask creation, and propagation by running the auto-matte.py script located in the Silhouette-Install/resources/scripts folder.
Point Opacity
Adjust the Add and Subtract point opacity with the Viewer > Blend Amount.
Multiple Objects / Object List
Mask ML supports multiple objects in the Object List. When Mask ML and Matte Assist ML’s data ports are connected and Matte Assist ML > Mask Source > Cryptomatte is selected, each Mask ML object is considered a separate object.
• The first point or rectangle added automatically creates a new object.
• To manually create an empty object, click the Add Object icon in the Object List.
• Selecting an object reveals its controls; changes apply only to the selected object.
• Use the Object List to rename, delete, or toggle visibility per object.
• If you delete all points from an object, it becomes inactive and won’t contribute to the render.