 |
Tracking troubleshooting
The following topics offer solutions for commonly encountered tracking issues.
❙ Tracking results in only a few (or one) face markers being tracked.
❙ Tracking results in a significant number of mismatched or swapped marker points.
❙ Tracking fails to track eyelid marker points.
❙ Tracking fails to track points that move in and out of occlusion in a number of camera views.
If the Tracking process has generally worked, but reconstructed 3D tracking results for one or more of the points is jittery (or jumpy), see Jittery 3D reconstructions troubleshooting
Tracking results in only a few (or one) face markers being tracked.
The most likely cause of this problem is that you have accidentally selected and tracked only a few (or one) points.
If this is the case, undo (Ctrl-Z) the last operation and re-track with all points de-selected; this will cause tracking to be run for all points.
If this was not the cause of the issue, try some of the following techniques for handling tracking failures.
Tracking results in a significant number of mismatched or swapped marker points.
Swapped or mismatched marker points are very time-consuming to fix manually in the final 3D data, so try to prevent this type of tracking error. Depending on the length of the take and the number of errors present, you can choose one of two general workflows to address the problem:
♦ Undo the tracking result (Ctrl-Z) and re-track the whole take with different tracking parameters.
Or:
1. Delete all mismatched or swapped points. To do this:
a. Select the points, and set the Active Range to the part of the take where the errors are present.
b. On the Edit menu, click Clear Selected [Active Range].
2. Before re-tracking the data, make sure that in the Properties pane, the Tracking Parameters are set as follows:
– Ensure that Remove Bad 2D Projections is enabled, and that Max Allowed 2D Projection Error is set to a reasonably low value (typically less than 5 pixels).
– Ensure that Remove Bad 3D Points is enabled, and that Max Allowed 3D Point Error is set to a reasonably low value (typically less than 5mm, but may need to be lower for a dense marker set).
– Ensure that the Face Point Tracking Distance and Brim Point Tracking Distance parameters are not set higher than they need to be for the amount of frame-to-frame motion present in the take. As an example, if the maximum frame-to-frame distance during the take moved by a face point is 50 pixels (as measured using the Start Measurements function in the 2D context menu), you should set the Face Point Tracking Distance parameter to a value just slightly higher than this, eg, 55 pixels.
3. Re-track only those points that were mismatched with different tracking parameters.
Tracking results in many brim points failing to track for some frame(s), but face point tracking is OK over the same frame range.
If a large number of brim points fail to track at certain frame(s) in the take, to address the cause of the problem you can change the tracking (and possibly blob detection) parameters, and re-run the automatic tracking in the failed frames for all points using Process > Track Forwards [Sequence] or Process > Track Forwards [Single Frame].
Work through the following steps:
1. Are blob detections present for the marker points which failed to track?
If not, ensure that the Blob Parameters are set correctly as explained in Blob detection troubleshooting.
2. Does the brim jump or move significantly at the start of the tracking failures?
If so, increase the Brim Point Tracking Distance tracking parameter so that it is just slightly larger than the maximum frame-to-frame image distance (in pixels) that the brim points move.
3. If steps 1. and 2. fail to address the problem, you may also need to relax other tracking parameters slightly, eg, decrease the Patch Match Threshold, and increase the Max Allowed 2D Projection Error and Max Allowed 3D Point Error parameter values.
Tracking results in many face marker points failing to track for some frame(s), but brim point tracking is OK over the same range.
If a large number of facial marker points fail to track at certain frame(s) in the take, to address the cause of the problem you can change the tracking (and possibly blob detection) parameters, and re-run the automatic tracking in the failed frames for all points using Process > Track Forwards [Sequence] or Process > Track Forwards [Single Frame].
Work through the following steps:
1. Are blob detections present for the marker points which failed to track?
If not, ensure that the Blob Parameters are set correctly as explained in Blob detection troubleshooting.
2. Do the facial marker points which fail to track move significantly at the start of the tracking failures?
If so, increase the Face Point Tracking Distance tracking parameter so that it is just slightly larger than the maximum frame-to-frame image distance (in pixels) that the face points move.
3. If steps 1. and 2. fail to address the problem, you may also need to relax other tracking parameters slightly, eg, decrease the Patch Match Threshold, and increase the Max Allowed 2D Projection Error and Max Allowed 3D Point Error parameter values.
When re-tracking selected point(s) during a second tracking pass trying different parameters, the tracking fails, despite good detections in multiple camera views.
Occasionally, you may come across points where, despite trying a range of sensible tracking parameters, and good detections being present, the point still fails to track. This is generally caused by the correct tracked point candidate either:
❙ Being rejected as a Bad 2D Projection or a Bad 3D Point; or
❙ The correct tracked point candidate hitting one of the tracking parameter thresholds, and not being considered further.
To solve this issue, re-track the failed point with the following changes to tracking parameters in the given order:
1. Disable the Remove Bad 2D Projections and/or Remove Bad 3D Points options, or leave these options enabled, but increase the Max Allowed 2D Projection Error and/or Max Allowed 3D Point Error parameter values.
2. If there is significant frame-to-frame motion present in the frame where the tracking failure occurs, if the failed point is on the face, increase the value for the Face Point Tracking Distance parameter; or if the failed point is on the brim, increase the value of the Brim Point Tracking Distance.
3. Decrease the Patch Match Threshold parameter value slightly eg, to 0.6.
Tracking fails to track eyelid marker points.
Tracking eyelid marker points can be prone to failure, as these points tend to move in and out of occlusion as the actor blinks, and also may move in a way which is uncorrelated with the motion of neighboring points.
To improve eyelid tracking results, complete the following steps:
1. Re-track the failed point(s) with the following changes to tracking parameters:
– Ensure that the Allow Single Sensor Tracking parameter is enabled.
– Decrease the Patch Match Threshold tracking parameter value slightly e.g. to 0.6.
– Increase the Face Point Max Elastic Prediction Distance tracking parameter value slightly e.g. to 30 pixels.
2. If the point(s) still fail to track, use the Target Tracker to track the failed point(s) manually. For detailed information on how to do this, see Target tracking.
Tracking fails to track points that move in and out of occlusion in a number of camera views.
Tracking marker points which move in and out of occlusion (eg, lip points and/or eye points) can be prone to failure, but there are some options which can improve your chances of tracking the point successfully.
❙ If the marker is completely occluded in all camera views, then it is will not be possible to track it. In this case, the best you can hope for is to use Process > Interpolate Gaps to fill in the predicted position of the marker in the frame(s) in which it is completely occluded.
❙ If the marker point is present in one or more camera views, then try the following:
a. Ensure that the Allow Single Sensor Tracking parameter is set to enabled, and possibly relax the Single Sensor Tracking Threshold and Patch Match Threshold tracking parameter values slightly (eg, to 0.75mm and 0.6 respectively).
b. Select the failed points, and re-track automatically in the failed frame(s).
c. If the point(s) still fail to track, use the Target Tracker to track the failed point(s) manually. For detailed information on how to do this, see Target tracking.