How to Set a Deepfake Detector Threshold: Calibration and Abstention

How should I set the threshold for a deepfake detector?

Pick it from data and costs, in that order. Run the detector on a validation set drawn from your channel, separate from the final test set, and plot false positive rate against recall across every possible threshold. Then mark the points your workflow can live with. A selfie check that routes flags to a human can accept a higher false positive rate than an upload filter that blocks content automatically.

If the scores are calibrated, decision theory gives a starting point. Flag a file when the estimated probability that it is fake is greater than the cost of a false positive divided by the sum of both error costs. If a missed deepfake costs 20 times as much as a wrongly flagged real customer, that ratio is 1 in 21, so you would flag anything above roughly 0.05. Few teams use the formula unadjusted, but it shows why high-loss workflows run low thresholds.

Freeze the chosen threshold before the final test, then confirm it on held-out data. Choosing the threshold on the test set and reporting results from the same set overstates performance.

What calibration means for a deepfake detection score

A score is calibrated when it behaves like a probability: among all files scored around 0.7, about 70 percent turn out to be fake. Many detector scores are not calibrated. They rank files well, so a higher score really is more suspicious, but 0.7 might correspond to a 30 percent or a 95 percent chance of being fake depending on the model and the data.

Check calibration with a reliability plot. Group validation results into score bands, then compare the share of actual fakes in each band with the band's average score. If the curve sits far from the diagonal, either recalibrate on your data with a method such as isotonic regression or Platt scaling, or treat the score purely as a ranking and pick thresholds from observed error rates.

The VendorBench-100 authors report that some systems that ranked well by ROC-AUC were poorly calibrated at their default thresholds. TruthScan, for example, scored a ROC-AUC of 0.923 but an MCC of 0.322 at its default setting. The ranking ability was there; the shipped threshold did not use it.

Why the share of fakes in your traffic changes the right threshold

Precision, the share of flags that are actually fake, depends on how common fakes are. Most buyers underestimate this effect. The table works through a detector with 95 percent recall and a 2 percent false positive rate on 10,000 submissions at three prevalence levels.

At low prevalence, even a strong detector produces more false flags than true ones. That is not a defect. It means low-prevalence workflows need either a stricter threshold, a second signal before action, or a review step sized for the false flags.

Designing an abstention band for uncertain results

A single threshold forces every file into pass or fail, including the ones the detector is unsure about. Two thresholds create three outcomes: low scores pass, high scores are blocked or escalated, and the middle band goes to review or a request to recapture. That middle band is the abstention zone.

Size the band to your review capacity. If reviewers can clear 400 cases a day and 20,000 files arrive, the band can hold about 2 percent of traffic, plus a margin for spikes. Track what reviewers decide in the band; if nearly all of it turns out authentic, narrow it from the bottom.

Treat unsupported files, corrupt uploads and timeouts as their own states, separate from the abstention band. Merging them hides operational faults inside detection statistics. For biometric presentation attack detection, ISO/IEC 30107-3 asks for error rates reported at a stated operating point, which is a useful habit for content detection as well.

Controlling threshold changes after launch

Thresholds drift out of date when attackers change tools, when your app changes image quality, or when the vendor updates the model. Assign one owner who can approve changes, log every change with its reason and date, and test the new value on recent labelled data before release.

Re-run the threshold analysis after any vendor model update, even when the vendor says scores are unchanged. Keep the ability to identify which threshold and model version produced each historical decision, because disputes and audits will ask.

How NeuroVerify supports your threshold policy

NeuroVerify returns one unified risk score with signal-level evidence from six signals, including face manipulation detection, AI generative model analysis and AI pixel-level manipulation. Your policy sets the thresholds and the actions; we do not make the decision for you. Signal-level evidence helps in the review band, because a reviewer can see whether the concern came from a swapped face or from a generated background.

On the public VendorBench-100 benchmark, Neural Defend ranked first of 36 systems with an MCC of 0.876, a specificity of 0.857 and a recall of 0.987, results that reflect performance at an operating threshold rather than ranking ability alone. One VendorBench author also co-authored our ARCAS 1B paper, and the corpus is public. During evaluation we help you fit thresholds on your own validation data and confirm them on a held-out set.