What Does a Deepfake Detection Score Mean, and How Should You Read One?

What Does a Deepfake Detection Score Mean?

A deepfake detection score is a number a model produces to express how much a file resembles the manipulated or generated examples it learned from. Higher usually means more suspicious. What the number does not tell you, unless the vendor says so explicitly, is the chance that this particular file is fake.

That distinction matters because people read 0.9 as 90% likely. If the model is uncalibrated, 0.9 might correspond to a much lower or higher real rate of fakes. If your traffic contains very few fakes, even a well-calibrated 0.9 can produce many false alarms, because a small error rate applied to a large volume of genuine files adds up.

A score also describes a file, not a person or an event. It does not show who made the media, whether they intended to deceive, or whether the scene it depicts happened. Those questions need other evidence.

How to Read a Deepfake Detector Score

Read a score in four steps before acting on it. First, find out what kind of number it is: a calibrated probability, an uncalibrated confidence, a ranking value or an anomaly score. Chuan Guo and colleagues showed at ICML 2017 that modern neural networks are often poorly calibrated and that a one-parameter method, temperature scaling, can correct much of the problem. Ask whether your vendor calibrates and on what data.

Second, find the threshold and what it was chosen to achieve. A threshold set to keep false positives below 1% behaves very differently from one set to catch 99% of fakes. Third, look at the evidence behind the score: which signals fired, which region of an image or which frames and audio segments drove it. Fourth, check fit: is this file like the media the detector was validated on, or is it heavily compressed, very short, a screenshot, or from a generator released last week?

Only then decide. A high score with clear signal evidence on a good-quality file is strong. A borderline score on a blurry forward is weak in either direction and belongs with a human reviewer.

How Does the Threshold Change What a Score Means?

The threshold converts a continuous score into a decision, and every threshold is a trade. Lower it and you catch more fakes while flagging more genuine customers; raise it and you inconvenience fewer people while missing more fakes. The right point depends on the cost of each error in your workflow, which is a business decision.

The VendorBench-100 benchmark shows how much this matters. Its authors report that MCC and ROC-AUC are strongly correlated across the 36 systems tested, but that some strong rankers are miscalibrated at their default threshold, so a high ROC-AUC can overstate real-world deployability. TruthScan, for example, scored a ROC-AUC of 0.923, meaning it ranks fakes above real images well, yet its MCC at the default threshold was 0.322 and its accuracy 0.49. Hive showed the same pattern, with ROC-AUC 0.90 and MCC 0.290. Neural Defend scored 0.979 and 0.876.

Set thresholds per workflow, write them into a versioned policy, and re-check them whenever the model or your traffic changes.

Why Do Different Deepfake Detectors Give Different Results for the Same File?

Different detectors give different results because they are different instruments. They read different evidence, such as pixel artefacts, frequency patterns, face geometry or audio spectra. They were trained on different generators, so one may know a new model and another may not. They preprocess differently, cropping faces, resizing or sampling frames in their own ways. And they report on different scales with different default thresholds, so a 0.6 from one tool and a 0.6 from another do not mean the same thing.

Disagreement is information, not noise. When two credible detectors split on a file, it usually sits near a boundary: a lightly edited photo, a heavily compressed clip, or a generator one of them has not seen. That file deserves a human look and source checks. Averaging the two scores hides exactly the uncertainty you need to see.

In-the-wild data makes the spread wider. On Deepfake-Eval-2024, open-source detectors lost about 45% to 50% of their AUC compared with earlier benchmarks, and the loss was not uniform, so tools that agree on lab data can diverge sharply on real circulating media.

What Should a Good Detection Result Include?

A result you can act on and defend later should include more than one number. Look for these elements:

NIST's AI 100-4 report recommends that detection evaluations reflect the content a tool will actually see, including benign distortions and unseen generators. The same logic applies to reading results: a score is only as meaningful as the match between the file and the conditions under which the detector was tested.

• The overall score and the threshold or band it falls in under your policy.
• Signal-level evidence: which detectors fired, and where in the image, video or audio.
• The model name and version, so results can be compared over time.
• Input quality notes, such as resolution, duration and compression, and whether the file was within tested conditions.
• A processing status that is separate from the result, so a timeout or unsupported file is never mistaken for a clean pass.

How Neural Defend Reports Detection Results

Neural Defend's NeuroVerify returns one unified risk score with signal-level evidence from six signals: visual safety screening, live presence verification, face manipulation detection, AI generative model analysis, AI pixel-level manipulation and AI content compliance. The customer's policy decides what each score means in each workflow, and eligible image requests return in under 200 ms at p95 under defined test conditions.

For investigations, NeuroDetect shows the score beside the original file, explainability heatmaps and attention views down to regions, frames and audio segments. Each analysis keeps model and version context, an LLM-assisted summary links its statements to the evidence, and the downloadable forensic report records observations and uncertainty, with the analyst making the final call. On VendorBench-100, Neural Defend ranked first of 36 systems by MCC; one author of that benchmark also co-wrote Neural Defend's ARCAS 1B paper, and the benchmark covers still images only.