What NIST AI 100-4 Says About Detecting Synthetic Content

What Is NIST AI 100-4?

NIST AI 100-4, titled Reducing Risks Posed by Synthetic Content: An Overview of Technical Approaches to Digital Content Transparency, was published by the National Institute of Standards and Technology on 20 November 2024 in its Trustworthy and Responsible AI series. It followed a December 2023 request for information and an April 2024 public draft, and was written in response to Section 4.5(a) of Executive Order 14110 on AI.

The report covers four areas: provenance data tracking through digital watermarking and metadata recording; synthetic content detection; testing and evaluation of both; and technical measures to prevent and reduce AI-generated child sexual abuse material and non-consensual intimate imagery. Appendices list current standards, tools, datasets and evaluation metrics. It is guidance, not law, and it creates no compliance obligation.

What Does NIST AI 100-4 Say About Detecting Synthetic Content?

Section 3.2 defines synthetic content detection as techniques that classify whether content, or part of it, is synthetic, and sorts them into three non-exclusive categories. Provenance data detection looks for watermarks or metadata that were recorded at creation. Automated content-based detection looks for traces left during generation or processing, such as pixel regularities or inconsistencies. Human-assisted detection puts crowd workers, labellers or domain experts in the loop to validate or supplement automated tools.

NIST is direct about the weaknesses of each. Watermark detection always carries some probability of error and metadata may be inaccurate. Automated detection is described as a constant cat-and-mouse game in which generators improve as soon as a new detector appears, and detectors are often tied to the specific generators they were trained on. Human-assisted detection requires extensive labour, incurs high cost and presumes people can detect synthetic content at all; NIST cites a 2024 study across audio, visual and audiovisual content that found human performance near chance.

The report also flags partial manipulation. An image with one object removed and inpainted by AI is mostly real, and NIST notes that whether such a change is substantial may be subjective. Detectors that only answer real or fake for a whole file struggle with these cases.

How Accurate Does NIST Say Deepfake Detectors Are?

Section 3.2.2 summarises published results by modality rather than endorsing any tool. For images, NIST says detectors trained on one generator do better on that generator than on others, and that in the more realistic cross-generator setting detectors tend to show substantial error rates, especially after compression and resizing. It cites one 2023 paper reporting accuracy of 61% to 70% and a 2022 paper reporting accuracy of 50% to 62% with AUC of 52% to 91%.

For video, one 2023 survey showed accuracy from 62% to 99% and AUC from 82% to 98%, with results depending on manipulation type, whether audio is examined, training data and post-processing. For audio, NIST reports accuracy ranging from 50% to well above 90% as of September 2024, notes that most research focuses on English voices, and warns that unseen generators and real-world noise degrade results. These ranges describe the academic literature as of 2024, not any specific commercial product.

How Does NIST Recommend Testing Detection Tools?

Section 4.2 sets out what a meaningful detector evaluation needs. The test data should match the content the tool will face: subject matter, cultural context such as languages and accents, benign distortions like compression and background noise, adversarial attacks designed to fool detectors, and sample sizes. It should include output from generators the detector never saw in training, because detectors are expected to generalise to new models.

On metrics, NIST notes that accuracy, AUROC and F1 are commonly reported but balance over- and under-detection in a way that may not fit the use case. Because labelling human content as AI-generated can cause major reputational harm or adverse treatment, some studies report the true positive rate at a fixed low false positive rate, written TPR@FPR=x. NIST also asks whether an AUROC of 0.95 is sufficient, and answers that it depends on how outputs feed into decisions. It calls for reproducible evaluations and warns that defences tested against one set of attacks have been broken by others.

Section 4.2.3 adds that where a model informs a human reviewer, the human's final accuracy, time taken and perceived difficulty are the metrics that matter.

What NIST AI 100-4 Means for Teams Buying Detection

Read vendor accuracy claims against NIST's checklist. Ask which generators were in the test set and which were held out, whether test media was compressed, resized or re-recorded, whether any adversarial evasion was attempted, and what the true positive rate is at the false positive rate your workflow can tolerate. A single accuracy figure on a balanced academic dataset answers none of these.

Treat provenance as a complement. NIST concludes that provenance tracking and detection must be applied together, and with social measures, because none can be considered a comprehensive solution. A practical stack checks Content Credentials or watermarks where they exist and runs content-based detection on everything, since fraud media rarely carries provenance.

How Neural Defend Maps to NIST's Recommendations

Neural Defend publishes results in the form NIST's evaluation guidance favours. On the public VendorBench-100 benchmark, a single adversarial corpus of 100 still images covering face swaps, text-to-video stills, AI photo edits and opaque-provenance images, 36 systems were ranked by MCC because the authors warn accuracy and F1 mislead on that imbalanced set. Neural Defend ranked first with MCC 0.876 and ROC-AUC 0.979, against 0.646 for the next vendor. One VendorBench author also co-wrote Neural Defend's ARCAS 1B paper; the corpus and results are public so anyone can rerun them, and the benchmark does not test video, audio, injection attacks or latency.

NIST's human-assisted category is where NeuroDetect fits. Analysts upload authorised media into a case; the analysis records model and version context and shows heatmaps and attention views down to regions, frames and audio segments, with an LLM-assisted summary linked to evidence while the analyst makes the final call. The forensic report records observations, uncertainty and provenance, matching NIST's emphasis on keeping uncertainty visible. For automated screening, NeuroVerify runs six signals in one API call and returns a unified risk score with signal-level evidence in under 200 ms p95 on eligible image requests, so a team can set thresholds against its own false positive budget.