Audit Evidence for Bank Fraud Workflows: Designing Records for Deepfake Detection Decisions

What Audit Evidence Do Bank Fraud Workflows Need?

Enough to answer three questions about any decision: what did the institution know, what did it do with that knowledge, and was that consistent with its policy at the time. For workflows that use deepfake detection, that means keeping the input media as received, the derived files used for analysis, the score and signal evidence, the model identity and version, the threshold or rule that turned the score into an action, and every human step that followed.

Records should be append-only. Reanalysis with a newer model creates a new result linked to the old one rather than overwriting it, and reviewer notes are added, not edited. Without that, a later audit sees today's model output and today's policy, and cannot tell whether the original decision was reasonable.

What Evidence Supports Audit and Model-Governance Reviews?

Decision records show individual cases; governance reviews need evidence about the detector as a system. That includes the vendor's documentation of what the model was tested on and where it is weak, the institution's own validation on representative data, thresholds and the reasoning behind them, monitoring of score distributions and override rates over time, and records of each model or threshold change with its approval.

Two public references help structure this. NIST's AI Risk Management Framework 1.0, released 26 January 2023, groups AI risk work into Govern, Map, Measure and Manage, and its Generative AI Profile (NIST AI 600-1) followed on 26 July 2024. Article 12 of the EU AI Act requires high-risk AI systems to technically allow automatic recording of events over their lifetime, to support identifying risks, post-market monitoring and deployer oversight. Even where neither applies directly, they describe what a reviewer will expect to see.

Ask detection vendors for the material that belongs in that file before you sign: the test data and benchmarks behind published accuracy figures, known weak spots by media type and attack, how model versions are released and announced, whether older versions can be pinned for reproducibility, and how scores are calibrated. A vendor that cannot answer leaves the institution to build that evidence alone, usually during an audit rather than before one.

How Long to Keep Fraud Workflow Evidence

Set retention from the strictest rule that applies to the workflow, then limit access. Some anchors from US rules: the broker-dealer Customer Identification Program rule, 31 CFR 1023.220, requires identifying information to be kept for five years after an account closes and other verification records for five years after they are made. FinCEN's SAR FAQ says filers are generally required to keep copies of their filings for five years, and that the BSA E-Filing System is not a recordkeeping program.

Biometric media raises the stakes on access and deletion. Store selfies, video and voice recordings separately from case notes, restrict who can open them, log every access, and delete on schedule unless a legal hold applies. Where data residency rules apply, such as RBI's requirement that V-CIP recordings stay on systems in India, the detection deployment has to respect them too.

Quality Assurance That Tests the Hard Cases

Random samples of approvals mostly confirm that easy cases were easy. Useful QA oversamples the states where errors hide: overrides of a detector flag, inconclusive results, low-quality media, cases closed just under a threshold, and decisions later linked to confirmed fraud or upheld complaints.

Replay each sampled case with only the evidence available at decision time, assign a root cause when the decision was wrong (data, model, threshold, procedure or judgement), and retest after remediation. Track how often reviewers agree with each other on the same case; low agreement points to unclear guidance rather than careless staff.

How Neural Defend Supports Audit Evidence

NeuroDetect is built around the case record. Each analysis of an authorised image, audio file, video or document keeps model and version context; detailed views (the original, explainability heatmaps and attention views) go down to regions, frames and audio segments; and the analyst's conclusion sits beside an LLM-assisted summary linked to the evidence. Cases export as a downloadable forensic report with observations, uncertainty, provenance, model context and analyst notes, plus structured results for the institution's own archive.

NeuroVerify returns a unified risk score with signal-level evidence from six signals for each request, which gives decision records more than a single number. For model-governance files, the ARCAS 1B paper on arXiv documents benchmark results such as 99.99% pooled accuracy on GenImage and 97.65% on OpenFake, and Neural Defend ranked first of 36 systems on the public VendorBench-100 still-image benchmark (MCC 0.876 against 0.646). One VendorBench author co-wrote the ARCAS paper, the benchmark covers still images only, and its corpus and results are public, so validation teams can rerun it. Neural Defend holds ISO/IEC 27001, ISO/IEC 27701, ISO/IEC 42001 and SOC 2 Type II.

What Good Records Cannot Fix

A complete record shows what happened; it does not make a weak control strong. If thresholds were never validated on the institution's own customers, or if reviewers routinely override flags without reasons, the audit trail will show it clearly. Records also cannot recover evidence that was never captured, such as the original file before a customer's app recompressed it.

Design the record before the workflow goes live, test that an auditor can rebuild a sample of decisions from it alone, and revisit it whenever the model, threshold or policy changes.