Neural Defend
Start Here

Evaluation foundation Shared method

Test with evidence that resembles the real workflow.

Start with the decision your team needs to make. Then compare Neural Defend with the control you use today, using authorized samples, stated acceptance criteria, and the failures the live workflow must handle.

Build an evaluation planReview the method

01 · Decision first

Choose metrics after you know what can go wrong.

A school checking submitted coursework, a fraud team reviewing a claim, and a meeting host protecting an approval call do not share the same error cost. The evaluation must follow the real decision.

Write the decision in one sentence

“Before [the business action], our workflow uses [the signal] with [other evidence] to choose [continue, recapture, review, or stop].”

If a genuine sample is escalated

What is the cost of interruption?

Consider delay, access, workload, abandonment, fairness, and the route available to correct the decision.

If a risky sample is missed

What is the cost of continuation?

Consider the protected action, downstream controls, recovery options, and who carries responsibility for the decision.

Write acceptance criteria before running the test. Keep those criteria separate from the measured results.

02 · Evaluation set

Include ordinary evidence, attacks, and evidence the system cannot use.

01

Genuine

Authorized, unaltered samples from the intended capture or submission path.

02

Manipulated

Edited, composited, replayed, face-swapped, voice-cloned, or otherwise altered samples relevant to the workflow.

03

AI-generated

Synthetic samples from relevant generation methods, kept separate from other manipulation types in the report.

04

Low-quality

Compression, poor lighting, noise, blur, short duration, weak connectivity, or other realistic degradation.

05

Unsupported

Formats, dimensions, durations, file conditions, or media types outside documented support.

06

Inconclusive

Samples where available evidence does not support a confident workflow decision.

Representative where relevant and lawful

Cover the geographies, demographics, devices, capture channels, codecs, network conditions, languages, and evidence types that can materially affect the intended workflow. Do not collect protected or personal data merely to fill a category. Record consent, authority, provenance, labels, and exclusions.

03 · Baseline comparison

Test the proposed control and today's control on the same work.

A product result without a baseline cannot show what changed. Use the same samples, labels, operating conditions, and review rules wherever the controls permit a fair comparison.

Comparison fieldCurrent controlNeural Defend evaluation
InputWhat it receives todayThe same authorized set
Decision pointWhere it actsThe proposed checkpoint
OutputResult, alert, or no checkSignal, status, or failure
PolicyCurrent routing ruleProposed threshold and routing
Human workExisting review effortReview and recapture effort
Failure pathCurrent fallbackTested fallback and audit record

04 · Measures in plain language

Every metric needs a practical consequence.

False positive

A genuine sample is flagged or routed as if it were suspicious.

More legitimate people, transactions, or evidence may be delayed, recaptured, or reviewed.

False negative

A manipulated or AI-generated sample is not flagged at the chosen threshold.

Risky evidence may continue to the next control without the intended escalation.

Precision

Of the samples flagged at a threshold, the share that belongs to the positive test class.

Lower precision can increase avoidable review work. The result depends on the test-set mix.

Recall

Of the positive test samples, the share flagged at a threshold.

Higher recall can catch more tested cases, but may also route more genuine samples for review.

Operating threshold

The point at which a score or signal triggers a workflow action.

Changing it changes the trade-off between missed cases and unnecessary escalation.

Latency / throughput

How long analysis takes and how much work completes over a defined period under stated conditions.

These measures show whether the tested integration can fit the intended workflow and load.

Completion / failure rate

The share of submitted requests that produce a usable result versus reject, time out, or error.

Unscored requests need a fallback path; they must not be counted as correct detections.

Report counts as well as rates, define the positive class, show the threshold, and break results down only where the sample size and lawful data support interpretation. A target written before testing is an acceptance criterion. A value observed during testing is a measured result. Do not present one as the other.

05 · Evaluation worksheet

Build the checklist your teams will use.

Choose the scope and enter the decision. The worksheet creates a planning checklist only; it does not predict performance or issue a pass result.

Planning inputs
0 / 240 characters
Product scope

Select every product involved in this evaluation.

Sample coverage

Keep a group selected only if you can label and use it lawfully.

Evaluation priorities

Use these to focus the working session and final report.

Generated planning checklist

Evaluation scope

Decision
Not yet defined. Write the protected business decision before testing.
Products
NeuroVerify (Live)
Sample groups
Genuine · Manipulated · AI-generated · Low-quality · Unsupported · Inconclusive
Priorities
Detection trade-offs · Operational fit
Before testing
  • Write acceptance criteria and failure costs.
  • Record sample permissions, provenance, labels, and exclusions.
  • Freeze the current-control comparison method.
  • Define the positive class, threshold candidates, and reviewer.
During testing
  • Keep counts, rates, conditions, product version, and dates.
  • Separate scored results from rejected, failed, and timed-out work.
  • Test quality loss, contradictory evidence, and fallback behavior.
  • Record reviewer actions without changing criteria after seeing results.
Before deciding
  • Compare both controls under equivalent conditions.
  • Report limitations beside each affected finding.
  • Choose accept, revise and retest, or do not adopt.
  • Name the owners of policy, integration, review, and approval.

This checklist contains no measured Neural Defend result and makes no claim about accuracy, latency, service availability, regulatory compliance, or customer outcome.

06 · Threshold and escalation

The product supplies a signal. Customer policy owns the action.

Evaluate the full route from product response to reviewer outcome. Do not test the detector in isolation and assume the operating workflow will behave the same way.

01 · Product

Signal or failure state

Record what the service returned, including unresolved input.

02 · Customer

Threshold and policy

Combine the response with the workflow's other evidence.

03 · Reviewer

Escalation and decision

Give an authorized person enough context to act and record why.

Low risk is not proof of authenticity.

It means the tested response was below a defined risk or review boundary under the conditions observed. Keep downstream controls and a route for contradictory evidence.

07 · Operational tests

Test the moments when no usable answer arrives.

01

Timeout

Test question

When does the request stop waiting, and what happens next?

Evidence to retain

Timeout record, retry behavior, idempotency handling, fallback path, and unresolved decision state.

02

Service unavailable

Test question

Can the workflow fail safely without silently approving?

Evidence to retain

Error handling, bounded retry or queue behavior, operator visibility, and recovery record.

03

Malformed or unsupported media

Test question

Can the workflow distinguish rejected input from a risk result?

Evidence to retain

Returned status and message, recapture route, and proof that no authenticity conclusion was assigned.

04

Quality degradation

Test question

Where does realistic quality loss make the result unusable?

Evidence to retain

Conditions tested, completion behavior, result changes, and the point at which recapture is required.

05

Contradictory evidence

Test question

What happens when product signals and other controls disagree?

Evidence to retain

Escalation rule, evidence shown to the reviewer, and the person or team authorized to decide.

06

Audit and human review

Test question

Can a reviewer reconstruct the request and the policy action?

Evidence to retain

Transaction reference, result or failure state, relevant context, reviewer action, and retained report.

08 · Evaluation report

Make the adoption decision traceable.

The report should let security, ML, operations, legal, privacy, accessibility, and procurement readers see what was tested, what was not, and which conditions remain open.

Recommended report structure
  1. 01

    Decision and workflow boundary

  2. 02

    Products, versions, integration path, and test dates

  3. 03

    Sample sources, labels, permissions, and exclusions

  4. 04

    Class balance and relevant operating conditions

  5. 05

    Current-control baseline and comparison method

  6. 06

    Results by sample group and operating threshold

  7. 07

    Latency, throughput, completion, rejection, and error behavior

  8. 08

    Threshold, escalation, fallback, and human-review design

  9. 09

    Observed limitations, unresolved cases, and retest requirements

  10. 10

    Acceptance decision with named owners and conditions

Accept

Proceed under stated conditions

The measured results meet the pre-written criteria, and owners accept the remaining limitations and operating controls.

Revise and retest

Change the plan or workflow

Adjust the set, threshold, integration, review route, or failure handling, then repeat the affected tests.

Do not adopt

Keep the current control

The tested approach does not meet the criteria, or its limitations and operational cost are not acceptable.

Limitations belong beside the decision

State excluded populations, unsupported inputs, weak labels, small subgroups, untested attacks, environment differences, version changes, and any result that could not be resolved.

09 · Product-status eligibility

Match the evaluation request to the product's current status.

NeuroVerify

Live
Can enter a scoped technical evaluation.

Define the authorized media, API path, response handling, and decision policy.

NeuroDetect

Live
Can enter a scoped technical evaluation.

Define the investigation set, analyst workflow, evidence record, and reporting needs.

NeuroCall

Beta
Requires a scoped Beta discussion.

Map the call stream, intended signal route, operator action, integration requirements, and Beta feedback process.

NeuroMeeting

Coming Soon
Starts with a workflow and product discussion.

Agree the intended meeting environment, participant permissions, test boundary, and future integration requirements.

10 · Scope the evaluation

Bring the workflow. Leave with a test plan.

You bring
  • The business decision and current control
  • Authorized, representative sample evidence
  • Expected formats, conditions, and workload
  • Acceptance criteria and failure costs
  • Security, privacy, review, and procurement questions
Neural Defend returns
  • A product-status and workflow-fit discussion
  • A scoped evaluation method and integration boundary
  • Expected response and failure states to test
  • A threshold, escalation, and reporting outline
  • Known scope limits and open questions for your report
Request an evaluation discussion

A discussion does not guarantee product fit, a measured result, service terms, or an adoption decision.