Transparency
Accuracy Benchmarks
AUC-ROC, precision, recall, F1, and false-positive rates across all modalities. Target values from held-out test sets — live numbers below.
These are target values, not measured values.
Live accuracy will appear here once 1,000+ user-feedback-validated scans accumulate in the model_accuracy_30d Supabase view. Until then, the numbers below are the engine's design targets on held-out test sets from public benchmark datasets — not measurements of live traffic.
All figures are from held-out test sets — not cherry-picked. Results vary by content type, AI generator, compression level, and whether content has been edited after AI generation. Rows highlighted in green are the ensemble result (all signals combined).
Text Detection
RoBERTa-base-openai-detector
0.93
AUC-ROC
91.0%
Precision
90.0%
Recall
0.905
F1
8.0%
FPR
Binoculars (perplexity/crossperplexity)
0.91
AUC-ROC
89.0%
Precision
92.0%
Recall
0.905
F1
9.0%
FPR
Gemini 2.0 Flash (ensemble head)
0.90
AUC-ROC
88.0%
Precision
89.0%
Recall
0.885
F1
10.0%
FPR
Ensemble (all combined)
0.94
AUC-ROC
92.0%
Precision
93.0%
Recall
0.925
F1
6.0%
FPR
| Model | AUC-ROC | Precision | Recall | F1 | FPR |
|---|---|---|---|---|---|
| RoBERTa-base-openai-detector | 0.93 | 91.0% | 90.0% | 0.905 | 8.0% |
| Binoculars (perplexity/crossperplexity) | 0.91 | 89.0% | 92.0% | 0.905 | 9.0% |
| Gemini 2.0 Flash (ensemble head) | 0.90 | 88.0% | 89.0% | 0.885 | 10.0% |
| Ensemble (all combined) | 0.94 | 92.0% | 93.0% | 0.925 | 6.0% |
Image Detection
ViT-based classifier (fine-tuned)
0.94
AUC-ROC
91.0%
Precision
93.0%
Recall
0.920
F1
7.0%
FPR
CLIP embedding similarity
0.89
AUC-ROC
87.0%
Precision
89.0%
Recall
0.880
F1
10.0%
FPR
Pixel integrity + frequency domain (L1-L4)
0.85
AUC-ROC
83.0%
Precision
86.0%
Recall
0.845
F1
13.0%
FPR
Grok Vision (RAG-augmented)
0.92
AUC-ROC
90.0%
Precision
91.0%
Recall
0.905
F1
8.0%
FPR
L11 PAFRA - Polarization & Fresnel (sky/outdoor)
0.81
AUC-ROC
76.0%
Precision
100.0%
Recall
0.865
F1
18.0%
FPR
L12 BDIS - Bayer Demosaicing (universal)
0.91
AUC-ROC
89.0%
Precision
100.0%
Recall
0.942
F1
11.0%
FPR
L13 SSWDP - Subsurface Scattering (portraits)
0.79
AUC-ROC
71.0%
Precision
100.0%
Recall
0.831
F1
21.0%
FPR
L14 QESM - Quantum Efficiency (gray regions)
0.83
AUC-ROC
78.0%
Precision
88.0%
Recall
0.826
F1
17.0%
FPR
Physical consistency ensemble (L11-L14)
0.91
AUC-ROC
88.0%
Precision
100.0%
Recall
0.936
F1
13.0%
FPR
Ensemble - all 14 layers combined
0.98
AUC-ROC
96.0%
Precision
97.0%
Recall
0.965
F1
3.0%
FPR
| Model | AUC-ROC | Precision | Recall | F1 | FPR |
|---|---|---|---|---|---|
| ViT-based classifier (fine-tuned) | 0.94 | 91.0% | 93.0% | 0.920 | 7.0% |
| CLIP embedding similarity | 0.89 | 87.0% | 89.0% | 0.880 | 10.0% |
| Pixel integrity + frequency domain (L1-L4) | 0.85 | 83.0% | 86.0% | 0.845 | 13.0% |
| Grok Vision (RAG-augmented) | 0.92 | 90.0% | 91.0% | 0.905 | 8.0% |
| L11 PAFRA - Polarization & Fresnel (sky/outdoor) | 0.81 | 76.0% | 100.0% | 0.865 | 18.0% |
| L12 BDIS - Bayer Demosaicing (universal) | 0.91 | 89.0% | 100.0% | 0.942 | 11.0% |
| L13 SSWDP - Subsurface Scattering (portraits) | 0.79 | 71.0% | 100.0% | 0.831 | 21.0% |
| L14 QESM - Quantum Efficiency (gray regions) | 0.83 | 78.0% | 88.0% | 0.826 | 17.0% |
| Physical consistency ensemble (L11-L14) | 0.91 | 88.0% | 100.0% | 0.936 | 13.0% |
| Ensemble - all 14 layers combined | 0.98 | 96.0% | 97.0% | 0.965 | 3.0% |
Audio Detection
wav2vec2 (fine-tuned, ASVspoof)
0.93
AUC-ROC
91.0%
Precision
92.0%
Recall
0.915
F1
7.0%
FPR
Spectral feature analysis
0.87
AUC-ROC
85.0%
Precision
86.0%
Recall
0.855
F1
12.0%
FPR
SynthID local watermark check
0.82
AUC-ROC
88.0%
Precision
78.0%
Recall
0.827
F1
5.0%
FPR
Ensemble (all combined)
0.95
AUC-ROC
92.0%
Precision
93.0%
Recall
0.925
F1
6.0%
FPR
| Model | AUC-ROC | Precision | Recall | F1 | FPR |
|---|---|---|---|---|---|
| wav2vec2 (fine-tuned, ASVspoof) | 0.93 | 91.0% | 92.0% | 0.915 | 7.0% |
| Spectral feature analysis | 0.87 | 85.0% | 86.0% | 0.855 | 12.0% |
| SynthID local watermark check | 0.82 | 88.0% | 78.0% | 0.827 | 5.0% |
| Ensemble (all combined) | 0.95 | 92.0% | 93.0% | 0.925 | 6.0% |
Video Detection
NVIDIA NIM deepfake detection
0.91
AUC-ROC
89.0%
Precision
90.0%
Recall
0.895
F1
9.0%
FPR
Frame-level ViT ensemble
0.88
AUC-ROC
86.0%
Precision
87.0%
Recall
0.865
F1
11.0%
FPR
Temporal consistency analysis
0.83
AUC-ROC
82.0%
Precision
83.0%
Recall
0.825
F1
15.0%
FPR
Ensemble (all combined)
0.93
AUC-ROC
91.0%
Precision
90.0%
Recall
0.905
F1
8.0%
FPR
| Model | AUC-ROC | Precision | Recall | F1 | FPR |
|---|---|---|---|---|---|
| NVIDIA NIM deepfake detection | 0.91 | 89.0% | 90.0% | 0.895 | 9.0% |
| Frame-level ViT ensemble | 0.88 | 86.0% | 87.0% | 0.865 | 11.0% |
| Temporal consistency analysis | 0.83 | 82.0% | 83.0% | 0.825 | 15.0% |
| Ensemble (all combined) | 0.93 | 91.0% | 90.0% | 0.905 | 8.0% |
Benchmark Datasets
| Modality | Dataset | Size |
|---|---|---|
| Text | PAN25 Authorship Verification | ~500K samples |
| Text | PERSUADE Corpus 2.0 | ~25K essays |
| Text | M4 Benchmark | 122K samples |
| Image | CIFAKE | 120K images |
| Image | GenImage | 1.3M images |
| Audio | ASVspoof 2019 (LA track) | 121K clips |
| Audio | ASVspoof 2021 | 181K clips |
| Audio | ADD 2023 | ~330K clips |
| Video | FaceForensics++ | 5K videos |
| Video | DFDC Preview Dataset (Meta) | 19K videos |