Scanner deep-dive

Gemini 3.5 Flash by Google DeepMind ↗

General-Purpose LLM · agentic-v1 · scored on 64/66 repositories. Strict scoring (unfinished repos counted as misses).

34.5

F3 (strict)

37.0

F2 (strict)

32.3%

Recall (strict)

89.8%

Precision

64/66

Repos scored

gemini-3.5-flash

Model

$81

Total cost

175s

Avg latency

Per-repository breakdown

Each bar shows true positives, false positives, and misses on one repository; bar length is proportional to that repo's labeled vulnerabilities. Ranked by F2.

True positiveFalse positiveMissed (FN)

intentionally-vulnerable-python-application76 F2 · 71%

vfapi71 F2 · 67%

dvblab67 F2 · 62%

insecure-web60 F2 · 56%

owasp-web-playground53 F2 · 48%

pythonssti52 F2 · 50%

vulnerable-api50 F2 · 45%

vc-codex-high-seeded-v2-marketplace-commerce-fastapi50 F2 · 44%

dsvpwa49 F2 · 44%

vc-kimi-code-seeded-v2-logistics-dispatch-fastapi49 F2 · 45%

vampi49 F2 · 44%

vc-codex-high-seeded-v2-property-management-fastapi48 F2 · 44%

python-app48 F2 · 44%

vulnerable-flask-app48 F2 · 43%

vc-codex-seeded-v2-logistics-dispatch-fastapi47 F2 · 42%

threatbyte46 F2 · 42%

vc-codex-high-seeded-v2-crm-saas-django45 F2 · 40%

vc-codex-high-seeded-v2-fintech-lending-fastapi45 F2 · 40%

vc-kimi-code-seeded-v2-fintech-lending-fastapi45 F2 · 39%

vulnerable-python-apps44 F2 · 39%

vc-codex-seeded-v2-fintech-lending-fastapi43 F2 · 38%

vc-kimi-code-seeded-v2-support-desk-fastapi43 F2 · 38%

vc-codex-high-seeded-v2-logistics-dispatch-fastapi43 F2 · 38%

lets-be-bad-guys42 F2 · 38%

vc-kimi-code-seeded-v2-legal-case-django42 F2 · 37%

vc-codex-high-seeded-v2-support-desk-fastapi42 F2 · 37%

vc-codex-high-seeded-v2-healthcare-clinic-django42 F2 · 37%

vc-codex-seeded-v2-marketplace-commerce-fastapi41 F2 · 36%

vc-claude-code-seeded-v2-legal-case-django41 F2 · 35%

vc-claude-code-seeded-v2-crm-saas-django41 F2 · 36%

vc-claude-code-seeded-v2-hr-payroll-django40 F2 · 35%

damn-vulnerable-graphql-application40 F2 · 39%

vc-claude-code-seeded-v2-logistics-dispatch-fastapi40 F2 · 35%

extremely-vulnerable-flask-app40 F2 · 34%

vc-codex-seeded-v2-support-desk-fastapi40 F2 · 34%

vc-kimi-code-seeded-v2-hr-payroll-django39 F2 · 34%

vc-claude-code-seeded-v2-support-desk-fastapi39 F2 · 34%

vc-kimi-code-seeded-v2-crm-saas-django39 F2 · 35%

vc-codex-seeded-v2-legal-case-django38 F2 · 33%

vulnpy38 F2 · 33%

vc-kimi-code-seeded-v2-property-management-fastapi38 F2 · 33%

vc-kimi-code-seeded-v2-education-lms-django38 F2 · 33%

vc-codex-seeded-v2-hr-payroll-django37 F2 · 32%

vc-codex-high-seeded-v2-legal-case-django37 F2 · 32%

vc-claude-code-seeded-v2-healthcare-clinic-django36 F2 · 31%

vc-claude-code-seeded-v2-property-management-fastapi35 F2 · 30%

vulnerable-tornado-app34 F2 · 31%

vc-codex-high-seeded-v2-education-lms-django34 F2 · 29%

vc-claude-code-seeded-v2-marketplace-commerce-fastapi33 F2 · 28%

vc-claude-code-seeded-v2-education-lms-django32 F2 · 27%

vc-codex-high-seeded-v2-hr-payroll-django31 F2 · 28%

vc-codex-seeded-v2-crm-saas-django31 F2 · 26%

pygoat29 F2 · 25%

dvpwa29 F2 · 25%

python-insecure-app29 F2 · 25%

vc-codex-seeded-v2-education-lms-django28 F2 · 24%

vc-codex-seeded-v2-healthcare-clinic-django26 F2 · 22%

vc-kimi-code-seeded-v2-marketplace-commerce-fastapi24 F2 · 21%

flask-xss22 F2 · 19%

vc-codex-seeded-v2-property-management-fastapi22 F2 · 19%

vc-claude-code-seeded-v2-fintech-lending-fastapi21 F2 · 19%

djangoat21 F2 · 17%

vulpy19 F2 · 16%

damn-vulnerable-flask-application13 F2 · 11%

Repository	TP	FP	FN	Recall %	F2
intentionally-vulnerable-python-application	5	0	2	71.4	75.8
vfapi	6	0	3	66.7	71.3
dvblab	14	1	8	62.1	66.7
insecure-web	5	1	4	55.6	59.5
owasp-web-playground	14	2	14	48.2	53.0
pythonssti	1	1	1	50.0	51.9
vulnerable-api	6	0	8	45.2	50.5
vc-codex-high-seeded-v2-marketplace-commerce-fastapi	11	0	14	44.0	49.5
dsvpwa	14	1	18	43.8	49.0
vc-kimi-code-seeded-v2-logistics-dispatch-fastapi	14	5	17	45.2	48.9
vampi	7	0	8	44.4	48.8
vc-codex-high-seeded-v2-property-management-fastapi	11	1	15	43.6	48.5
python-app	9	3	12	44.4	48.1
vulnerable-flask-app	9	1	12	42.9	47.9
vc-codex-seeded-v2-logistics-dispatch-fastapi	12	0	18	41.7	47.2
threatbyte	11	4	15	42.3	46.4
vc-codex-high-seeded-v2-crm-saas-django	10	0	15	40.0	45.2
vc-codex-high-seeded-v2-fintech-lending-fastapi	12	0	18	39.7	45.1
vc-kimi-code-seeded-v2-fintech-lending-fastapi	13	0	20	39.4	44.8
vulnerable-python-apps	8	0	14	38.6	43.8
vc-codex-seeded-v2-fintech-lending-fastapi	14	1	23	37.8	42.9
vc-kimi-code-seeded-v2-support-desk-fastapi	10	0	18	37.5	42.9
vc-codex-high-seeded-v2-logistics-dispatch-fastapi	11	2	18	37.9	42.8
lets-be-bad-guys	9	1	15	37.5	42.5
vc-kimi-code-seeded-v2-legal-case-django	10	0	16	37.2	42.4
vc-codex-high-seeded-v2-support-desk-fastapi	10	1	18	36.9	41.9
vc-codex-high-seeded-v2-healthcare-clinic-django	10	0	16	36.5	41.5
vc-codex-seeded-v2-marketplace-commerce-fastapi	10	1	18	36.2	41.2
vc-claude-code-seeded-v2-legal-case-django	11	0	20	35.5	40.7
vc-claude-code-seeded-v2-crm-saas-django	10	1	18	35.7	40.6
vc-claude-code-seeded-v2-hr-payroll-django	10	0	18	35.2	40.5
damn-vulnerable-graphql-application	14	13	22	38.9	40.2
vc-claude-code-seeded-v2-logistics-dispatch-fastapi	12	0	22	34.8	40.0
extremely-vulnerable-flask-app	11	0	21	34.4	39.6
vc-codex-seeded-v2-support-desk-fastapi	10	0	20	34.4	39.6
vc-kimi-code-seeded-v2-hr-payroll-django	10	1	19	34.5	39.4
vc-claude-code-seeded-v2-support-desk-fastapi	12	1	22	34.3	39.2
vc-kimi-code-seeded-v2-crm-saas-django	9	2	18	34.6	38.9
vc-codex-seeded-v2-legal-case-django	11	0	22	33.3	38.5
vulnpy	27	1	53	33.3	38.0
vc-kimi-code-seeded-v2-property-management-fastapi	10	3	20	33.3	37.6
vc-kimi-code-seeded-v2-education-lms-django	9	3	19	33.3	37.5
vc-codex-seeded-v2-hr-payroll-django	12	0	26	32.0	37.1
vc-codex-high-seeded-v2-legal-case-django	8	0	17	32.0	37.0
vc-claude-code-seeded-v2-healthcare-clinic-django	9	0	20	31.0	35.9
vc-claude-code-seeded-v2-property-management-fastapi	10	1	23	30.3	35.0
vulnerable-tornado-app	4	0	10	30.9	34.4
vc-codex-high-seeded-v2-education-lms-django	7	1	18	29.3	33.6
vc-claude-code-seeded-v2-marketplace-commerce-fastapi	9	0	23	28.1	32.8
vc-claude-code-seeded-v2-education-lms-django	9	0	23	27.1	31.6
vc-codex-high-seeded-v2-hr-payroll-django	7	1	18	28.0	31.4
vc-codex-seeded-v2-crm-saas-django	9	2	25	26.5	30.7
pygoat	20	4	58	25.0	29.1
dvpwa	6	1	17	24.6	28.7
python-insecure-app	2	1	6	25.0	28.6
vc-codex-seeded-v2-education-lms-django	8	0	26	24.3	28.5
vc-codex-seeded-v2-healthcare-clinic-django	9	0	32	21.9	25.9
vc-kimi-code-seeded-v2-marketplace-commerce-fastapi	6	1	21	21.0	24.1
flask-xss	6	2	24	18.9	21.9
vc-codex-seeded-v2-property-management-fastapi	6	0	25	19.4	21.9
vc-claude-code-seeded-v2-fintech-lending-fastapi	6	2	24	19.0	21.3
djangoat	9	1	43	17.3	20.6
vulpy	9	1	48	15.8	18.9
damn-vulnerable-flask-application	2	1	13	11.1	12.6

Detection by severity

Severity	TP	FP	FN	Recall %
Critical	125	0	24	83.9
High	284	2	448	38.8
Medium	194	1	687	22.0
Low	6	0	78	7.1

Detection by vulnerability class

CWE family	TP	FP	FN	Recall %
HTTP Header Injection	1	0	0	100.0
Insecure Deserialization	35	0	4	89.7
SQL Injection	65	0	9	87.8
Code Injection / RFI	28	0	4	87.5
Open Redirect	33	0	5	86.8
XML External Entities	31	1	6	83.8
Command / OS Injection	41	0	8	83.7
Path Traversal	28	1	15	65.1
Server-Side Request Forgery	24	1	16	60.0
Cross-Site Scripting	40	0	65	38.1
XPath Injection	1	0	2	33.3
Security Misconfiguration	34	0	73	31.8
Hardcoded Credentials	17	0	50	25.4
Other	190	0	664	22.2
Missing Authentication / Authorization	17	0	63	21.2
Broken Access Control / IDOR	16	0	67	19.3
Sensitive Data Exposure	8	0	143	5.3
Denial of Service	0	0	43	0.0

LLM operational metrics

133,767

Avg input tokens

4,170

Avg output tokens

840,691

Avg total tokens

175s

Avg latency / repo

2.7%

JSON repair rate

149

Total runs

±11.6

F2 run-to-run σ

Cost

$81

Total cost

$0.55

Cost / run

$0.027

Cost / 100 LOC

131,215

Python LOC scanned

147

Successful runs

← Back to the leaderboard