Scanner deep-dive

DeepSeek V4 Pro by DeepSeek ↗

General-Purpose LLM · agentic-v1 · scored on 66/66 repositories. Strict scoring (unfinished repos counted as misses).

50.3

F3 (strict)

52.3

F2 (strict)

48.3%

Recall (strict)

78.5%

Precision

66/66

Repos scored

deepseek-v4-pro

Model

$11

Total cost

334s

Avg latency

Per-repository breakdown

Each bar shows true positives, false positives, and misses on one repository; bar length is proportional to that repo's labeled vulnerabilities. Ranked by F2.

True positiveFalse positiveMissed (FN)

python-app80 F2 · 81%

dsvw78 F2 · 75%

vfapi75 F2 · 74%

insecure-web72 F2 · 70%

vampi71 F2 · 70%

dsvpwa71 F2 · 69%

vulnerable-api70 F2 · 67%

dvblab67 F2 · 64%

pythonssti67 F2 · 67%

vulnpy64 F2 · 60%

vc-codex-high-seeded-v2-education-lms-django63 F2 · 64%

intentionally-vulnerable-python-application62 F2 · 62%

damn-vulnerable-flask-application61 F2 · 58%

vc-codex-high-seeded-v2-logistics-dispatch-fastapi59 F2 · 55%

vc-kimi-code-seeded-v2-legal-case-django57 F2 · 54%

vc-codex-high-seeded-v2-hr-payroll-django57 F2 · 52%

owasp-web-playground56 F2 · 52%

vc-kimi-code-seeded-v2-marketplace-commerce-fastapi56 F2 · 52%

vc-kimi-code-seeded-v2-fintech-lending-fastapi56 F2 · 52%

vc-kimi-code-seeded-v2-healthcare-clinic-django56 F2 · 53%

vc-kimi-code-seeded-v2-logistics-dispatch-fastapi56 F2 · 52%

vulnerable-flask-app55 F2 · 52%

vc-codex-high-seeded-v2-crm-saas-django55 F2 · 52%

vc-codex-high-seeded-v2-property-management-fastapi55 F2 · 50%

vc-kimi-code-seeded-v2-education-lms-django54 F2 · 50%

vc-kimi-code-seeded-v2-property-management-fastapi54 F2 · 50%

vc-codex-seeded-v2-logistics-dispatch-fastapi54 F2 · 50%

python-insecure-app54 F2 · 50%

vc-claude-code-seeded-v2-legal-case-django54 F2 · 48%

vc-kimi-code-seeded-v2-hr-payroll-django54 F2 · 52%

vc-codex-high-seeded-v2-support-desk-fastapi53 F2 · 50%

vc-codex-high-seeded-v2-legal-case-django53 F2 · 48%

vc-codex-seeded-v2-fintech-lending-fastapi53 F2 · 49%

vc-codex-high-seeded-v2-fintech-lending-fastapi53 F2 · 48%

vc-claude-code-seeded-v2-hr-payroll-django52 F2 · 48%

vc-codex-seeded-v2-property-management-fastapi52 F2 · 48%

vc-claude-code-seeded-v2-fintech-lending-fastapi52 F2 · 48%

vc-claude-code-seeded-v2-support-desk-fastapi51 F2 · 47%

extremely-vulnerable-flask-app51 F2 · 46%

vc-kimi-code-seeded-v2-support-desk-fastapi51 F2 · 46%

vc-claude-code-seeded-v2-crm-saas-django50 F2 · 46%

vc-codex-seeded-v2-legal-case-django50 F2 · 45%

vulnerable-tornado-app50 F2 · 46%

vc-claude-code-seeded-v2-education-lms-django50 F2 · 47%

lets-be-bad-guys50 F2 · 46%

threatbyte49 F2 · 44%

vc-claude-code-seeded-v2-healthcare-clinic-django49 F2 · 45%

pygoat48 F2 · 45%

vc-codex-seeded-v2-crm-saas-django48 F2 · 44%

vc-claude-code-seeded-v2-logistics-dispatch-fastapi48 F2 · 42%

vc-kimi-code-seeded-v2-crm-saas-django48 F2 · 44%

vc-codex-seeded-v2-hr-payroll-django48 F2 · 44%

vc-codex-high-seeded-v2-marketplace-commerce-fastapi47 F2 · 44%

vc-codex-seeded-v2-support-desk-fastapi47 F2 · 43%

vc-codex-high-seeded-v2-healthcare-clinic-django46 F2 · 42%

vc-codex-seeded-v2-marketplace-commerce-fastapi46 F2 · 41%

vulpy45 F2 · 40%

dvpwa44 F2 · 41%

flask-xss44 F2 · 39%

vc-claude-code-seeded-v2-property-management-fastapi44 F2 · 39%

vulnerable-python-apps42 F2 · 38%

vc-claude-code-seeded-v2-marketplace-commerce-fastapi42 F2 · 38%

damn-vulnerable-graphql-application42 F2 · 38%

vc-codex-seeded-v2-healthcare-clinic-django39 F2 · 34%

vc-codex-seeded-v2-education-lms-django35 F2 · 31%

djangoat32 F2 · 29%

Repository	TP	FP	FN	Recall %	F2
python-app	17	5	4	81.0	80.4
dsvw	20	3	7	75.3	77.6
vfapi	7	2	2	74.1	74.6
insecure-web	6	2	3	70.4	71.9
vampi	10	3	4	70.0	71.4
dsvpwa	22	6	10	68.8	70.6
vulnerable-api	9	1	5	66.7	70.0
dvblab	14	2	8	63.6	67.3
pythonssti	1	1	1	66.7	66.7
vulnpy	48	8	32	60.0	63.9
vc-codex-high-seeded-v2-education-lms-django	16	11	9	64.0	63.0
intentionally-vulnerable-python-application	4	2	3	61.9	61.9
damn-vulnerable-flask-application	9	3	6	57.8	60.6
vc-codex-high-seeded-v2-logistics-dispatch-fastapi	16	3	13	55.2	59.3
vc-kimi-code-seeded-v2-legal-case-django	14	4	12	53.8	57.4
vc-codex-high-seeded-v2-hr-payroll-django	13	1	12	52.0	57.0
owasp-web-playground	15	4	13	52.4	56.0
vc-kimi-code-seeded-v2-marketplace-commerce-fastapi	14	3	13	51.8	56.0
vc-kimi-code-seeded-v2-fintech-lending-fastapi	17	3	16	51.5	55.9
vc-kimi-code-seeded-v2-healthcare-clinic-django	16	7	14	53.3	55.9
vc-kimi-code-seeded-v2-logistics-dispatch-fastapi	16	4	15	51.6	55.6
vulnerable-flask-app	11	5	10	52.4	55.1
vc-codex-high-seeded-v2-crm-saas-django	13	5	12	52.0	55.1
vc-codex-high-seeded-v2-property-management-fastapi	13	1	13	50.0	55.1
vc-kimi-code-seeded-v2-education-lms-django	14	3	14	50.0	54.3
vc-kimi-code-seeded-v2-property-management-fastapi	15	3	15	50.0	54.3
vc-codex-seeded-v2-logistics-dispatch-fastapi	15	4	15	50.0	54.0
python-insecure-app	4	1	4	50.0	53.8
vc-claude-code-seeded-v2-legal-case-django	15	1	16	48.4	53.6
vc-kimi-code-seeded-v2-hr-payroll-django	15	9	14	51.7	53.6
vc-codex-high-seeded-v2-support-desk-fastapi	14	5	14	50.0	53.4
vc-codex-high-seeded-v2-legal-case-django	12	1	13	48.0	53.1
vc-codex-seeded-v2-fintech-lending-fastapi	18	4	19	48.6	52.9
vc-codex-high-seeded-v2-fintech-lending-fastapi	14	3	15	48.3	52.6
vc-claude-code-seeded-v2-hr-payroll-django	13	3	14	48.1	52.4
vc-codex-seeded-v2-property-management-fastapi	15	4	16	48.4	52.4
vc-claude-code-seeded-v2-fintech-lending-fastapi	14	4	15	48.3	52.2
vc-claude-code-seeded-v2-support-desk-fastapi	16	5	18	47.1	51.0
extremely-vulnerable-flask-app	15	1	17	45.8	50.9
vc-kimi-code-seeded-v2-support-desk-fastapi	13	3	15	46.4	50.8
vc-claude-code-seeded-v2-crm-saas-django	13	4	15	46.4	50.4
vc-codex-seeded-v2-legal-case-django	15	2	18	45.5	50.3
vulnerable-tornado-app	6	2	8	46.4	50.0
vc-claude-code-seeded-v2-education-lms-django	15	7	17	46.9	50.0
lets-be-bad-guys	11	4	13	45.8	49.7
threatbyte	12	1	14	44.2	49.3
vc-claude-code-seeded-v2-healthcare-clinic-django	13	4	16	44.8	48.9
pygoat	35	12	43	44.9	48.5
vc-codex-seeded-v2-crm-saas-django	15	4	19	44.1	48.4
vc-claude-code-seeded-v2-logistics-dispatch-fastapi	14	0	19	42.4	47.9
vc-kimi-code-seeded-v2-crm-saas-django	12	6	15	44.4	47.6
vc-codex-seeded-v2-hr-payroll-django	17	6	22	43.6	47.5
vc-codex-high-seeded-v2-marketplace-commerce-fastapi	11	5	14	44.0	47.4
vc-codex-seeded-v2-support-desk-fastapi	13	4	17	43.3	47.4
vc-codex-high-seeded-v2-healthcare-clinic-django	11	5	15	42.3	45.8
vc-codex-seeded-v2-marketplace-commerce-fastapi	12	3	17	41.4	45.8
vulpy	23	6	34	40.4	44.7
dvpwa	9	4	14	40.6	44.0
flask-xss	12	1	18	38.9	44.0
vc-claude-code-seeded-v2-property-management-fastapi	13	3	20	39.4	43.9
vulnerable-python-apps	8	2	14	37.9	42.3
vc-claude-code-seeded-v2-marketplace-commerce-fastapi	12	2	20	37.5	42.3
damn-vulnerable-graphql-application	14	4	22	38.0	42.2
vc-codex-seeded-v2-healthcare-clinic-django	14	3	27	34.2	38.7
vc-codex-seeded-v2-education-lms-django	11	5	24	31.4	35.3
djangoat	15	9	37	28.8	32.4

Detection by severity

Severity	TP	FP	FN	Recall %
Critical	140	1	15	90.3
High	391	1	361	52.0
Medium	351	2	559	38.6
Low	40	0	46	46.5

Detection by vulnerability class

CWE family	TP	FP	FN	Recall %
HTTP Header Injection	2	0	0	100.0
XPath Injection	4	0	0	100.0
Open Redirect	39	1	1	97.5
XML External Entities	37	1	1	97.4
SQL Injection	74	0	4	94.9
Command / OS Injection	46	0	4	92.0
Code Injection / RFI	31	0	3	91.2
Insecure Deserialization	36	0	4	90.0
Path Traversal	34	0	10	77.3
Hardcoded Credentials	45	1	23	66.2
Server-Side Request Forgery	27	0	14	65.9
Cross-Site Scripting	65	0	45	59.1
Security Misconfiguration	53	0	59	47.3
Sensitive Data Exposure	73	0	82	47.1
Other	306	1	571	34.9
Missing Authentication / Authorization	24	0	59	28.9
Broken Access Control / IDOR	19	0	64	22.9
Denial of Service	7	0	37	15.9

LLM operational metrics

32,943

Avg input tokens

12,064

Avg output tokens

222,097

Avg total tokens

334s

Avg latency / repo

0.0%

JSON repair rate

118

Total runs

±9.6

F2 run-to-run σ

Cost

$11

Total cost

$0.10

Cost / run

$0.007

Cost / 100 LOC

133,782

Python LOC scanned

111

Successful runs

← Back to the leaderboard