Scanner deep-dive

Kimi K2.5 by Moonshot AI ↗

General-Purpose LLM · agentic-v1 · scored on 66/66 repositories. Strict scoring (unfinished repos counted as misses).

46.1

F3 (strict)

48.1

F2 (strict)

44.3%

Recall (strict)

72.4%

Precision

66/66

Repos scored

kimi-k2.5

Model

Total cost

127s

Avg latency

Per-repository breakdown

Each bar shows true positives, false positives, and misses on one repository; bar length is proportional to that repo's labeled vulnerabilities. Ranked by F2.

True positiveFalse positiveMissed (FN)

vfapi84 F2 · 89%

vulnpy74 F2 · 71%

intentionally-vulnerable-python-application70 F2 · 67%

dsvw69 F2 · 65%

insecure-web68 F2 · 67%

lets-be-bad-guys67 F2 · 65%

python-app63 F2 · 62%

vampi62 F2 · 62%

dsvpwa58 F2 · 54%

dvblab57 F2 · 55%

vulnerable-api56 F2 · 52%

vc-codex-high-seeded-v2-education-lms-django55 F2 · 52%

damn-vulnerable-flask-application55 F2 · 56%

vc-codex-high-seeded-v2-property-management-fastapi55 F2 · 51%

vc-claude-code-seeded-v2-fintech-lending-fastapi54 F2 · 51%

vc-codex-seeded-v2-logistics-dispatch-fastapi53 F2 · 49%

vc-kimi-code-seeded-v2-marketplace-commerce-fastapi53 F2 · 48%

vulnerable-flask-app53 F2 · 51%

vc-codex-high-seeded-v2-crm-saas-django53 F2 · 49%

vc-codex-high-seeded-v2-marketplace-commerce-fastapi52 F2 · 49%

pythonssti52 F2 · 50%

vc-claude-code-seeded-v2-crm-saas-django52 F2 · 48%

vc-codex-high-seeded-v2-fintech-lending-fastapi52 F2 · 47%

vc-codex-high-seeded-v2-logistics-dispatch-fastapi51 F2 · 47%

vc-kimi-code-seeded-v2-property-management-fastapi51 F2 · 47%

vc-kimi-code-seeded-v2-crm-saas-django51 F2 · 48%

vulnerable-python-apps51 F2 · 48%

dvpwa50 F2 · 49%

vc-kimi-code-seeded-v2-legal-case-django50 F2 · 46%

vc-codex-high-seeded-v2-support-desk-fastapi50 F2 · 45%

python-insecure-app49 F2 · 46%

vulnerable-tornado-app49 F2 · 45%

threatbyte49 F2 · 49%

vc-kimi-code-seeded-v2-logistics-dispatch-fastapi49 F2 · 45%

vc-kimi-code-seeded-v2-hr-payroll-django49 F2 · 45%

vc-kimi-code-seeded-v2-fintech-lending-fastapi49 F2 · 45%

vc-kimi-code-seeded-v2-support-desk-fastapi48 F2 · 45%

vc-claude-code-seeded-v2-hr-payroll-django48 F2 · 43%

vc-codex-seeded-v2-marketplace-commerce-fastapi47 F2 · 43%

vc-codex-seeded-v2-crm-saas-django46 F2 · 42%

owasp-web-playground46 F2 · 43%

pygoat45 F2 · 44%

vc-codex-high-seeded-v2-hr-payroll-django45 F2 · 41%

vc-claude-code-seeded-v2-support-desk-fastapi44 F2 · 39%

vc-codex-high-seeded-v2-legal-case-django44 F2 · 40%

vc-kimi-code-seeded-v2-education-lms-django44 F2 · 40%

extremely-vulnerable-flask-app44 F2 · 40%

vc-claude-code-seeded-v2-education-lms-django44 F2 · 40%

vc-codex-seeded-v2-legal-case-django44 F2 · 39%

vc-codex-seeded-v2-support-desk-fastapi44 F2 · 39%

vc-codex-seeded-v2-hr-payroll-django43 F2 · 38%

vc-codex-high-seeded-v2-healthcare-clinic-django42 F2 · 38%

vc-claude-code-seeded-v2-logistics-dispatch-fastapi42 F2 · 38%

vc-claude-code-seeded-v2-healthcare-clinic-django41 F2 · 38%

vc-codex-seeded-v2-property-management-fastapi40 F2 · 35%

vc-codex-seeded-v2-fintech-lending-fastapi40 F2 · 36%

vc-kimi-code-seeded-v2-healthcare-clinic-django39 F2 · 36%

damn-vulnerable-graphql-application39 F2 · 35%

vc-claude-code-seeded-v2-legal-case-django38 F2 · 34%

flask-xss37 F2 · 34%

vc-codex-seeded-v2-education-lms-django36 F2 · 31%

vc-claude-code-seeded-v2-property-management-fastapi35 F2 · 30%

vc-claude-code-seeded-v2-marketplace-commerce-fastapi33 F2 · 30%

vc-codex-seeded-v2-healthcare-clinic-django33 F2 · 29%

djangoat31 F2 · 28%

vulpy30 F2 · 26%

Repository	TP	FP	FN	Recall %	F2
vfapi	8	4	1	88.9	83.9
vulnpy	57	3	23	71.2	74.5
intentionally-vulnerable-python-application	5	1	2	66.7	70.1
dsvw	18	2	9	65.4	68.9
insecure-web	6	2	3	66.7	68.5
lets-be-bad-guys	16	4	8	65.3	67.4
python-app	13	6	8	61.9	63.3
vampi	9	6	6	62.2	61.8
dsvpwa	17	5	15	54.2	57.7
dvblab	12	6	10	54.5	56.8
vulnerable-api	7	2	7	52.4	56.1
vc-codex-high-seeded-v2-education-lms-django	13	5	12	52.0	55.1
damn-vulnerable-flask-application	8	7	7	55.5	54.8
vc-codex-high-seeded-v2-property-management-fastapi	13	5	13	51.3	54.7
vc-claude-code-seeded-v2-fintech-lending-fastapi	15	6	14	50.6	53.5
vc-codex-seeded-v2-logistics-dispatch-fastapi	15	3	15	48.9	53.0
vc-kimi-code-seeded-v2-marketplace-commerce-fastapi	13	2	14	48.1	53.0
vulnerable-flask-app	11	6	10	50.8	52.9
vc-codex-high-seeded-v2-crm-saas-django	12	4	13	49.3	52.8
vc-codex-high-seeded-v2-marketplace-commerce-fastapi	12	6	13	49.3	52.3
pythonssti	1	1	1	50.0	51.9
vc-claude-code-seeded-v2-crm-saas-django	14	4	14	48.2	51.9
vc-codex-high-seeded-v2-fintech-lending-fastapi	14	2	15	47.1	51.8
vc-codex-high-seeded-v2-logistics-dispatch-fastapi	14	3	15	47.1	51.3
vc-kimi-code-seeded-v2-property-management-fastapi	14	2	16	46.7	51.3
vc-kimi-code-seeded-v2-crm-saas-django	13	6	14	48.1	51.2
vulnerable-python-apps	11	6	11	48.5	51.0
dvpwa	11	9	12	49.3	50.3
vc-kimi-code-seeded-v2-legal-case-django	12	3	14	46.2	50.2
vc-codex-high-seeded-v2-support-desk-fastapi	13	3	15	45.2	49.6
python-insecure-app	4	1	4	45.8	49.4
vulnerable-tornado-app	6	2	8	45.2	49.4
threatbyte	13	13	13	48.7	49.0
vc-kimi-code-seeded-v2-logistics-dispatch-fastapi	14	5	17	45.2	49.0
vc-kimi-code-seeded-v2-hr-payroll-django	13	4	16	44.8	48.9
vc-kimi-code-seeded-v2-fintech-lending-fastapi	15	7	18	45.5	48.6
vc-kimi-code-seeded-v2-support-desk-fastapi	13	7	15	45.2	48.2
vc-claude-code-seeded-v2-hr-payroll-django	12	2	15	43.2	47.7
vc-codex-seeded-v2-marketplace-commerce-fastapi	12	4	17	42.5	46.6
vc-codex-seeded-v2-crm-saas-django	14	6	20	42.2	45.8
owasp-web-playground	12	7	16	42.9	45.6
pygoat	34	32	44	44.0	45.1
vc-codex-high-seeded-v2-hr-payroll-django	10	5	15	41.3	45.0
vc-claude-code-seeded-v2-support-desk-fastapi	13	1	21	39.2	44.2
vc-codex-high-seeded-v2-legal-case-django	10	3	15	40.0	44.2
vc-kimi-code-seeded-v2-education-lms-django	11	5	17	40.5	44.2
extremely-vulnerable-flask-app	13	3	19	39.6	44.0
vc-claude-code-seeded-v2-education-lms-django	13	4	19	39.6	43.8
vc-codex-seeded-v2-legal-case-django	13	4	20	39.4	43.7
vc-codex-seeded-v2-support-desk-fastapi	12	2	18	38.9	43.6
vc-codex-seeded-v2-hr-payroll-django	15	3	24	38.5	43.1
vc-codex-high-seeded-v2-healthcare-clinic-django	10	4	16	38.5	42.1
vc-claude-code-seeded-v2-logistics-dispatch-fastapi	13	6	20	38.4	41.9
vc-claude-code-seeded-v2-healthcare-clinic-django	11	8	18	37.9	40.9
vc-codex-seeded-v2-property-management-fastapi	11	2	20	35.5	39.8
vc-codex-seeded-v2-fintech-lending-fastapi	13	7	24	36.0	39.7
vc-kimi-code-seeded-v2-healthcare-clinic-django	11	5	19	35.5	39.3
damn-vulnerable-graphql-application	13	8	23	35.2	38.6
vc-claude-code-seeded-v2-legal-case-django	10	4	20	33.9	38.0
flask-xss	10	7	20	34.4	37.1
vc-codex-seeded-v2-education-lms-django	11	3	24	31.4	35.8
vc-claude-code-seeded-v2-property-management-fastapi	10	1	23	30.3	34.8
vc-claude-code-seeded-v2-marketplace-commerce-fastapi	10	5	22	29.7	33.3
vc-codex-seeded-v2-healthcare-clinic-django	12	4	29	29.3	33.3
djangoat	14	7	38	27.6	31.2
vulpy	15	6	42	25.7	29.5

Detection by severity

Severity	TP	FP	FN	Recall %
Critical	137	0	18	88.4
High	336	7	416	44.7
Medium	325	2	585	35.7
Low	31	0	55	36.0

Detection by vulnerability class

CWE family	TP	FP	FN	Recall %
Code Injection / RFI	34	0	0	100.0
XPath Injection	4	0	0	100.0
XML External Entities	37	1	1	97.4
Insecure Deserialization	38	0	2	95.0
Open Redirect	38	0	2	95.0
SQL Injection	73	4	5	93.6
Command / OS Injection	44	1	6	88.0
Path Traversal	34	1	10	77.3
Server-Side Request Forgery	29	0	12	70.7
Cross-Site Scripting	65	0	45	59.1
Hardcoded Credentials	38	0	30	55.9
HTTP Header Injection	1	0	1	50.0
Security Misconfiguration	47	0	65	42.0
Sensitive Data Exposure	53	0	102	34.2
Other	263	0	614	30.0
Missing Authentication / Authorization	16	2	67	19.3
Broken Access Control / IDOR	12	0	71	14.5
Denial of Service	3	0	41	6.8

LLM operational metrics

35,305

Avg input tokens

5,278

Avg output tokens

203,853

Avg total tokens

127s

Avg latency / repo

0.5%

JSON repair rate

198

Total runs

±10.3

F2 run-to-run σ

Cost

Total cost

$0.04

Cost / run

$0.002

Cost / 100 LOC

133,782

Python LOC scanned

192

Successful runs

← Back to the leaderboard