Scanner deep-dive
DeepSeek V4.1 Flash by DeepSeek ↗
General-Purpose LLM · agentic-v1 · scored on 140/140 repositories. Strict scoring (unfinished repos counted as misses).
Methodology note
Full coverage (140/140); preview model. DeepSeek's V4.1 Flash preview (API id deepseek-v4.1-flash-expires-on-0910), run on 2026-09-09/10 through the standard agentic-v1 harness with the same prompts as V4 Flash, in the two days before the preview was withdrawn. F3 59.8 on Python, 43.4 on TypeScript / JS, 50.9 Overall — 9.1 points above V4 Flash on the identical harness, mostly from recall (50.7% against 40.7%) at slightly lower precision (52.3% against 56.3%). DeepSeek published no rate card for the preview, so its cost is shown as unknown; token counts are stored per run and the figure will be backfilled if a price appears. This was also the campaign that exposed the headless-opencode permission failure: 24 of the first-pass runs died mid-read on an auto-rejected permission prompt (reading a .env file, or writing a scratch file to /tmp). With the harness fix in place, the affected repositories re-ran to 140/140 with zero denials.50.9
F3 (strict)
51.0
F2 (strict)
50.7%
Recall (strict)
52.3%
Precision
140/140
Repos scored
deepseek-v4.1-flash-expires-on-0910
Model
Free
Total cost
98s
Avg latency
§
Per-repository breakdown
Each bar shows true positives, false positives, and misses on one repository; bar length is proportional to that repo's labeled vulnerabilities. Ranked by F2.
| Repository | TP | FP | FN | Recall % | F2 |
|---|---|---|---|---|---|
| react-test-bench | 1 | 0 | 0 | 100.0 | 100.0 |
| vuln-node-express | 5 | 0 | 0 | 100.0 | 100.0 |
| vfapi | 9 | 3 | 0 | 100.0 | 93.8 |
| vampi | 13 | 6 | 2 | 86.7 | 82.3 |
| insecure-web | 7 | 1 | 2 | 77.8 | 79.5 |
| dvblab | 17 | 2 | 5 | 77.3 | 79.4 |
| dsvpwa | 25 | 5 | 7 | 78.1 | 79.1 |
| vc-kimi-code-seeded-v2-property-management-fastapi | 24 | 11 | 6 | 80.0 | 77.4 |
| vc-codex-seeded-v2-hr-payroll-django | 32 | 20 | 7 | 82.0 | 76.9 |
| vc-kimi-code-seeded-v2-legal-case-django | 22 | 19 | 4 | 84.6 | 75.9 |
| intentionally-vulnerable-python-application | 5 | 0 | 2 | 71.4 | 75.8 |
| dsvw | 19 | 0 | 8 | 70.4 | 74.8 |
| python-app | 15 | 2 | 6 | 71.4 | 74.3 |
| vc-codex-high-seeded-v2-hr-payroll-django | 19 | 10 | 6 | 76.0 | 73.6 |
| owasp-web-playground | 24 | 28 | 4 | 85.7 | 73.2 |
| vulnerable-flask-app | 17 | 20 | 4 | 81.0 | 70.2 |
| threatbyte | 17 | 1 | 9 | 65.4 | 69.7 |
| dvpwa | 16 | 7 | 7 | 69.6 | 69.6 |
| vc-claude-code-seeded-v2-legal-case-django | 21 | 7 | 10 | 67.7 | 69.1 |
| vc-codex-high-seeded-v2-marketplace-commerce-fastapi | 17 | 6 | 8 | 68.0 | 69.1 |
| vulnerable-tornado-app | 9 | 2 | 5 | 64.3 | 67.2 |
| vc-kimi-code-seeded-v2-marketplace-commerce-fastapi | 18 | 8 | 9 | 66.7 | 67.2 |
| vc-claude-code-seeded-v3-fintech-lending-express | 27 | 18 | 13 | 67.5 | 65.9 |
| vc-kimi-code-seeded-v2-support-desk-fastapi | 18 | 7 | 10 | 64.3 | 65.7 |
| vc-codex-high-seeded-v2-property-management-fastapi | 17 | 9 | 9 | 65.4 | 65.4 |
| vc-kimi-code-seeded-v2-logistics-dispatch-fastapi | 20 | 9 | 11 | 64.5 | 65.4 |
| vc-claude-code-seeded-v2-fintech-lending-fastapi | 19 | 11 | 10 | 65.5 | 65.1 |
| vc-codex-high-seeded-v2-education-lms-django | 16 | 7 | 9 | 64.0 | 65.0 |
| vc-codex-high-seeded-v2-logistics-dispatch-fastapi | 19 | 12 | 10 | 65.5 | 64.6 |
| vc-kimi-code-seeded-v2-crm-saas-django | 17 | 7 | 10 | 63.0 | 64.4 |
| vc-claude-code-seeded-v2-crm-saas-django | 18 | 10 | 10 | 64.3 | 64.3 |
| vc-claude-code-seeded-v3-property-management-fastify-vue | 24 | 35 | 8 | 75.0 | 64.2 |
| vulnpy | 50 | 20 | 30 | 62.5 | 64.1 |
| dvna | 11 | 8 | 6 | 64.7 | 63.2 |
| vulnerable-node | 13 | 6 | 8 | 61.9 | 63.1 |
| lets-be-bad-guys | 14 | 2 | 10 | 58.3 | 62.5 |
| nextjs-vulnerable-app | 1 | 3 | 0 | 100.0 | 62.5 |
| vulnerable-api | 8 | 0 | 6 | 57.1 | 62.5 |
| vulnerable-app-angular | 6 | 10 | 2 | 75.0 | 62.5 |
| vc-codex-seeded-v2-logistics-dispatch-fastapi | 18 | 8 | 12 | 60.0 | 61.6 |
| vc-codex-high-seeded-v2-crm-saas-django | 15 | 7 | 10 | 60.0 | 61.5 |
| vc-claude-code-seeded-v2-healthcare-clinic-django | 18 | 13 | 11 | 62.1 | 61.2 |
| vc-kimi-code-seeded-v2-fintech-lending-fastapi | 20 | 12 | 13 | 60.6 | 61.0 |
| vc-codex-high-seeded-v2-fintech-lending-fastapi | 16 | 3 | 13 | 55.2 | 59.3 |
| vulpy | 31 | 3 | 26 | 54.4 | 59.2 |
| vc-kimi-code-seeded-v3-logistics-dispatch-fastify-vue | 27 | 25 | 17 | 61.4 | 59.2 |
| vc-kimi-code-seeded-v2-education-lms-django | 17 | 15 | 11 | 60.7 | 59.0 |
| vc-codex-high-seeded-v2-healthcare-clinic-django | 16 | 17 | 10 | 61.5 | 58.4 |
| flask-xss | 16 | 3 | 14 | 53.3 | 57.6 |
| vc-kimi-code-seeded-v3-marketplace-commerce-express | 25 | 20 | 18 | 58.1 | 57.6 |
| vc-codex-seeded-v2-support-desk-fastapi | 17 | 11 | 13 | 56.7 | 57.4 |
| vc-codex-seeded-v2-property-management-fastapi | 18 | 15 | 13 | 58.1 | 57.3 |
| extremely-vulnerable-flask-app | 17 | 4 | 15 | 53.1 | 57.0 |
| vc-claude-code-seeded-v2-hr-payroll-django | 15 | 9 | 12 | 55.6 | 56.8 |
| vc-codex-high-seeded-v2-support-desk-fastapi | 15 | 6 | 13 | 53.6 | 56.4 |
| vc-kimi-code-seeded-v2-hr-payroll-django | 16 | 10 | 13 | 55.2 | 56.3 |
| vc-codex-high-seeded-v3-marketplace-commerce-express | 24 | 14 | 20 | 54.5 | 56.1 |
| vc-deepseek-v4-flash-seeded-v3-legal-case-remix | 13 | 15 | 9 | 59.1 | 56.0 |
| pygoat | 41 | 14 | 37 | 52.6 | 55.9 |
| damn-vulnerable-graphql-application | 22 | 32 | 14 | 61.1 | 55.6 |
| pythonssti | 1 | 0 | 1 | 50.0 | 55.6 |
| vc-claude-code-seeded-v3-logistics-dispatch-fastify-vue | 28 | 20 | 23 | 54.9 | 55.6 |
| node-api-goat | 4 | 5 | 3 | 57.1 | 54.1 |
| python-insecure-app | 4 | 1 | 4 | 50.0 | 54.1 |
| vc-codex-high-seeded-v3-legal-case-remix | 25 | 20 | 22 | 53.2 | 53.6 |
| vc-claude-code-seeded-v2-education-lms-django | 16 | 6 | 16 | 50.0 | 53.3 |
| vc-codex-high-seeded-v3-property-management-fastify-vue | 22 | 18 | 20 | 52.4 | 52.9 |
| vc-kimi-code-seeded-v3-fintech-lending-express | 23 | 27 | 19 | 54.8 | 52.8 |
| vc-codex-seeded-v2-legal-case-django | 16 | 4 | 17 | 48.5 | 52.6 |
| vc-claude-code-seeded-v3-hr-payroll-remix | 19 | 14 | 18 | 51.3 | 52.5 |
| vc-claude-code-seeded-v2-logistics-dispatch-fastapi | 16 | 5 | 17 | 48.5 | 52.3 |
| vc-claude-code-seeded-v3-legal-case-remix | 15 | 18 | 13 | 53.6 | 51.7 |
| vc-claude-code-seeded-v3-marketplace-commerce-express | 20 | 14 | 20 | 50.0 | 51.5 |
| vc-kimi-code-seeded-v3-legal-case-remix | 23 | 37 | 18 | 56.1 | 51.3 |
| vc-codex-seeded-v2-crm-saas-django | 17 | 13 | 17 | 50.0 | 51.2 |
| vc-kimi-code-seeded-v3-property-management-fastify-vue | 18 | 30 | 14 | 56.2 | 51.1 |
| vc-codex-high-seeded-v3-crm-saas-nestjs-angular | 18 | 11 | 19 | 48.6 | 50.8 |
| vc-codex-high-seeded-v3-support-desk-nestjs-angular | 18 | 19 | 17 | 51.4 | 50.8 |
| vc-claude-code-seeded-v3-crm-saas-nestjs-angular | 23 | 9 | 26 | 46.9 | 50.4 |
| vc-claude-code-seeded-v3-education-lms-nextjs | 20 | 27 | 18 | 52.6 | 50.3 |
| vc-codex-seeded-v2-healthcare-clinic-django | 20 | 15 | 21 | 48.8 | 50.3 |
| vc-claude-code-seeded-v2-support-desk-fastapi | 16 | 8 | 18 | 47.1 | 50.0 |
| vc-deepseek-v4-flash-seeded-v3-education-lms-nextjs | 14 | 14 | 14 | 50.0 | 50.0 |
| djangoat | 24 | 9 | 28 | 46.2 | 49.8 |
| vc-codex-seeded-v2-marketplace-commerce-fastapi | 14 | 11 | 15 | 48.3 | 49.6 |
| vc-kimi-code-seeded-v2-healthcare-clinic-django | 14 | 7 | 16 | 46.7 | 49.6 |
| ivna | 10 | 15 | 9 | 52.6 | 49.5 |
| vc-deepseek-v4-flash-seeded-v3-healthcare-clinic-nextjs | 12 | 15 | 12 | 50.0 | 48.8 |
| vc-kimi-code-seeded-v3-support-desk-nestjs-angular | 17 | 30 | 15 | 53.1 | 48.6 |
| vc-codex-high-seeded-v3-logistics-dispatch-fastify-vue | 21 | 16 | 24 | 46.7 | 48.4 |
| vc-claude-code-seeded-v2-marketplace-commerce-fastapi | 14 | 4 | 18 | 43.8 | 47.9 |
| vc-claude-code-seeded-v2-property-management-fastapi | 15 | 10 | 18 | 45.5 | 47.8 |
| vc-codex-seeded-v2-fintech-lending-fastapi | 16 | 5 | 21 | 43.2 | 47.3 |
| vulnerable-python-apps | 10 | 8 | 12 | 45.5 | 47.2 |
| vc-codex-high-seeded-v3-hr-payroll-remix | 20 | 24 | 22 | 47.6 | 47.2 |
| vulnnodeapp | 13 | 11 | 16 | 44.8 | 46.4 |
| vc-deepseek-v4-pro-seeded-v3-support-desk-nestjs-angular | 9 | 12 | 10 | 47.4 | 46.4 |
| vc-deepseek-v4-flash-seeded-v3-crm-saas-nestjs-angular | 11 | 21 | 11 | 50.0 | 45.8 |
| vc-codex-high-seeded-v3-healthcare-clinic-nextjs | 19 | 21 | 23 | 45.2 | 45.7 |
| vc-codex-seeded-v2-education-lms-django | 15 | 9 | 20 | 42.9 | 45.7 |
| vc-deepseek-v4-flash-seeded-v3-logistics-dispatch-fastify-vue | 13 | 18 | 15 | 46.4 | 45.5 |
| vc-deepseek-v4-pro-seeded-v3-property-management-fastify-vue | 9 | 14 | 10 | 47.4 | 45.5 |
| vc-deepseek-v4-pro-seeded-v3-healthcare-clinic-nextjs | 12 | 17 | 14 | 46.2 | 45.1 |
| vc-claude-code-seeded-v3-support-desk-nestjs-angular | 19 | 28 | 22 | 46.3 | 45.0 |
| vc-deepseek-v4-flash-seeded-v3-fintech-lending-express | 10 | 13 | 12 | 45.5 | 45.0 |
| vc-kimi-code-seeded-v3-healthcare-clinic-nextjs | 16 | 20 | 20 | 44.4 | 44.4 |
| vc-codex-high-seeded-v3-fintech-lending-express | 17 | 23 | 22 | 43.6 | 43.4 |
| vc-deepseek-v4-flash-seeded-v3-hr-payroll-remix | 8 | 14 | 10 | 44.4 | 42.6 |
| vc-claude-code-seeded-v3-healthcare-clinic-nextjs | 18 | 23 | 25 | 41.9 | 42.3 |
| vc-deepseek-v4-pro-seeded-v3-logistics-dispatch-fastify-vue | 12 | 12 | 18 | 40.0 | 41.7 |
| vc-deepseek-v4-flash-seeded-v3-marketplace-commerce-express | 9 | 17 | 12 | 42.9 | 40.9 |
| nodejs-goof | 7 | 11 | 10 | 41.2 | 40.7 |
| vc-kimi-code-seeded-v3-crm-saas-nestjs-angular | 16 | 21 | 24 | 40.0 | 40.6 |
| vc-deepseek-v4-pro-seeded-v3-legal-case-remix | 10 | 17 | 15 | 40.0 | 39.4 |
| vulnerable-rest-api-owasp-2023 | 7 | 22 | 8 | 46.7 | 39.3 |
| vc-codex-high-seeded-v3-education-lms-nextjs | 18 | 19 | 30 | 37.5 | 39.3 |
| vc-deepseek-v4-pro-seeded-v3-marketplace-commerce-express | 11 | 35 | 15 | 42.3 | 36.7 |
| vc-deepseek-v4-flash-seeded-v3-property-management-fastify-vue | 9 | 15 | 16 | 36.0 | 36.3 |
| react-security | 3 | 11 | 4 | 42.9 | 35.7 |
| vc-deepseek-v4-flash-seeded-v3-support-desk-nestjs-angular | 8 | 10 | 16 | 33.3 | 35.1 |
| dvws-node | 24 | 20 | 53 | 31.2 | 34.1 |
| vc-deepseek-v4-pro-seeded-v3-education-lms-nextjs | 9 | 18 | 18 | 33.3 | 33.3 |
| juice-shop-goof | 16 | 22 | 35 | 31.4 | 33.1 |
| vuln-node-express-swagger | 15 | 17 | 34 | 30.6 | 32.9 |
| nodegoat | 8 | 12 | 20 | 28.6 | 30.3 |
| xvna | 4 | 15 | 8 | 33.3 | 29.9 |
| vc-deepseek-v4-pro-seeded-v3-hr-payroll-remix | 8 | 23 | 18 | 30.8 | 29.6 |
| vulnerable-nodejs | 5 | 8 | 14 | 26.3 | 28.1 |
| vc-deepseek-v4-pro-seeded-v3-crm-saas-nestjs-angular | 8 | 18 | 22 | 26.7 | 27.4 |
| juice-shop | 20 | 26 | 61 | 24.7 | 27.0 |
| ninjasworkout | 6 | 17 | 16 | 27.3 | 27.0 |
| oss-oopssec-store | 11 | 34 | 29 | 27.5 | 26.8 |
| damn-vulnerable-flask-application | 3 | 8 | 11 | 21.4 | 22.4 |
| simply-vulnerable-react | 1 | 19 | 0 | 100.0 | 20.8 |
| vulnerable-react-fatih | 2 | 21 | 5 | 28.6 | 19.6 |
| vc-deepseek-v4-pro-seeded-v3-fintech-lending-express | 5 | 16 | 22 | 18.5 | 19.4 |
| vc-codex-high-seeded-v2-legal-case-django | 5 | 27 | 20 | 20.0 | 18.9 |
| reactvulna | 0 | 9 | 1 | 0.0 | 0.0 |
| vc-kimi-code-seeded-v3-education-lms-nextjs | 0 | 36 | 46 | 0.0 | 0.0 |
| vc-kimi-code-seeded-v3-hr-payroll-remix | 0 | 34 | 36 | 0.0 | 0.0 |
§
Detection by severity
| Severity | TP | FP | FN | Recall % |
|---|---|---|---|---|
| Critical | 143 | 0 | 15 | 90.5 |
| High | 1430 | 0 | 1418 | 50.2 |
| Medium | 481 | 5 | 485 | 49.8 |
| Low | 45 | 0 | 121 | 27.1 |
§
Detection by vulnerability class
| CWE family | TP | FP | FN | Recall % |
|---|---|---|---|---|
| HTTP Header Injection | 2 | 0 | 0 | 100.0 |
| Insecure Deserialization | 46 | 0 | 2 | 95.8 |
| Command / OS Injection | 125 | 0 | 11 | 91.9 |
| XML External Entities | 58 | 0 | 6 | 90.6 |
| SQL Injection | 136 | 0 | 18 | 88.3 |
| XPath Injection | 15 | 0 | 2 | 88.2 |
| Path Traversal | 90 | 0 | 14 | 86.5 |
| Code Injection / RFI | 67 | 0 | 13 | 83.8 |
| Open Redirect | 72 | 0 | 16 | 81.8 |
| Server-Side Request Forgery | 71 | 2 | 22 | 76.3 |
| Cross-Site Scripting | 139 | 0 | 67 | 67.5 |
| Security Misconfiguration | 106 | 0 | 98 | 52.0 |
| Denial of Service | 39 | 0 | 48 | 44.8 |
| Hardcoded Credentials | 94 | 0 | 122 | 43.5 |
| Broken Access Control / IDOR | 99 | 0 | 133 | 42.7 |
| Sensitive Data Exposure | 138 | 0 | 211 | 39.5 |
| Other | 736 | 0 | 1149 | 39.0 |
| Missing Authentication / Authorization | 66 | 3 | 107 | 38.2 |
§
LLM operational metrics
84,245
Avg input tokens
21,747
Avg output tokens
987,669
Avg total tokens
98s
Avg latency / repo
0.0%
JSON repair rate
140
Total runs
§
Cost
Free
Total cost
741,034
Lines of code scanned
140
Successful runs