Scanner deep-dive
Kolega DevSec Max V0.1.0 by Kolega DevSec Platform ↗
Security-Specialized · Kolega DevSec Platform · scored on 140/140 repositories. Strict scoring (unfinished repos counted as misses).
Methodology note
Full 140-repository coverage. Hybrid deterministic + LLM-review scan over the 66 Python and 74 TypeScript/JavaScript repositories. The Python results are the V0.0.1 run published in 2.1.0 (unchanged; the scanner's Python path did not change between the two versions); the TS/JS results are the 3.0.0 run. Cost is $17.73 in total at off-peak API rates: $3.17 for Python and $14.56 for TS/JS. Ground truth was never supplied to the scanner.84.4
F3 (strict)
79.2
F2 (strict)
90.2%
Recall (strict)
53.2%
Precision
140/140
Repos scored
—
Model
Free
Total cost
—
Avg latency
§
Per-repository breakdown
Each bar shows true positives, false positives, and misses on one repository; bar length is proportional to that repo's labeled vulnerabilities. Ranked by F2.
| Repository | TP | FP | FN | Recall % | F2 |
|---|---|---|---|---|---|
| nextjs-vulnerable-app | 1 | 0 | 0 | 100.0 | 100.0 |
| pythonssti | 2 | 0 | 0 | 100.0 | 100.0 |
| react-test-bench | 1 | 0 | 0 | 100.0 | 100.0 |
| vulnpy | 79 | 9 | 1 | 98.8 | 96.8 |
| damn-vulnerable-flask-application | 14 | 3 | 0 | 100.0 | 95.9 |
| vc-codex-high-seeded-v2-legal-case-django | 25 | 8 | 0 | 100.0 | 94.0 |
| vc-codex-high-seeded-v2-fintech-lending-fastapi | 29 | 10 | 0 | 100.0 | 93.5 |
| node-api-goat | 7 | 3 | 0 | 100.0 | 92.1 |
| vc-claude-code-seeded-v2-hr-payroll-django | 27 | 13 | 0 | 100.0 | 91.2 |
| vc-codex-high-seeded-v2-support-desk-fastapi | 28 | 14 | 0 | 100.0 | 90.9 |
| vc-codex-high-seeded-v2-healthcare-clinic-django | 24 | 5 | 2 | 92.3 | 90.2 |
| python-insecure-app | 7 | 0 | 1 | 87.5 | 89.7 |
| vc-codex-high-seeded-v2-logistics-dispatch-fastapi | 28 | 12 | 1 | 96.5 | 89.7 |
| vc-kimi-code-seeded-v2-healthcare-clinic-django | 28 | 8 | 2 | 93.3 | 89.7 |
| vc-kimi-code-seeded-v2-fintech-lending-fastapi | 31 | 10 | 2 | 93.9 | 89.6 |
| vulpy | 52 | 11 | 5 | 91.2 | 89.3 |
| vc-kimi-code-seeded-v2-hr-payroll-django | 28 | 13 | 1 | 96.5 | 89.2 |
| extremely-vulnerable-flask-app | 29 | 6 | 3 | 90.6 | 89.0 |
| vulnerable-api | 13 | 4 | 1 | 92.9 | 89.0 |
| insecure-web | 8 | 1 | 1 | 88.9 | 88.9 |
| vc-claude-code-seeded-v2-marketplace-commerce-fastapi | 30 | 11 | 2 | 93.8 | 88.8 |
| vc-claude-code-seeded-v2-legal-case-django | 28 | 6 | 3 | 90.3 | 88.6 |
| vc-codex-high-seeded-v2-property-management-fastapi | 24 | 8 | 2 | 92.3 | 88.2 |
| vc-codex-seeded-v2-crm-saas-django | 30 | 4 | 4 | 88.2 | 88.2 |
| vc-codex-seeded-v2-marketplace-commerce-fastapi | 27 | 10 | 2 | 93.1 | 88.2 |
| vc-codex-seeded-v2-logistics-dispatch-fastapi | 27 | 7 | 3 | 90.0 | 87.7 |
| damn-vulnerable-graphql-application | 34 | 16 | 2 | 94.4 | 87.6 |
| vc-codex-high-seeded-v2-education-lms-django | 24 | 13 | 1 | 96.0 | 87.6 |
| vc-kimi-code-seeded-v2-property-management-fastapi | 28 | 12 | 2 | 93.3 | 87.5 |
| vc-kimi-code-seeded-v2-crm-saas-django | 25 | 10 | 2 | 92.6 | 87.4 |
| vc-claude-code-seeded-v3-crm-saas-nestjs-angular | 47 | 27 | 2 | 95.9 | 87.0 |
| owasp-web-playground | 25 | 8 | 3 | 89.3 | 86.2 |
| vc-claude-code-seeded-v3-legal-case-remix | 28 | 23 | 0 | 100.0 | 85.9 |
| threatbyte | 24 | 12 | 2 | 92.3 | 85.7 |
| ivna | 19 | 16 | 0 | 100.0 | 85.6 |
| vulnerable-tornado-app | 13 | 7 | 1 | 92.9 | 85.5 |
| intentionally-vulnerable-python-application | 7 | 6 | 0 | 100.0 | 85.4 |
| vc-claude-code-seeded-v2-fintech-lending-fastapi | 27 | 16 | 2 | 93.1 | 84.9 |
| vc-claude-code-seeded-v2-logistics-dispatch-fastapi | 30 | 15 | 3 | 90.9 | 84.7 |
| vc-deepseek-v4-flash-seeded-v3-legal-case-remix | 21 | 15 | 1 | 95.5 | 84.7 |
| flask-xss | 25 | 3 | 5 | 83.3 | 84.5 |
| vc-codex-high-seeded-v2-marketplace-commerce-fastapi | 25 | 23 | 0 | 100.0 | 84.5 |
| vulnerable-flask-app | 19 | 10 | 2 | 90.5 | 84.1 |
| vc-claude-code-seeded-v2-crm-saas-django | 25 | 12 | 3 | 89.3 | 83.9 |
| vc-codex-high-seeded-v2-crm-saas-django | 23 | 14 | 2 | 92.0 | 83.9 |
| vc-codex-seeded-v2-education-lms-django | 30 | 9 | 5 | 85.7 | 83.8 |
| vc-codex-high-seeded-v3-hr-payroll-remix | 41 | 36 | 1 | 97.6 | 83.7 |
| vc-codex-seeded-v2-healthcare-clinic-django | 34 | 5 | 7 | 82.9 | 83.7 |
| vc-kimi-code-seeded-v3-legal-case-remix | 41 | 40 | 0 | 100.0 | 83.7 |
| vc-codex-high-seeded-v3-healthcare-clinic-nextjs | 41 | 37 | 1 | 97.6 | 83.3 |
| vc-kimi-code-seeded-v2-logistics-dispatch-fastapi | 28 | 16 | 3 | 90.3 | 83.3 |
| vc-codex-seeded-v2-hr-payroll-django | 32 | 5 | 7 | 82.0 | 82.9 |
| python-app | 19 | 12 | 2 | 90.5 | 82.6 |
| vc-claude-code-seeded-v3-support-desk-nestjs-angular | 38 | 29 | 3 | 92.7 | 82.3 |
| vfapi | 9 | 10 | 0 | 100.0 | 81.8 |
| vc-codex-high-seeded-v3-marketplace-commerce-express | 43 | 44 | 1 | 97.7 | 81.7 |
| vc-codex-high-seeded-v3-support-desk-nestjs-angular | 32 | 24 | 3 | 91.4 | 81.6 |
| vc-codex-seeded-v2-legal-case-django | 27 | 7 | 6 | 81.8 | 81.3 |
| dvpwa | 19 | 6 | 4 | 82.6 | 81.2 |
| vc-claude-code-seeded-v3-property-management-fastify-vue | 31 | 32 | 1 | 96.9 | 81.2 |
| vc-codex-high-seeded-v3-education-lms-nextjs | 46 | 46 | 2 | 95.8 | 81.0 |
| pygoat | 69 | 46 | 9 | 88.5 | 80.8 |
| vc-codex-high-seeded-v3-crm-saas-nestjs-angular | 36 | 39 | 1 | 97.3 | 80.7 |
| vc-deepseek-v4-flash-seeded-v3-education-lms-nextjs | 26 | 23 | 2 | 92.9 | 80.7 |
| dsvpwa | 25 | 2 | 7 | 78.1 | 80.6 |
| vc-claude-code-seeded-v3-marketplace-commerce-express | 39 | 43 | 1 | 97.5 | 80.6 |
| vc-kimi-code-seeded-v3-logistics-dispatch-fastify-vue | 42 | 43 | 2 | 95.5 | 80.5 |
| vc-codex-seeded-v2-property-management-fastapi | 27 | 17 | 4 | 87.1 | 80.4 |
| vampi | 13 | 8 | 2 | 86.7 | 80.2 |
| vc-claude-code-seeded-v3-logistics-dispatch-fastify-vue | 47 | 42 | 4 | 92.2 | 80.2 |
| vc-kimi-code-seeded-v3-healthcare-clinic-nextjs | 34 | 34 | 2 | 94.4 | 80.2 |
| dvblab | 19 | 12 | 3 | 86.4 | 79.8 |
| vc-claude-code-seeded-v2-education-lms-django | 26 | 9 | 6 | 81.2 | 79.8 |
| vc-deepseek-v4-flash-seeded-v3-support-desk-nestjs-angular | 24 | 31 | 0 | 100.0 | 79.5 |
| vc-kimi-code-seeded-v3-fintech-lending-express | 38 | 33 | 4 | 90.5 | 79.5 |
| vuln-node-express-swagger | 43 | 32 | 6 | 87.8 | 79.3 |
| vc-claude-code-seeded-v3-healthcare-clinic-nextjs | 39 | 35 | 4 | 90.7 | 79.3 |
| vc-codex-high-seeded-v2-hr-payroll-django | 23 | 22 | 2 | 92.0 | 79.3 |
| vc-kimi-code-seeded-v3-support-desk-nestjs-angular | 32 | 42 | 0 | 100.0 | 79.2 |
| djangoat | 44 | 27 | 8 | 84.6 | 78.9 |
| vc-claude-code-seeded-v3-fintech-lending-express | 39 | 48 | 1 | 97.5 | 78.9 |
| vc-claude-code-seeded-v3-hr-payroll-remix | 35 | 39 | 2 | 94.6 | 78.8 |
| vc-codex-seeded-v2-fintech-lending-fastapi | 29 | 7 | 8 | 78.4 | 78.8 |
| vulnerable-node | 19 | 18 | 2 | 90.5 | 78.5 |
| vc-claude-code-seeded-v2-property-management-fastapi | 27 | 13 | 6 | 81.8 | 78.5 |
| vc-codex-high-seeded-v3-legal-case-remix | 41 | 32 | 6 | 87.2 | 78.5 |
| vulnerable-nodejs | 16 | 10 | 3 | 84.2 | 78.4 |
| vc-codex-high-seeded-v3-property-management-fastify-vue | 38 | 37 | 4 | 90.5 | 78.2 |
| vc-kimi-code-seeded-v2-marketplace-commerce-fastapi | 23 | 16 | 4 | 85.2 | 78.2 |
| dvna | 15 | 13 | 2 | 88.2 | 78.1 |
| vc-codex-seeded-v2-support-desk-fastapi | 24 | 10 | 6 | 80.0 | 77.9 |
| vc-deepseek-v4-pro-seeded-v3-hr-payroll-remix | 24 | 26 | 2 | 92.3 | 77.9 |
| vc-kimi-code-seeded-v2-legal-case-django | 21 | 10 | 5 | 80.8 | 77.8 |
| vc-claude-code-seeded-v2-healthcare-clinic-django | 23 | 9 | 6 | 79.3 | 77.7 |
| vulnnodeapp | 25 | 20 | 4 | 86.2 | 77.6 |
| lets-be-bad-guys | 20 | 13 | 4 | 83.3 | 77.5 |
| vc-claude-code-seeded-v3-education-lms-nextjs | 33 | 28 | 5 | 86.8 | 77.5 |
| vc-kimi-code-seeded-v3-education-lms-nextjs | 45 | 62 | 1 | 97.8 | 77.3 |
| dvws-node | 64 | 51 | 13 | 83.1 | 75.7 |
| vc-kimi-code-seeded-v3-hr-payroll-remix | 33 | 42 | 3 | 91.7 | 75.3 |
| vc-kimi-code-seeded-v3-marketplace-commerce-express | 41 | 61 | 2 | 95.3 | 74.8 |
| vc-codex-high-seeded-v3-logistics-dispatch-fastify-vue | 36 | 25 | 9 | 80.0 | 74.7 |
| vc-codex-high-seeded-v3-fintech-lending-express | 36 | 50 | 3 | 92.3 | 74.4 |
| vc-deepseek-v4-pro-seeded-v3-education-lms-nextjs | 26 | 41 | 1 | 96.3 | 74.3 |
| dsvw | 19 | 1 | 8 | 70.4 | 74.2 |
| vc-kimi-code-seeded-v3-crm-saas-nestjs-angular | 37 | 53 | 3 | 92.5 | 74.0 |
| vulnerable-python-apps | 17 | 10 | 5 | 77.3 | 73.9 |
| vc-kimi-code-seeded-v2-education-lms-django | 22 | 16 | 6 | 78.6 | 73.3 |
| vc-deepseek-v4-flash-seeded-v3-healthcare-clinic-nextjs | 21 | 27 | 3 | 87.5 | 72.9 |
| vc-kimi-code-seeded-v3-property-management-fastify-vue | 31 | 54 | 1 | 96.9 | 72.8 |
| vc-deepseek-v4-pro-seeded-v3-logistics-dispatch-fastify-vue | 29 | 52 | 1 | 96.7 | 72.1 |
| oss-oopssec-store | 35 | 48 | 5 | 87.5 | 72.0 |
| vulnerable-app-angular | 8 | 16 | 0 | 100.0 | 71.4 |
| vc-deepseek-v4-pro-seeded-v3-healthcare-clinic-nextjs | 22 | 28 | 4 | 84.6 | 71.4 |
| nodegoat | 25 | 39 | 3 | 89.3 | 71.0 |
| juice-shop | 68 | 89 | 13 | 84.0 | 70.7 |
| nodejs-goof | 14 | 17 | 3 | 82.3 | 70.7 |
| vc-deepseek-v4-pro-seeded-v3-legal-case-remix | 22 | 34 | 3 | 88.0 | 70.5 |
| vc-deepseek-v4-flash-seeded-v3-fintech-lending-express | 20 | 34 | 2 | 90.9 | 70.4 |
| vulnerable-rest-api-owasp-2023 | 14 | 26 | 1 | 93.3 | 70.0 |
| vuln-node-express | 4 | 5 | 1 | 80.0 | 69.0 |
| vc-deepseek-v4-flash-seeded-v3-crm-saas-nestjs-angular | 20 | 37 | 2 | 90.9 | 69.0 |
| vc-deepseek-v4-flash-seeded-v3-hr-payroll-remix | 15 | 22 | 3 | 83.3 | 68.8 |
| vc-kimi-code-seeded-v2-support-desk-fastapi | 22 | 27 | 6 | 78.6 | 68.3 |
| vc-deepseek-v4-flash-seeded-v3-logistics-dispatch-fastify-vue | 22 | 29 | 6 | 78.6 | 67.5 |
| juice-shop-goof | 45 | 86 | 6 | 88.2 | 67.2 |
| vc-claude-code-seeded-v2-support-desk-fastapi | 25 | 26 | 9 | 73.5 | 66.8 |
| vc-deepseek-v4-pro-seeded-v3-support-desk-nestjs-angular | 17 | 35 | 2 | 89.5 | 66.4 |
| vc-deepseek-v4-pro-seeded-v3-marketplace-commerce-express | 24 | 56 | 2 | 92.3 | 65.2 |
| vc-deepseek-v4-pro-seeded-v3-crm-saas-nestjs-angular | 25 | 49 | 5 | 83.3 | 64.4 |
| ninjasworkout | 19 | 41 | 3 | 86.4 | 64.2 |
| vc-deepseek-v4-flash-seeded-v3-property-management-fastify-vue | 23 | 58 | 2 | 92.0 | 63.5 |
| reactvulna | 1 | 3 | 0 | 100.0 | 62.5 |
| vc-deepseek-v4-flash-seeded-v3-marketplace-commerce-express | 18 | 43 | 3 | 85.7 | 62.1 |
| xvna | 11 | 30 | 1 | 91.7 | 61.8 |
| vc-deepseek-v4-pro-seeded-v3-fintech-lending-express | 25 | 72 | 2 | 92.6 | 61.0 |
| react-security | 5 | 10 | 2 | 71.4 | 58.1 |
| vulnerable-react-fatih | 5 | 10 | 2 | 71.4 | 58.1 |
| vc-deepseek-v4-pro-seeded-v3-property-management-fastify-vue | 16 | 67 | 3 | 84.2 | 50.3 |
| simply-vulnerable-react | 1 | 14 | 0 | 100.0 | 26.3 |
§
Detection by severity
| Severity | TP | FP | FN | Recall % |
|---|---|---|---|---|
| Critical | 145 | 1 | 13 | 91.8 |
| High | 2586 | 4 | 262 | 90.8 |
| Medium | 856 | 5 | 110 | 88.6 |
| Low | 146 | 0 | 20 | 88.0 |
§
Detection by vulnerability class
| CWE family | TP | FP | FN | Recall % |
|---|---|---|---|---|
| SQL Injection | 154 | 1 | 0 | 100.0 |
| Open Redirect | 88 | 2 | 0 | 100.0 |
| Server-Side Request Forgery | 92 | 2 | 1 | 98.9 |
| Code Injection / RFI | 79 | 0 | 1 | 98.8 |
| Path Traversal | 101 | 1 | 3 | 97.1 |
| XML External Entities | 62 | 0 | 2 | 96.9 |
| Security Misconfiguration | 197 | 0 | 7 | 96.6 |
| Insecure Deserialization | 46 | 0 | 2 | 95.8 |
| Command / OS Injection | 129 | 1 | 7 | 94.9 |
| XPath Injection | 16 | 0 | 1 | 94.1 |
| Sensitive Data Exposure | 328 | 0 | 21 | 94.0 |
| Other | 1740 | 0 | 145 | 92.3 |
| Denial of Service | 80 | 0 | 7 | 92.0 |
| Cross-Site Scripting | 175 | 0 | 31 | 85.0 |
| Hardcoded Credentials | 163 | 1 | 53 | 75.5 |
| Missing Authentication / Authorization | 128 | 0 | 45 | 74.0 |
| Broken Access Control / IDOR | 154 | 2 | 78 | 66.4 |
| HTTP Header Injection | 1 | 0 | 1 | 50.0 |
§
Cost
Free
Total cost
0
Lines of code scanned
0
Successful runs