Why this matters
The Overall tab ranks only scanners that have covered every one of the 140 repositories in both languages, which means a scanner cannot appear on it until someone has paid to run it over roughly three-quarters of a million lines of code. When 3.0.0 shipped, seven scanners had done that. Three more have now, from three different vendors and through three different harnesses, and between them they fill in the middle of the table.
What we did
Each new entrant ran on every repository with the standard RealVuln prompt, at a fixed reasoning-effort setting where the model exposes one, and was scored with the same strict F3 as everything else: unfinished repositories count as misses. GPT-6 Astra ran through OpenAI's Codex CLI at high effort, the same setting as GPT-5.6 Sol and Daybreak Blue. Claude Sonnet 5 ran through the Claude Code CLI headless, also at high effort, with its file-editing tools disabled so the agent could only read and report. GLM-5.3 ran through the standard agentic harness on Z.ai's API.
What we observed
GLM-5.3 lands fourth Overall at F3 54.6 — the best result yet from an open-weight model family in this benchmark, and the strongest Python score outside the top three at 63.3. It finished 136 of the 140 repositories: all 66 Python and 70 of 74 TypeScript / JS. The four it did not finish count as misses under strict scoring; scored only on what it completed, its TS/JS figure would be 51.3 rather than 47.4 and its Overall 56.9 rather than 54.6. Those four were not the largest repositories and did not run out of context: in three the agent was still reading files when its session ended, so it never reached the point of writing its findings, and in one the model hit its own output-length limit mid-answer. We now keep the raw output of every failed run, which is how we know. The whole run cost $105.73.
GPT-6 Astra lands fifth at F3 52.1 — 49.4 on Python, 54.3 on TypeScript / JS — a hair behind GLM-5.3 and well behind the two older Codex-family models above it, Sol (74.7) and Daybreak Blue (79.5), at the identical effort setting and prompt. OpenAI's newest and most expensive model in this benchmark is not its strongest one. Astra is also the one full-coverage scanner whose F3 rises from Python to TypeScript; issue 16 looks at why most of the others fall.
Claude Sonnet 5 lands sixth at F3 43.3 — 52.8 on Python, 35.3 on TypeScript / JS — for $130.67 across the whole corpus, about $0.93 per repository. That is remarkably cheap for a frontier model at high effort, and the reason is prompt caching: Claude Code re-reads the repository through its cache, so the billed input for a typical repository is a few dozen tokens against roughly 200,000 cached ones. It also has the best precision of the six full-coverage LLM scanners on Python (69.3%), but loses a third of its recall on TypeScript.
The Overall table now reads: Kolega DevSec Max V0.1.0 84.4, Daybreak Blue 79.5, GPT-5.6 Sol 74.7, GLM-5.3 54.6, GPT-6 Astra 52.1, Claude Sonnet 5 43.3, DeepSeek V4 Flash 41.8, DeepSeek V4 Pro 38.0, Semgrep 10.0.
What changed on the leaderboard itself
Two things readers asked for. Each scanner's row now states the reasoning-effort setting it ran at (for example "codex-cli · high effort") where the model exposes one, because effort is part of a scanner's configuration and comparing a high-effort run against a default-effort run is comparing different scanners. And the cost column is now explicitly cost per 100,000 lines of code scanned, alongside a new cost per 100 vulnerabilities found — the two questions a buyer actually has. The Recall and Precision percentages have given way to Missed Vulns, the plain count of real vulnerabilities a scanner did not report, which is the number that matters most in a recall-weighted benchmark.
What's next
Nine scanners on the Overall tab is enough to rank but not enough to generalise from. The remaining agentic LLM scanners that ran on Python are queued for the TypeScript / JS corpus. The harness now retains the raw model output of every failed run, and the GLM-5.3 failures have already shown why that matters: a run that ends while the agent is still reading, before it has written anything, is indistinguishable from a model that found nothing unless you can see the transcript. All figures are from the frozen 3.0.0 release at realvuln.com/v/3.0.0/.