What 3.1.0 is
A minor release. Our release rules say published data is never rewritten, so the 3.0.0 snapshot at realvuln.com/v/3.0.0/ stays exactly as it was and new results arrive as a new version. 3.1.0 adds one scanner, changes nothing in the ground truth (same 140 repositories, same 4,138 vulnerabilities, same manifest hash), and ships a harness fix that affects how future runs are collected rather than how anything is scored. It is frozen at realvuln.com/v/3.1.0/.
DeepSeek V4.1 Flash
DeepSeek released V4.1 Flash as a short-lived preview — the API model id literally carried its expiry date, 10 September — so we ran it on all 140 repositories in the two days it was available, through the standard agentic harness with the same prompts as V4 Flash. It lands sixth Overall at F3 50.9 (59.8 on Python, 43.4 on TypeScript / JS): 9.1 points above V4 Flash on the identical harness, almost entirely from recall (50.7% against 40.7%) at slightly lower precision (52.3% against 56.3%), and just under GPT-6 Astra (52.1). DeepSeek published no rate card for the preview, so its cost column reads unknown; the token counts for every run are stored and the figure will be backfilled if a price ever appears.
One reproducibility caveat is worth stating plainly: the model no longer exists to be re-run. The findings themselves, the per-run metrics and every scoring step are in the repository, so the score can be regenerated from the stored outputs — but a fresh scan against V4.1 Flash cannot be. That is true of every preview model, and it is why we record the exact API id.
The harness bug
Running V4.1 Flash surfaced a bug that had been quietly costing every agentic LLM scanner coverage. The harness drives models through an open-source coding agent in headless mode. In that mode the agent auto-rejects any tool call that would normally prompt a human for permission — and a rejection ends the session on the spot. Its defaults prompt on three things a security scanner does constantly: reading a .env file (where the hardcoded credentials it is meant to find live), writing a scratch file to /tmp, and re-reading files it has already seen. So a run would be part-way through a repository, hit a .env, and stop — with no findings written and nothing in the logs but a validation failure that looked identical to a model that had found nothing.
Because we started keeping the raw transcript of every failed run, we could see it: 25 of the 28 failed GLM-5.3 and DeepSeek V4.1 runs from this period ended on exactly such a denial, after a handful of steps, in ordinary-sized repositories with plenty of context left. Retries had been recovering about half of them each time, which is what a random cut-off looks like. The fix is an explicit permission policy injected into every run — .env reads and /tmp scratch files allowed; editing the repository, leaving it, and destructive or network shell commands denied — plus disabling the agent's habit of loading configuration from inside the repository being scanned. Re-running V4.1 Flash's nine remaining failures under the policy recovered eight on the first pass with zero denials, and the last on the second.
We are not rescoring earlier releases over this. The affected scanners were retried until their coverage cleared the ranking threshold, and their frozen numbers stand. But it does mean that some of the "did not finish" repositories attributed to GLM-5.3 (136/140 in 3.0.0) were the harness, not the model, and we will say so on its scanner page and re-run those four under the fixed harness in a future release.
What's next
The remaining Python-only agentic scanners onto the TypeScript / JS corpus, now that the harness will not drop runs; false-positive traps for TS/JS; and Java. Every figure in this release is regenerable from the repository with make dashboard.