Enterprises are pausing development to fix vulnerabilities. A comparison of the approaches on the table.
In April 2026 Anthropic released Claude Mythos Preview to a closed group of companies and government agencies under Project Glasswing, a coalition that includes AWS, Apple, Cisco, CrowdStrike, Google, JPMorganChase, Microsoft, NVIDIA and Palo Alto Networks. The model finds software vulnerabilities on its own and at scale. By late May the partners had reported more than 10,000 high or critical vulnerabilities in widely used software, and a review of a sample of those findings by Anthropic and six independent security firms confirmed more than 90 percent of them as real. Anthropic has kept the model restricted and has described the ease of finding vulnerabilities, compared with the difficulty of fixing them, as the main challenge now facing cybersecurity.
The problem
Enterprises have responded by moving people onto remediation. Bank of America told CNBC on October 14 that it added 1,000 external contractors to 3,000 internal staff to patch its own code and vendor software exposed by frontier models, at a cost of a couple of hundred million dollars in one quarter. We spoke with CISOs at several large enterprises last week and heard the same thing from each of them: development work is paused or slowed, contractors are in, engineers have been moved onto the backlog, and the board is asking for a date when the backlog is closed.
The constraint in every one of those conversations is people. Contractors are paid by the quarter, the engineers have products to ship, and the list of findings keeps growing from every scan. A team in this position has to choose which approach to fund so that it can find what it has, fix what matters and prove the fixes hold. This post compares the approaches in use and explains why deterministic extraction combined with AI-based reasoning is the right design for the problem.
What the defender needs from any approach
Attackers and defenders need different things from the same class of tool. An attacker needs one working path into a system and can make as many attempts as it takes to find it. A defender needs to cover every path, on every code change, with a fixed budget of people and time. Any approach should therefore be judged on five requirements:
- It finds all instances of a vulnerability class in the code it supports, rather than a sample of them.
- It returns the same result on the same code every time it runs.
- It is cheap and fast enough to run on every commit.
- It separates findings an attacker can use from findings that are theoretical, because the people who fix them are the scarce resource.
- It confirms that a fix removed the problem.
Approaches in use today
Adding people
Contractors and reassigned engineers review the backlog, decide what to fix and fix it by hand. This is the approach most enterprises started with, because it needs no new tooling and the people bring judgement and context that tools lack. Each fix is confirmed by the person who made it. The limits are the ones the Bank of America numbers show: coverage depends on who looked at what, two reviewers produce two different lists, the cost grows with headcount, and the work stops when the contract ends.
Traditional scanners
Rule-based static and dynamic scanners have run in enterprise pipelines for a decade. They are repeatable and inexpensive per run, and they are the source of most of the findings now sitting in the backlog. Their weakness is the backlog itself. The output is large, every finding carries the same weight, and the scanner has no way to say which findings an attacker could use against the running system. Teams end up paying people to sort the scanner's output, which brings them back to the first approach.
LLM-based scanning of code
Pointing a large language model at a repository or a pull request and asking it to find vulnerabilities is the approach that matches the threat most directly, since the attacker is using the same kind of model. It needs no setup, it reads everything, and the findings it reports are usually correct. Published benchmarks show where it falls short of the defender's requirements. A 2026 study of 300 repeated scans on the same JavaScript applications found that nearly half of the vulnerabilities reported by the model appeared in only one of five identical runs, that the best configuration reproduced about three quarters of the findings of a deterministic scanner, and that the most expensive configuration cost several times more and scored lower. An earlier peer-reviewed evaluation of eight models reached the same conclusion on robustness: renaming a variable was enough to change the model's verdict in a measurable share of cases. High precision makes the problem harder to see: the findings a team receives are right, so the team has no signal about the findings it did not receive.
Autonomous exploitation tools
Agentic tools attempt to break into the running system the way an attacker would, and each success is proof by construction. For a periodic deep assessment of a critical application this is valuable. As the primary approach for a backlog it has two limits. It finds one path at a time and reports what worked, so it produces no inventory of what exists or of what it did not try. It is also too slow and too expensive to run against every commit, so the backlog refills between runs.
Deterministic extraction with AI-based reasoning and runtime validation
The fifth approach separates the work into three layers and uses a different kind of tool in each. A deterministic layer extracts the facts from code and dependencies: routes, parameters, authentication boundaries, data flow between them, and the libraries in use. A reasoning layer applies rules to those facts and uses a model to build the test scenario and explain the finding. A validation layer runs the test against the live API and records whether the attack succeeded. The rest of this post explains why this design meets the five requirements and the other four approaches each meet some of them.
Comparison
| Approach | Complete | Repeatable | Runs on every commit | Ranked by proof | Confirms the fix |
|---|---|---|---|---|---|
| Adding people | No | No | No | Partial | Yes |
| Traditional scanners | Partial | Yes | Yes | No | Partial |
| LLM-based scanning of code | No | No | No | No | No |
| Autonomous exploitation tools | No | Partial | No | Yes | Partial |
| Deterministic extraction, AI reasoning, runtime validation | Yes | Yes | Yes | Yes | Yes |
Adding people confirms fixes and brings judgement, and fails on coverage, repeatability and cost. Traditional scanners are repeatable and cheap, and they rank nothing, so a fix is confirmed only when a person re-reads the output. LLM-based scanning gives a confident, partial and changing list at a cost that rules it out for every commit. Autonomous exploitation proves what it finds and finds one thing at a time. Each of these is a single layer of the fifth approach used on its own.
Why deterministic extraction with AI-based reasoning meets all five requirements
The requirements separate by layer, and the design assigns each requirement to the layer that can meet it.
Completeness, repeatability and cost come from the extraction layer, which has no model in it. Routes, parameters, authentication boundaries, data flow and dependencies are read from the code by parsers and rules. The same commit produces the same inventory every run, the run takes seconds, and it needs no network access, so it fits in a pipeline on every change. Coverage is bounded by the rule set rather than by the behaviour of a model, which means it can be measured, and it grows each time a rule is added. The repeatability studies point to the same conclusion: the deterministic layer belongs first, and the model belongs on top of its output.
Judgement comes from the reasoning layer, which uses rules first and a model second. The rules decide which facts constitute a risk: an endpoint that takes an object identifier and reads a record without an ownership check, a parameter that flows into a query, a library version with a known exploit on a reachable path. The model works on those structured facts rather than on raw code. It builds the test scenario, selects the values and the identities the test needs, and writes the explanation. This is the work a model does well, and it is work that can vary slightly between runs without changing the inventory underneath it.
Ranking and confirmation come from the validation layer. Each scenario runs against the live API, and the result is recorded as confirmed, cleared or blocked on a missing prerequisite. The fix list is ordered by what succeeded, so the people available work on the paths an attacker can use. After a fix lands, the same scenario runs again and the result changes from confirmed to cleared, which is the evidence the board is asking for.
The design also answers the question the first two approaches cannot. When a contractor asks which of the 2,000 findings to start with, the answer is the ones that were confirmed against the running system, in order of the data they exposed. When the board asks when the backlog is closed, the answer is a count of confirmed findings that are now cleared, measured the same way on every commit.
What this gives a team in fix mode
- One complete inventory of endpoints, parameters, authentication boundaries and dependencies, refreshed on every commit and identical for every reader.
- A fix list ordered by proven exploitability, so contractors and engineers spend the quarter on paths that were confirmed rather than on paths that were reported.
- Evidence that each fix closed its path, produced by re-running the same test, so development work resumes with a record rather than an assumption.
Surface CLI
Surface CLI is the extraction layer of this design, and it is available today. One command in the pipeline reads the application code and its dependency manifests and produces the inventory: every route and parameter, the authentication applied to each, the data flows between them, and the libraries in use with their known vulnerabilities and whether the vulnerable code is reachable. It runs on the build host, the source stays on the host, and the same commit produces the same output on every run. The inventory is published to the APIsec platform, where the reasoning layer applies the rule catalog, builds the test scenarios, and the validation layer runs them against the live API and records the verdicts.
Teams that are already in fix mode can start with the inventory on their own code and a first validation run against one application. Read about Surface CLI and the APIsec platform, or run the first test on your own API, which is free.
Closing
The backlog is a people problem. Every item on it costs human hours, and those hours are the scarcest thing in the programme. What protects them is a list that is complete, produced the same way every time, ordered by what an attacker can actually use, and closed with proof. So before committing hours to any approach next quarter, test the list itself: run it three times on the same commit and compare the three outputs. If the code did not change and the lists did, the difference is noise, and noise is where human hours get wasted.
Sources: CNBC, October 14 2026; Help Net Security, May 26 2026, Project Glasswing update; Snyk VulnBench JS 1.0, arXiv 2606.15762, June 2026; Ullah et al., LLMs Cannot Reliably Identify and Reason About Security Vulnerabilities (Yet?), IEEE Symposium on Security and Privacy 2024.