Google Found 100 Critical Bugs in Two Days. The Pipeline Is the Story

More than 100 critical vulnerabilities in two days is the sort of number that can empty a security-tool budget before lunch. Google says its Mandiant team reached that result during an incident response investigation after corporate source repositories had been stolen. The company described the system behind the result on 18 August 2026: a chain of specialised AI agents called the Agentic Vulnerability Discovery Harness, or AVDH.

The number deserves attention. It does not deserve worship. Google’s own account makes clear that the model was one component inside a much larger engineering system, with a threat model, narrow analysis stages, independent challenges, consultant-written rules, synthetic test code, and human reproduction of every accepted finding. Help Net Security’s report on 19 August and GBHackers’ report on the same date both describe that staged process and the final expert check. They confirm what Google published, although neither report is an independent audit of Mandiant’s internal measurements.

That distinction matters for anyone building AI into code review. A prompt did not find 100 bugs. A controlled investigation pipeline did. Copy the prompt and you will mostly copy the least interesting part.

What Google actually reported

Google Threat Intelligence Group published its AVDH architecture on 18 August 2026 after, it says, ten months of internal use. During that period, the system analysed environments containing tens of millions of lines of code, ran thousands of analysis pipelines, and produced tens of thousands of candidate findings. Google also says the work led to 12 assigned Common Vulnerabilities and Exposures records, including CVE-2026-13242 and CVE-2026-55803, with another dozen flaws in active disclosure at the time of publication. Those are Google’s reported operational results, not a public benchmark that an outside team can rerun.

The headline case came from incident response. Attackers had obtained corporate source repositories, which meant defenders had a narrowing window in which to find the paths an attacker might find. Google says AVDH surfaced more than 100 true-positive critical vulnerabilities in two days. It also describes a separate client review where the system found a remote-code-execution path that could provide initial access.

Those statements tell us that the tool has been useful in serious work. They do not tell us the denominator. Google’s post does not publish how many candidate findings had to be discarded in the two-day case, how the same reviewers would have performed without AVDH, what the total compute bill was, or how much expert time the final checks consumed. “True positive” sounds like a complete metric, but precision needs both accepted and rejected counts, while recall needs a defensible account of what the system missed.

This is not a complaint that Google should reveal a client’s incident file. Confidentiality makes that impossible. It is a warning about reading a case result as a product specification. More than 100 confirmed flaws in one stolen code estate may say as much about that estate, its size, and the urgency of the engagement as it says about the general performance of the tool.

The defensible conclusion is narrower and more useful. As of 20 August 2026, Mandiant has described a working internal system that can compress a large amount of source review into a short incident window, provided experienced people define the method and verify the output. That is a strong result. It is also a very different proposition from “give an agent your repository and trust the report.”

The system reduced ambiguity before asking the model to reason

Most weak AI code-review demos begin with the repository and a broad instruction: find security problems. That leaves the model to decide what the application does, which files matter, who can reach each function, what counts as dangerous, and how much evidence is enough. Every one of those decisions can go wrong while the final prose still sounds confident.

AVDH starts by shrinking that ambiguity. An Explorer agent identifies the purpose and shape of the codebase. Specialist agents examine areas such as authentication, authorisation, and routing. A synthesis stage turns their work into a threat model, which is simply a map of what is exposed, who can interact with it, and which effects matter. A consultant reviews that map before the deeper analysis proceeds.

That approval point carries more weight than the model choice. Imagine a route that deletes a tenant’s export file. The relevant question is not whether a function named delete_export looks risky. The relevant question is whether a caller from tenant A can make it act on tenant B’s file, whether the route is reachable in the deployed configuration, and whether an earlier gateway has already enforced the ownership check. Source text alone rarely answers all of that in one place.

Google’s next stages gather the missing shape. Discovery agents find entry points, including web routes and inter-process listeners, then isolate sources of user-controlled input. An Enrichment agent follows the nearby calls and collects scattered controls such as permission checks, sanitising functions, routing conditions, and database boundaries. Only then do separate agents examine access control and data flow.

This is ordinary analysis discipline expressed as software. A strong human reviewer also starts with scope, maps entry points, follows trust boundaries, gathers the relevant call chain, forms a hypothesis, and tries to disprove it. The useful innovation is that AVDH makes those steps explicit enough to run many times without relying on one person’s working memory.

Engineering teams should steal that shape before they buy anything. If your AI review starts with an unbounded repository prompt, add a machine-readable inventory first. Name the deployed services, public routes, privileged operations, identity boundaries, generated directories, test fixtures, and code that never ships. A smaller, better framed question gives the model less room to invent the system it thinks it is reviewing.

Candidate generation and proof need different jobs

AVDH deliberately separates finding possibilities from deciding that a bug exists. During hypothesis generation, access-control agents look for missing checks and confused identity decisions while data-flow agents trace input toward sensitive functions. Google says those agents perform only light self-validation because their job is to search broadly.

A second group challenges the candidates. Multiple validation agents examine each hypothesis, and a synthesis agent places it into one of three buckets: confirmed, disproven, or rejected because it does not fit the threat model. After deduplication and risk rating, a human consultant attempts to reproduce the issue and checks whether a control outside the model blocks the path. Failed reproductions are discarded.

That split solves a common organisational mistake. Teams often ask one model call to discover, explain, grade, and approve a finding. The same reasoning trace then becomes both the accusation and the evidence. A polished report can pass through CI even when its central path depends on a function that is unreachable, a permission that exists elsewhere, or test-only code that never reaches production.

Search should be generous. Release should be sceptical.

A practical internal pipeline can keep those concerns apart without copying Google’s full architecture. One worker proposes a finding with file paths, call edges, and an explicit attacker capability. A second worker receives the candidate but not the first worker’s persuasive conclusion, then tries to locate the missing control or unreachable step. Deterministic checks compile the affected code, resolve symbols, inspect routes, and run a bounded test. A person accepts the report only when the evidence can be replayed.

The word “confirmed” should therefore mean more than two models agreed. Models trained on similar data can share the same blind spot, and a second answer can merely restate the first in cleaner language. Confirmation should point to an artefact: a failing test, a reproducible request in an isolated environment, a trace through named functions, or a state transition that should have been impossible. The more severe the label, the stronger that artefact should be.

Google’s account is unusually clear on this point. Consultants dynamically reproduce the path and check for compensating controls before disclosure. Help Net Security’s summary highlights the same handoff: candidates that fail the human test are thrown out. The human is not there to add ceremony after the model finishes. The human decides whether the model’s claim survives contact with the real application.

The benchmark is part of the product

A code-review agent can look excellent on a public vulnerability set for the wrong reason. Modern models may have seen the repository, the patch, an advisory, or a write-up during training. When the system identifies the known flaw, you cannot easily tell whether it followed the code or remembered an answer associated with familiar text.

Google says Mandiant addressed that problem with proprietary synthetic codebases. Security consultants manually verify that each planted vulnerability is reachable and can be exercised. The cases vary by software domain, language, architecture, and depth. A grading pipeline compares findings with the known answers, routes unmatched findings for secondary review, resolves duplicates, and puts the grading output in front of another expert. Google reports running that cycle across several domains for each major release, averaging results to account for model variability.

This is the part many teams postpone. They build an agent, point it at three memorable repositories, read a few plausible reports, and call the prototype accurate. The first prompt change then trades one class of misses for another, but there is no stable measurement to reveal the exchange.

A useful evaluation set does not need Google’s budget. It needs cases that belong to your software. Take resolved security defects from your own history, remove details that would make the answer obvious, and preserve the surrounding code needed to reason about reachability. Add clean cases that resemble the vulnerable ones, because a scanner that finds every flaw by accusing every route is operationally useless. Include controls located far from the dangerous function, since that is where shallow review often fails.

Measure the workflow rather than one model score. Record whether the system found the known path, whether an engineer could reproduce it, how many clean cases it accused, how many duplicate tickets it created, how long review took, and what the run cost. A finding that saves ten minutes but creates an hour of triage is a regression, even if the demo looks intelligent.

The release decision also needs a fixed comparison. Run the candidate version and the current version against the same held-out cases. Keep prompts, consultant rules, parsers, model versions, and orchestration code under version control. If the candidate finds one new issue but loses two high-value cases, do not hide that trade behind an average score. Decide whether the new capability is worth the lost coverage, or keep working.

This is normal software engineering. NIST’s Secure Software Development Framework treats secure development practices as work that must be integrated into the software lifecycle rather than added as a final inspection. An AI reviewer belongs inside that same discipline. Its prompts and rules may be written in prose, but changes to them can alter production decisions. Test them like code.

Human expertise moved upstream into rules

The phrase “human in the loop” often describes a tired person clicking approve at the end of a long queue. AVDH shows a more credible arrangement. Mandiant consultants contribute knowledge before the run by defining language, framework, and vulnerability rules. They approve the threat model during the run. They reproduce accepted findings at the end. Human judgement appears at the points where context changes the answer.

That placement matters because the hard part of source review is rarely recognising a suspicious function name. The hard part is knowing which paths are exposed in this framework, which middleware runs first, which identity a helper uses, where generated code changes control flow, and which apparent sink is harmless in the deployed configuration. Those details live in practitioners’ habits long before they appear in a generic model.

Google describes its rule hierarchy with the software domain at the top, followed by language, framework, and vulnerability knowledge. Language and framework rules guide several stages of the pipeline. Vulnerability rules become more specific near discovery, validation, and risk rating. That structure lets a team correct a repeated mistake once instead of adding a warning to every final report.

Suppose an internal framework applies tenant checks through a generated route wrapper. A generic reviewer may repeatedly accuse handler functions because the check does not appear in the handler file. A good system should encode how that wrapper is discovered and how its guarantee is verified. The correction belongs in the context and validation rules, not in a standing instruction to “be careful about false positives.”

The same principle applies outside security. If an agent reviews data migrations, feed it the organisation’s rollback rules and the actual deployment order. If it reviews payment code, teach it where idempotency keys are created and which retries are safe. If it reviews AI features, give it the model gateway, evaluation contract, and data-retention boundaries. Generic intelligence becomes useful when the system supplies local facts at the right stage.

There is a maintenance cost. Rules age when frameworks change, generated code moves, or deployment policy drifts. Every rule therefore needs an owner, a reason, a test case, and a review date. A folder of consultant wisdom with no failing test becomes folklore in a new format.

The AI Native Engineer’s central distinction earns its keep here: the prompt sets intent; the code sets limits. AVDH uses prompts to ask specialised questions, but orchestration controls the order, the scope, the handoffs, and the decision points. The model proposes. The surrounding software determines what that proposal is allowed to become.

A fast scanner creates a slower downstream problem

Finding more defects changes the bottleneck. If a system can produce tens of thousands of candidates, the scarce resource moves from discovery to validation, prioritisation, repair, and proof that the repair worked. Speed at the first stage can make the whole programme slower when every later queue stays the same size.

The two-day figure illustrates the issue. More than 100 critical findings could exceed a team’s immediate repair capacity, even if every finding is accurate. Several may share one root cause. Some may sit behind the same exposed service. Others may affect dead versions or replacement systems. Treating each as an isolated top-priority ticket creates noise at exactly the moment an incident team needs a coherent path.

A useful output contract should support decisions. Each accepted finding needs the affected deployable component, reachable entry point, required attacker access, sensitive effect, evidence, confidence basis, owner, and relation to other findings. It also needs a clear statement of what the system did not verify. That last field prevents a passing test in one environment from quietly becoming a claim about every deployment.

Deduplication cannot stop at matching file and line. Five injection paths may all arise from one unsafe query helper, while three reports around the same endpoint may describe genuinely different identity failures. Grouping should preserve the fix boundary: what single change, if any, would remove the set of paths? Engineers can then repair the shared control and rerun the evidence for every affected route.

Incident response adds another constraint. When source has been stolen, defenders must assume the other side can inspect the same code. Yet hurried fixes can break authentication, logging, or business logic and create fresh exposure. The scanner’s result should enter the existing incident command structure, not create a parallel queue with its own severity language.

Set capacity limits before the first large run. Decide how many high-severity candidates can receive expert reproduction per day, which services get priority, and who may change the status from candidate to confirmed. If the system produces more than the team can check, narrow the scope by exposed component or attack path. A smaller reviewed set beats a huge untrusted report.

Cost belongs in the same dashboard. Google did not publish AVDH’s compute cost in the 18 August post, so nobody outside the engagement can calculate the price of the two-day result from the public material. Your own pipeline can. Record tokens, model calls, wall time, isolated test time, and expert minutes per confirmed finding. Those numbers will tell you whether another validation pass saves money or merely moves cost from false positives into inference.

What to build before you trust an AI code reviewer

Start with one narrow path that your team can measure end to end, rather than a repository-wide scan. Pick a service with known history, a willing owner, and an isolated test environment. Keep the system away from production credentials and live customer data while you learn what it gets wrong.

The following sequence is deliberately less glamorous than a demo. It is also much more likely to survive contact with a release process.

1. Write the scope contract. Name the repository revision, deployable component, included directories, excluded generated or test code, public entry points, privileged effects, and the identity boundaries that matter. Make the run fail when required context is missing rather than letting the model fill gaps.

2. Create a small ground-truth set. Start with several fixed vulnerabilities from your own history and several similar clean cases. Preserve realistic call chains and distant controls. Hold some cases back from prompt development so that tuning cannot quietly memorise the full set.

3. Separate search from challenge. Let one stage propose paths broadly. Give another stage the job of finding the broken assumption, missing edge, unreachable route, or compensating control. Do not let agreement between agents count as proof.

4. Require replayable evidence. Every accepted candidate should point to named code locations and produce a bounded test, trace, or isolated reproduction that another engineer can run. A paragraph of reasoning can explain the evidence. It cannot replace it.

5. Put a person at the decision boundary. Define who can mark a finding confirmed, who owns disclosure, and which severity labels require hands-on reproduction. Give reviewers a reject reason so repeated false patterns can improve the upstream rules.

6. Version the whole review system. Store prompts, rule packs, model identifiers, orchestration code, thresholds, parsers, and benchmark revisions together. A model upgrade is a software change. Run the held-out set before releasing it.

7. Measure the queue after discovery. Track candidate volume, confirmed findings, duplicate groups, review minutes, remediation time, escaped defects, and cost. If triage grows faster than useful findings, narrow the scope or improve validation before increasing scan volume.

8. Keep execution contained. A source-review agent may need to compile code or run a proof in order to test a claim. Give it an isolated environment, synthetic credentials, restricted network access, and disposable state. Reading code does not justify a route to production systems.

A team that completes those steps will own a small but defensible engineering system. It may find fewer bugs in its first week than a broad scanner claims to find in an hour. That is fine. The unit of value is a confirmed, repairable defect found soon enough to matter, not a page of severe labels.

Once that path works, expand by domain rather than by repository count. Add one framework with its own rules and cases. Add one vulnerability class with a clear reproduction method. Add one new deployment shape. Growth by measured capability keeps failures local and makes rollback possible.

The number is impressive because the process was strict

Google’s result will be repeated as a story about agent speed. The more durable lesson is restraint. AVDH narrows context before analysis, gives generation and validation different jobs, tests against private ground truth, injects consultant knowledge into the method, and requires a person to reproduce accepted paths. The reported speed sits on top of those constraints.

Remove the constraints and the same apparent speed becomes a liability. A model can produce security findings faster than a team can disprove them. It can write a persuasive exploit story around dead code. It can turn two views of the same bug into ten tickets. None of that makes the software safer.

The practical move for engineering leaders is to stop evaluating AI reviewers by the quality of one report. Evaluate the system that decides what enters the model, how a claim is challenged, what evidence survives, who can approve it, and whether the next version performs better on cases whose answers are known. That is the product.

As of 20 August 2026, Google’s public account offers a valuable architecture and a striking internal case result. It does not offer a public reproduction of the hundred-bug figure, a full precision and recall table, or a cost model that another team can adopt unchanged. Treat the result as evidence that carefully built agent pipelines can accelerate expert review, then demand your own measurements before trusting one with a release decision.

If you want one practical security and AI engineering note like this each month, the newsletter sends one email per month. The signup is on this site.

Sources

• Google Threat Intelligence Group: Staying Ahead of Adversarial AI Through Agentic Source Code Review, accessed 2026-08-20
• Help Net Security: Google’s AI security agents found 100+ critical software vulnerabilities in just two days, accessed 2026-08-20
• GBHackers: Google Mandiant AI Agents Find Over 100 Critical Vulnerabilities in Source Code Within Two Days, accessed 2026-08-20
• NIST: Secure Software Development Framework, SP 800-218, accessed 2026-08-20