OpenAI's Hacking Model Passes 95% of Exploit Tests. Its Own Defense Benchmark: 0 of 20 Pass.

The number that OpenAI published about its own model

On August 10, 2026, OpenAI expanded its cyber-defense program, Daybreak, into 2 access tiers and shipped a new model built specifically for the work: GPT-5.6-Cyber. Buried in the announcement is a benchmark that says more about AI safety than any refusal-rate chart in the last year: the Advanced Cybersecurity Completion Rate, which measures how often a model completes exploit-chain development, authentication bypass, and privilege-escalation requests rather than refusing them. The standard model, GPT-5.6 Sol, completes 1.5% of that test set. Grant vetted defenders reduced guardrails through Daybreak Blue and the same Sol model barely moves, to 2.0%. Swap in GPT-5.6-Cyber, and completion jumps to 95.0%.

Read that gap carefully. It isn't a capability jump between GPT-5.6 Sol and GPT-5.6-Cyber in the sense of raw intelligence, both trace back to the same frontier model family, released the same week. It's the difference between a safety filter left on and a safety filter removed. The 95% capability was already latent in the model. OpenAI's own numbers say the guardrail, not the intelligence, was the only thing standing between 1.5% and 95%.

What 95% actually finds

This isn't an abstract leaderboard number. OpenAI credits GPT-5.6-Cyber with real, disclosed vulnerabilities in software most of your stack depends on:

The model that can do this at a 95% clip isn't public. OpenAI restricts Daybreak Red to a named list of vetted partners, including Accenture, Cisco, Cloudflare, CrowdStrike, IBM, and Palo Alto Networks. The capability exists at scale. Access to it doesn't, outside roughly 10 companies.

The same model, ranked 2 different ways

Here's the part that doesn't fit the simple “bigger number wins” narrative. On ExploitGym, a benchmark that only checks whether an agent can turn a known bug into working code execution, GPT-5.6-Cyber cleanly beats both Sol and GPT-5.5-Cyber. But OpenAI also runs an internal Vulnerability Discovery and Report Writing evaluation: find a real bug in a live repository, prove it works, then write a report a human security team can act on without redoing the work. On that eval, GPT-5.6-Cyber, the 95% model, scores lowerthan plain GPT-5.6 Sol. OpenAI's own explanation is that the specialized model produces shorter, less detailed reports. On a harder benchmark called ExploitBench, with the sandbox left on, Sol running under Daybreak Blue beats Cyber outright at the standard 300-turn limit, and the gap only narrows once the run length doubles.

The model that's best at breaking in is not automatically the model you want writing the incident report, or running the slower, more careful version of the same job. That's a real design trade-off, not a footnote, for any team deciding which model to point at defensive work.

Now ask it to defend instead of attack

3 weeks before OpenAI's announcement, 1Password's Off-by-1 Labs ran a completely separate test, with no connection to OpenAI or Daybreak: generate patches, not exploits, for 6 real disclosed high-impact vulnerabilities using ChatGPT 5.5 and Claude Opus 4.8. Over 6,000 AI-generated patches later, the results describe a field that hasn't caught up to what offense can already do.

More than half of AI-generated patches in that study either didn't fix the problem or quietly created a new one while claiming to. The researchers' own conclusion: “no conditions we identified resulted in generated patches that would be consistently safe to trust without significant human supervision.”

The SOC-analyst test nobody passes

A second, unrelated study points the same direction. Simbian AI's Cyber Defense Benchmark, published July 18, 2026, ran 20 different LLMs, from GPT-5.6 to Claude Opus 4.6, through 1,002 real attack investigations, each with over 100,000 security events, spanning all 13 MITRE ATT&CK tactics, the standard taxonomy for how real intrusions unfold. The job simulates a SOC analyst: given the raw logs, reconstruct what the attacker actually did.

The benchmark's own pass threshold is 50% tactic coverage. Not one of the 20 models tested reached it, including GPT-5.6 Sol, the same model family that completes 95% of offensive exploit-development requests once its guardrails come off. Put plainly: the top defensive score on this benchmark, 44.5%, is worse than the completion rate of the guardrailed, refusing-almost-everything version of GPT-5.6 on the offensive test.

The audit habit this data trail argues for

If you're shipping AI-assisted patches, code review, or incident response inside your own stack, the 1Password numbers are the ones to sit with: a 53.9% failure-or-new- vulnerability rate isn't a reason to stop using AI for security work, it's a reason to never let a patch merge on the model's word alone. A simple gate, enforced in CI rather than left to reviewer discipline, catches most of the risk before it ships:

# .github/workflows/ai-patch-gate.yml (excerpt)
# Any diff whose commit trailer marks it AI-generated
# requires 2 human approvals AND a passing regression
# suite before merge — no exceptions for "it's just a
# security patch, ship it fast."

on:
  pull_request:
    types: [opened, synchronize]

jobs:
  gate:
    if: contains(github.event.pull_request.body, 'Ai-Generated-Patch: true')
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Require 2 human approvals
        run: gh pr view ${{ github.event.pull_request.number }} \
          --json reviews --jq \
          '[.reviews[] | select(.state=="APPROVED")] | length >= 2'
      - name: Run full regression suite, not just the new test
        run: npm run test:regression -- --changed-behavior-check

That last flag matters more than it looks. 20.1% of the 1Password patches “fixed” the vulnerability by changing application behavior, the kind of regression a test suite scoped only to the new fix will never catch. Run the full suite, not the diff-scoped one, on anything labeled AI-generated.

The takeaway

The AI safety conversation has spent the last year asking whether a model can hack. That question now has an answer, and it's yes, at 95%, rising every few months, and the gap between “can” and “won’t” turned out to be a single configuration flag. The harder question, whether a model can clean up afterward, patch it, explain it, prove the fix didn't break something else, isn't close. Offense scaled first. Every independent benchmark published this month says defense is still waiting for its own 95% moment.

If you're routing AI-assisted security work, patch generation, or incident triage through your stack today, the model choice matters as much as the guardrail policy: the model that's best at finding a bug is not automatically the model you want writing the fix. MegaBrainroutes across 500+ models through one gateway with transparent, at-cost pricing, so testing a defensive-writing model against an offensive-completion model on your own workload doesn't mean managing 5 separate vendor accounts. And if you want an agent watching benchmark releases like these while you sleep, mhermes runs 24/7 on its own isolated VM and can be scheduled to track exactly this kind of story and flag the ones worth your attention.