The First Model to Finish the AISI's Network Exercise

The AI Security Institute assessed Anthropic's Claude Mythos Preview on two tracks: standard capture-the-flag challenges, and its own 32-step simulation of an attack on a corporate network. The model reached 73% at expert-level CTF and completed the full network exercise on three of ten attempts — something no model had managed before. The most notable finding is not the scores themselves but that capability scales log-linearly with compute, and has not yet reached a ceiling.

73%success rate at expert-level CTF
3 / 10attempts that completed the full network exercise
22 / 32average steps completed; the previous best was 16
100Mtoken budget per attempt

Context · The institute and the method · Content 1 / 3

The UK AI Security Institute who ran the evaluation

Part of the UK Department for Science, Innovation and Technology, founded in 2023.
Renamed from the AI Safety Institute to the AI Security Institute, reflecting a shift towards concrete national security threats.
It has built a graduated framework: conversational probing, then CTF challenges, then multi-step attack simulation.
It tracks the offensive cyber capability of frontier models over time, and is among the most systematic government evaluators anywhere.

Two Tracks: CTF and TLO one tests technique, the other a whole campaign

CTF challenges
A single technical situation
- The security industry's standard simulated attacks, graded across four levels from entry to expert.
- They measure offensive capability within one technical scenario.
- Cybench, a public 40-task benchmark, is included as well.
The TLO exercise
A 32-step corporate network attack
built by AISI, not public
- From initial reconnaissance to full network takeover, across nine milestones, M1 to M9.
- A human expert takes roughly 20 hours to complete it.
- Each attempt is capped at 100M tokens.

The Last Ones: Nine Milestones the exercise's attack chain

Milestone	Task
M1	Initial reconnaissance
M2	Lateral movement and credential extraction
M3	Browser credential theft
M4	Wiki exploitation and credential replay
M5	Web application exploitation and privilege escalation
M6	Command-and-control reverse engineering and cryptanalysis
M7	Advanced persistence
M8	Infrastructure compromise
M9	Full network takeover

Pivot · Results and scaling · Content 2 / 3

CTF: 73% at Expert Level and Cybench is saturated

Entry level is near saturation, intermediate scores are high, and expert level reaches 73%; all 40 Cybench tasks pass. Before April 2025 no model could complete an expert-level CTF task at all. Cybench is now fully saturated — it no longer discriminates between frontier models, and that is itself a finding.

Cybench 100%All 40 tasks pass; the benchmark no longer discriminates.
Expert 73%A year earlier the success rate at this level was zero.

TLO: Completed for the First Time average steps out of 32, at a 100M token budget

Model	Average steps	Full completions
Mythos Preview	22	3 ✓
Claude Opus 4.6	16	0
GPT-5.4	~14	0
Claude Opus 4.5	11	0
GPT-4o (Aug 2024)	1.7	0, on a 10M token budget

Capability Scales Log-Linearly With Compute the finding that matters most

Capability scales log-linearly with inference-time compute. Going from 10M to 100M tokens raised performance by as much as 59%, and at the test's 100M ceiling there was still no plateau. AISI expects that a budget beyond 100M tokens would keep improving results.

+59%the largest gain in going from 10M to 100M tokens
No ceiling foundMore compute means more offensive capability, with no visible limit yet.

Capability Across Benchmarks comparisons where available

Benchmark	Mythos	Comparison
Firefox exploitation	84%	Opus 4.6: 15.2%
SWE-bench Verified	93.9%	Opus 4.6: 80.8%
CyberGym	83.1%	—
Cybench (40 tasks)	100%	saturated
Zero-day discovery	thousands	including one latent in OpenBSD for 27 years

Of 198 vulnerability reports from Anthropic's red team, 89% agreed exactly with Mythos's own severity assessment. The zero-days span every major operating system and browser.

Resolution · The decision and the limits · Content 3 / 3

Project Glasswing: The Decision Not to Release named after the glasswing butterfly

The name
The glasswing butterfly
- Revealing what is hidden without causing harm.
Scale
From 11 to more than 40 partners
- Partners include Google, Microsoft, AWS, Nvidia and JPMorgan.
- Funding: a usage allowance of up to US$100 million.

Reason for withholding release	Weight (estimated)
A genuine security threat — releasing it would amount to a free red-team tool	~70%
Marketing by alarm, and brand narrative	~15%
Insufficient compute, or the cost of serving it	~15%

What the Test Environment Left Out why the scores do not transfer directly

No active defender — no security operations staff responding in real time.
No endpoint detection — EDR tooling did not exist in the environment.
No penalty for noisy behaviour — actions that would expose an attacker in reality cost nothing here.

The NCSC's Advice: Return to Fundamentals what to do about it

Apply security updates promptly, so known vulnerabilities cannot be exploited quickly.
Strong access control alongside secure configuration.
Comprehensive logging, and NCSC Cyber Essentials certification.

Why this is a turning point

This is the first time an AI has reached human-expert autonomous capability across a complete simulated corporate attack. Two things are true at once, though: the environment had no defender, so the score is an upper bound rather than a prediction about the real world — and capability keeps rising with compute, so that upper bound will keep moving.