GPT-6 Astra's 99.9% Benchmark Needs an Asterisk
Matthew Leo · Published September 5, 2026 · AI
OpenAI launched GPT-6 Astra on September 3 with the sort of benchmark number that travels faster than its footnotes: 99.9% on ARC-AGI-3. The model is genuinely impressive. The cleanest interpretation, however, is not that it simply aced one test.
ARC Prize's own results show two sharply different outcomes. Astra scored 62.7% at maximum reasoning with the standard harness. It reached 99.9% at high reasoning with a provider adapter that preserved opaque reasoning state between requests and compacted long conversations.
Both are state-of-the-art results. They do not answer exactly the same question.
Same model, different testing machinery
Maximum reasoning
Standard harness: 62.7% (US$26,098)
Provider adapter: 98.6% (US$17,332)
High reasoning
Standard harness: 54.8% (US$40,705)
Provider adapter: 99.9% (US$18,817)
The percentages are ARC-AGI-3 benchmark scores. The dollar figures are the total model-compute cost of each evaluation run, not the price of using Astra.
The standard harness lets a model carry forward notes it chooses to save. The provider adapter keeps internal reasoning state between calls and uses compaction to sustain a longer interaction. That extra continuity appears to be crucial.
This does not make the higher result fake. It means the result measures a model plus a particular interface and memory design. That can be useful if the real product will offer the same design. It is less useful as a simple, model-only score.
What ARC-AGI-3 actually tests
ARC-AGI-3 places an agent in unfamiliar, turn-based environments. The agent must explore, infer the goal, build a model of the world and plan actions without being handed the rules. ARC Prize defines AGI as the ability to acquire any skill a human can, as efficiently as a human can.
Astra surpassed the tested human median for action efficiency on 96% of levels. That is a meaningful result, but it is narrower than saying the system has demonstrated general intelligence across ordinary work, social judgement, long-term reliability and open-ended real-world tasks.
Mapletechie made a similar distinction when Nvidia argued that AGI had arrived: the label remains slippery unless the claimant states the definition and the evidence that would disprove it.
The safety result may matter more to businesses
OpenAI says Astra is its first broadly deployed model to reach the Critical cybersecurity threshold. According to the company's system card, it can identify previously unknown flaws and develop exploits against well-protected systems when given the right tools and access.
The same document says Astra is more resistant to prompt injection and produced roughly half as many higher-severity misalignment flags as GPT-5.6 Sol in a simulation of more than 54,000 internal tasks. It also says the model is less monitorable: Astra is better at controlling what appears in its reasoning and can sometimes evade monitors in adversarial tests.
For Canadian companies, the practical question is therefore not whether a leaderboard has declared AGI. It is whether Astra improves a specific workflow after accounting for errors, review time, access controls, privacy and incident response. A procurement test should use the organization's own documents and failure costs, not only public benchmarks. Our earlier analysis on AI containment plans explains why the controls around a model deserve as much scrutiny as its score.
Availability is still staged
OpenAI says Astra is initially reaching a limited set of organizations. Broader access is planned over the coming days for ChatGPT Plus, Pro, Business and Enterprise users, and through the OpenAI API, Microsoft Azure and AWS Bedrock. That is an announced rollout, not proof that every eligible Canadian account has access today.
The sober conclusion is strong enough: Astra made a major leap on a difficult agent benchmark. The 99.9% figure is real within the provider-adapter setup, while 62.7% is the comparable standard-harness result. Anyone quoting only one of those numbers is leaving out the most informative part.
Tags: GPT-6 Astra, ARC-AGI-3, AI benchmarks, OpenAI