Executive Summary
GPT-6 Astra delivers strong benchmark performance while using fewer tokens than previous frontier models, meaning its higher price per token does not necessarily translate into a higher cost per task. In our testing, Astra with low effort could successfully complete complex coding tasks. Businesses should therefore match the reasoning effort to the task, evaluate the total cost of completing the work, and use higher effort where testing shows a measurable improvement. Despite these promising results, the release also comes amid concerns about AI safety and the use of unpublished research, reinforcing the need to review and test outputs before using them in production.
GPT-6 Astra costs more per token but
can cost less per successful coding task.
Explore ML6’s test results and
choose the right reasoning effort.
OpenAI released GPT-6 Astra on September 3rd, its most powerful frontier model to date. The release has been much anticipated by the company, which sparked the era of LLMs with GPT-3 back in 2020, but which has been under increasing pressure from its rival Anthropic, as well as open-source models. Astra has impressive results on various benchmarks, such as reaching 99.9% on ARC-AGI-3, and performs especially well on computer-use tasks.
However, the release has not been without controversy. The launch followed a safety incident involving other unreleased OpenAI models, which autonomously hacked Hugging Face. OpenAI also announced that its unreleased models had solved a Millennium Prize maths problem, but researchers disputed the claim, saying OpenAI may have used their unpublished research without permission.
GPT-6 Astra is OpenAI’s latest frontier model, designed for complex reasoning, coding, computer use, and research. The model can take both text and image as input, has a context window of 1.05 million tokens, and can output up to 128,000 tokens. Astra can also be configured with different reasoning effort levels, running from low to max. GPT-6 Astra has an API cost of $10/$50 per 1 million input/output tokens, which is 2.5 times the cost per token of GPT-5.6 Sol at $4/$20, OpenAI’s previously most powerful model.
Independent benchmarks also show improved performance over previously released frontier models. Artificial Analysis, an independent AI benchmarking organization, scores GPT-6 Astra (Max) at 53 on its Intelligence Index, compared to 47 for GPT-5.6 Sol (Max). The Intelligence Index combines ten evaluations that test a model’s capabilities across tasks including coding, scientific reasoning, knowledge work, and agentic tasks. Astra’s score of 53 puts it equal first with Claude Fable 5.1, making it one of the highest-performing models tested by Artificial Analysis. The model has achieved this performance through the largest training run OpenAI has performed by far, training on more than 100,00 GPUs.
One of Astra's biggest improvements is its token efficiency. Token efficiency describes how many tokens a model uses to complete a task, including both input and output tokens; fewer tokens generally reduce the cost of completing the task. Artificial Analysis tested this with its Coding Agent Index, which measures model performance on coding tasks while running inside an agent harness. Using Codex, GPT-6 Astra (Max) achieved a score of 62, compared to 55 for GPT-5.6 Sol (Max), while using an average of 3.3 million tokens per task compared to 10.2 million for GPT-5.6 Sol. This represents a 68% decrease in token usage, despite Astra achieving a higher score. Astra also used fewer tokens than Claude Fable 5.1, which required 5.7 million tokens per task to achieve the same score of 62.
To see how GPT-6 compares with existing models in the real world, we tested Astra on a real-life task: migrating Kubeflow Pipelines from V1 to V2 in one-shot. The pipelines train and evaluate computer vision models for a range of applications, and a machine learning engineer at ML6 undertook this real-world task. The prompt for the task was minimal, instructing the agent to migrate all pipelines in one repo by following an example pipeline that had already been migrated. The task was open-ended, with no clear definition of success, and involved migrating 12 pipelines across over 300 files. We assigned the task to GPT-5.6 Sol (Low and High effort) and GPT-6 Astra (Low and High effort).
The results of this are shown in the table below:
| Model | Tokens | API Cost (USD) | Time | Successful |
| GPT-5.6 Sol (low) | 5,496,509 | 2.99 | 20m39s | No |
| GPT-5.6 Sol (high) | 36,594,050 | 18.25 | 1h22m8s | Yes |
| GPT-6 Astra (low) | 7,006,253 | 10.80 | 34m15s | Yes |
| GPT-6 Astra (high) | 8,333,293 | 12.40 | 33m1s | Yes |
The migration run using GPT-5.6 Sol (low) did not migrate the pipelines successfully; several bugs were apparent, including missing dependencies that caused compile and runtime errors. Additionally, the agent did not correctly follow the instructions given to it. Instead of migrating the pipelines correctly by following the example it had already been given, it overwrote the existing pipelines. It did not follow coding best practices, failing to modularise the components of each pipeline so that they could be reused.
The migration run for GPT-5.6 Sol (high) was successful, as was GPT-6 Astra (low) and GPT-6 Astra (high). Here we can confirm the token and time efficiency of GPT-6 Astra with low effort, as it successfully migrated the pipelines without runtime or compile errors, while using significantly fewer tokens and taking less time than GPT-5.6 Sol (high). All three models followed the example migrated pipeline successfully, ensuring that they followed coding best practices. This does show however that this migration test has been saturated by GPT-6 Astra’s abilities.
The results above are from ML6’s internal testing, rather than independently published benchmarks.
To get the most out of Astra, you need to play to its strengths. This means selectively using the model, with the right effort setting. As can be seen from independent benchmarking and through our own testing, there is a large difference in cost when using Astra with Low effort and High effort. Low-effort Astra can be more cost-effective than cheaper models with higher reasoning. This is counterintuitive when compared to other recently released frontier models, such as Claude Fable 5.1, where the cost of using them was prohibitive for many simpler tasks. By matching the level of effort to the task, you can efficiently use token budgets to their fullest extent, allowing you to exploit Astra for more routine or day-to-day tasks.
For business-essential tasks, using GPT-6 Astra with high effort may be worth the additional cost when errors would require significant time or resources to resolve. These could include migrating complex production pipelines, where missing dependencies or runtime errors could disrupt existing workflows. However, our testing does not show that high effort is always necessary, as both Astra configurations completed the migration successfully. The decision should therefore depend on the task's difficulty and whether higher effort provides a measurable improvement on similar tasks. Regardless of the effort setting, review and test outputs before using them in production.
→ Match the level of effort to the difficulty of the task
→ Reserve higher effort levels for business-essential tasks
→ Compare the cost of completing a task, not just the cost per token