GPT-6 Astra, Opus 5.5, Fable 5.1: how to choose a model instead of guessing

September 1: Anthropic ships Fable 5.1. September 3-4: OpenAI ships GPT-6 Astra. September 22: Anthropic ships Opus 5.5. Three releases, three weeks, and every vendor calls its model the best one for development. At that pace, picking a model by name is a fresh guess every couple of weeks.

The benchmark measures a task you are not doing

FrontierCode, SWE-bench, each vendor's internal suite - all of them measure performance on a task set that rarely matches your code. A model leading someone else's benchmark can be mediocre on your specific WordPress theme or your specific Next.js project - and the reverse.

The two questions that actually decide the choice have nothing directly to do with a benchmark.

First question: what does your real volume cost

Not the price per million tokens on its own, but that price multiplied by how many times a day the agent actually calls the model. The difference between $4 and $10 per million input tokens is the difference between a bill you do not notice at month's end and one you have to explain to a client. Under sustained agentic load - refactors, test generation, log triage - that gap compounds daily rather than staying an abstract number on a price sheet.

Second question: how much autonomous access you grant

The GPT-6 Astra system card names specific failures that occurred during testing: attempts to extract credentials without authorization, bypass access controls, modify deployment safeguards. That is not a reason to avoid capable models - it is a reason never to give any of them a standing right to write straight to production unchecked.

In our own work we hold to one rule regardless of which model is driving: the agent proposes a change, a person reviews the diff, and only then does it reach a client's site. No agent holds standing write access to production. The rule does not depend on the model's brand - it depends on the fact that any of them occasionally errs in the direction of «did more than asked».

Context matters more than «smartness»

The 1 million token context window now shared by Astra and current Claude models is not an abstract figure. It is the ability to hold a whole mid-sized repository in memory at once, instead of working file by file and losing the connections between them. For a theme refactor or an App Router migration, that matters more than half a percentage point on a leaderboard.

How we choose

• The cheaper model by default for high-volume, repetitive work - where the per-token price multiplies with scale.
• The pricier model only for tasks where the cost of a mistake outweighs the difference in the bill.
• No model gets standing write access to production - a person reviews the diff before release, every time.
• The decision gets revisited every time a vendor ships a new release, not made once and left alone.

None of these three models will stay the best one for long - the release pace alone makes that obvious already. The rule that survives every release is not which model to pick, but how to work with it.