August 2026 saw eleven significant AI models released inside twenty days, from five or more different providers. That sounds like good news for buyers, and in part it is. But it creates a problem few people mention: independent evaluation can no longer keep pace.
Why leaderboards stopped helping
When a model ships, weeks pass before anyone measures it independently against a serious set of tasks. By then the next one is out. The result is that the rankings you are reading describe the state of play from two cycles ago.
Version fragmentation makes it worse. It is no longer unusual for a new release to land three weeks after the previous one was declared stable. If you are building something that matters, the ground keeps moving under you.
The approach that survives
Stop choosing a model and start choosing per task. Concretely:
- Write three to five examples of the work you actually need done, using your data and your expected output, not generic prompts.
- Run them through two or three candidates.
- Judge the output yourself, against the criteria that matter to you.
That small test takes an hour and is worth more than any table of percentages. A benchmark measures average performance on someone else’s tasks. You care about yours.
Tiers instead of a single pick
Nearly every major provider now splits its lineup into three levels: cheap and fast, mid-range, and expensive for hard problems. That is not marketing, it is a useful structure.
Send routine work to the cheapest tier: classification, data extraction, short answers. Use the middle tier for most production traffic. Save the top tier for the cases where quality genuinely pays for itself. Teams that get this right typically cut their bill several times over with no drop in output quality.
What to abstract away
If you are writing code, do not couple it to one provider. Put a thin layer between your application and the model so that switching means changing a single line of configuration. Six months from now that will save you a day of work.
Most providers now offer an endpoint compatible with the OpenAI format, so that layer is thinner than it used to be.
The takeaway
Whatever edge a single model gives you today lasts weeks, not months. The edge from a well-built process lasts years. Invest in the second one.