Artificial Analysis for model, price, and provider comparison
Artificial Analysis is useful because it compares models across quality, speed, latency, price, context, and provider behavior instead of only one leaderboard score.
If model choice were really about intelligence, the highest-scoring model would be the obvious pick. When I last checked this site, the top intelligence score (57) cost $6.12 per task, while a model scoring 43 cost $0.10 — sixty-one times cheaper for a fourteen-point gap. The “best model” question has no answer until you say best at what cost, at what speed, through which provider.
Source
By: Artificial Analysis
Type: Independent benchmark and comparison site (continuously updated; viewed Sep 2026 — 643 models tracked)
This lives in my bookmarks because it’s the one comparison site that treats a model as a full API decision. Quality matters, but so do speed, price, context, and who actually hosts the weights. For anything running in production, those axes can decide the architecture before quality ever does.
One decision, three axes — and a Pareto line
The site’s core move is refusing to collapse a model into one number. Its Intelligence Index (v4.2 as of my visit) blends ten evaluations — AA-Briefcase, GDPval-AA v2, 𝜏³-Banking, Terminal-Bench v2.1, SciCode, Humanity’s Last Exam, GDP.pdf, CritPt, AA-Omniscience, and AA-LCR v1.1 — so a model can’t ace one famous test and coast. Alongside that sit raw throughput (output tokens per second) and the metric I find most underused: cost per task, the weighted average USD to complete one Intelligence-Index-style task, computed from five separate token prices — input, cache hit, cache write, reasoning, and answer.
Why per-task instead of per-token? Because reasoning models burn tokens invisibly before they answer. Two models with identical input pricing can differ wildly in what a finished job costs once you pay for the thinking. The site’s scatter plots make the trade-offs physical: intelligence on one axis, cost or time on the other, a Pareto line drawn through the models nobody can beat on both axes, and a “most attractive quadrant” flagging the sweet spot. The current spread is dramatic — output speed ranges from ~41 to ~277 tokens per second across leading models (roughly 7×), and cost per task from $0.10 to $6.12 across the top ten (61×).
The same machinery then slices the market usefully: open weights versus proprietary (with licence restrictions labelled), reasoning versus non-reasoning, multimodal versus text-only, by country and by lab. For open-weight models there’s a provider dimension too — the same published weights served by different hosts can differ in speed, price, and even measured quality, so the site rates hosting paths, not just weights. Two newer tools push the same philosophy further: a Model Recommender that optimises for your stated priorities, and Optima, which lets you build a custom benchmark instead of trusting anyone’s default mix.
Three cautions. First, index versions are not comparable: v4.2 removed GPQA Diamond and rebalanced weights, so scores move when methodology moves — check the changelog before comparing a number you saved months ago. Second, variants multiply again here as everywhere else: each effort level (low → max) of the same release is rated separately, and effort changes cost as much as it changes intelligence, so match variants when comparing. Third, “independent” is the site’s genuine selling point but its eval mix is still a mix — a proxy for your workload, not a measurement of it.
What I want to remember
Remember: Never pick a model from a single leaderboard. Compare the job end to end — intelligence, tokens per second, cost per task, context, and provider path — and remember index scores aren’t stable across methodology versions. The Pareto line is the pick list; anything off it needs a reason.