The Known Good — AI model & API provider analysis

Quality

Quality of leading AI models, aggregated from published independent evaluations

Known Good IndexCoding IndexAgentic Index

Known Good Index

Known Good Index v1.3 incorporates 7 evaluations: GPQA Diamond, Mock AIME 2024–25, MATH Level 5, SWE-bench Verified, LiveBench, Humanity's Last Exam, Terminal-Bench. Normalised 0–100. Scores ingested from Epoch AI (CC-BY) and LiveBench.

⇩

💡 A lightbulb marks a model its provider publishes as a reasoning model

The Known Good

Known Good Index v1.3 combines seven public benchmarks, each min-max normalised across tracked models then weighted equally. We do not run these evaluations ourselves — scores are ingested from Epoch AI (CC-BY) and LiveBench. See the Index methodology for weights, normalisation, and per-eval provenance.

Coding Index

Software engineering and agentic terminal ability

Coding IndexSWE-bench VerifiedTerminal-BenchLiveBench CodingAgentic Index

Coding Index

Coding Index, built from SWE-bench Verified, LiveBench Coding and Terminal-Bench, each min-max normalised across tracked models and averaged, scaled 0–100. Ingested from Epoch AI (CC-BY 4.0) and LiveBench (Apache-2.0). We do not run these evaluations.

⇩

100

89

85

78

77

76

74

71

68

67

64

61

60

50

42

27

💡

💡

💡

💡

claude-opus-4-5-20251101

💡

Gemini 3 Flash
Preview

💡

GPT-5.2

gemini-3-pro-preview

claude-3-7-sonnet-20250219

💡

💡

💡

💡

💡

claude-sonnet-4-5-20250929

claude-opus-4-1-20250805

💡

GPT-5 Mini

💡

Gemini 2.5 Pro

GPT-4o

💡 A lightbulb marks a model its provider publishes as a reasoning model

The Known Good

Image & Video Leaderboards

Elo from blind human preference votes, with 95% confidence intervals

Text to ImageImage EditingText to VideoImage to Video

Text to Image Arena

Elo rating from blind pairwise preference votes. Ingested from Arena, updated daily. Higher is better.

⇩

1385

1302

1280

1271

1261

1257

1250

1245

The Known Good

Speech Leaderboards

Text-to-speech quality from blind listening preference votes

Text to Speech Arena

Elo rating from blind A/B listening tests. Ingested from TTS Arena, updated daily. Higher is better.

⇩

The Known Good

Capability Indices

Capability-specific indices built from the relevant evaluations

MathematicsReasoningAgentic

Mathematics Index

Mathematics Index, built from Mock AIME 2024–25 and MATH Level 5, each min-max normalised across tracked models and averaged, scaled 0–100. Ingested from Epoch AI (CC-BY 4.0). We do not run these evaluations.

⇩

💡 A lightbulb marks a model its provider publishes as a reasoning model

The Known Good

Quality Breakdown

Individual evaluations behind the index, as measured by their publishers

GPQA DiamondMock AIME 2024–25MATH Level 5SWE-bench VerifiedLiveBenchHumanity's Last ExamTerminal-Bench

GPQA Diamond

Graduate-level science questions written to be Google-proof. Aggregated from Epoch AI's AI Benchmarking Hub (gpqa_diamond.csv, column 'mean_score'), used under CC-BY 4.0. The Known Good aggregates published results; it runs no evaluations. Higher is better.

⇩

💡 A lightbulb marks a model its provider publishes as a reasoning model

The Known Good

Arena Elo

Human preference rating from blind pairwise votes

OverallStyle controlled

Openness Index

Weight availability and licence permissiveness

Openness Index — distribution

How many tracked models sit in each openness band. Scores weight availability, licence permissiveness, and commercial-use restrictions, derived from models.dev and Epoch AI metadata, not from evaluations. Bars are bands, not models, so bar colour carries no creator here.

All 5 bands · 947 models classified⇩

Each bar is an openness band; the value is how many tracked models sit in itBands: 100 open weights, permissive · 90 open weights, permissive, no retrievable repository · 60 open weights, restricted use · 40 open weights, non-commercial · 5 proprietary

75

11

110

8

743

Open weights,
permissive

Open weights,
no repository

Open weights,
restricted use

Open weights,
non-commercial

Proprietary

The Known Good

Context Window

Maximum input tokens accepted

Context windowMax output tokens

Output Tokens

Maximum tokens a model will emit in a single response

Max output tokensContext window

Max Output Tokens

Provider-declared maximum completion length, from provider metadata via models.dev and OpenRouter. Higher allows longer single-pass generation.

⇩

The Known Good

Price and Cost

Published API pricing, blended and by token type

Blended priceInput priceOutput priceCache hit priceQuality per dollar

Blended Price

USD per 1M tokens at a 3:1 input:output blend. Live from OpenRouter, cross-checked against models.dev. Lower is better. Excludes 18 models published at no cost (free and promotional tiers).

⇩

The Known Good

We publish blended price rather than cost-per-task. Cost-per-task requires token counts from a proprietary evaluation suite we do not run. Blended price is computed from published rates and is fully reproducible — see pricing methodology.

Speed & Latency

Throughput and time-to-first-token, measured across hosting providers

Output speedLatency (TTFT)

Output Speed

Median output tokens per second across all tracked providers. Derived from OpenRouter endpoint telemetry. Higher is better.

⇩

The Known Good

API Provider Performance

How inference hosts compare when serving the same model

Output speedLatencyPriceContext served

Output Speed by Provider: gpt-oss-120b

Median tokens per second per host, serving the same model (gpt-oss-120b). Derived from OpenRouter endpoint telemetry. Higher is better.

⇩

501

367

352

342

231

196

177

171

146

114

The Known Good

Trends

How frontier model quality has moved over time, by creator

Frontier Model Quality, Over Time

Known Good Index v1.3 incorporates 7 evaluations: GPQA Diamond, Mock AIME 2024–25, MATH Level 5, SWE-bench Verified, LiveBench, Humanity's Last Exam, Terminal-Bench. Each step is a new release that raised that creator's best score. Scores ingested from Epoch AI (CC-BY) and LiveBench.

All 7 of 7 model creators⇩

OpenAIGoogleAnthropicAlibabaDeepSeekZ.aixAI

The Known Good

Reference data for AI models

Quality

Coding Index

Image & Video Leaderboards

Speech Leaderboards

Capability Indices

Quality Breakdown

Arena Elo

Openness Index

Context Window

Output Tokens

Price and Cost

Speed & Latency

API Provider Performance

Trends

Hugging Face Wasn't the Target. It Was in the Way.

Delaware County's Cyberattack and the Missing 911 Boundary

When the Router Exfiltrates Its Own Configuration

AI Worked Both Sides of the Ledger This Week. The Lesson Isn't "Patch Faster."

Pennsylvania Says Its Statewide 911 Disruption Wasn't a Cyberattack. The Alternative May Be Harder to Defend Against.

PamStealer Skips the Process Chains Defenders Watch. Not All of Them.

Your Agent Framework Is a Pile of API Keys on a Public IP

There's No Patch for FortiBleed. Public-Sector Networks Are Where That Hurts Most.

Four Failures, One Excuse: "Patch Faster" Was Never the Answer

Vibe Coding Isn't the Problem. Not Understanding the Stack Is.

I Handed Claude Code the Keys. Turns Out I'm Not the Only One Using Them.

The Circuit Nobody Could Find: How Exact-Match Searching Nearly Cost Me the Audit

The HTTP/2 Bomb Sat in Plain Sight for a Decade. An AI Just Had to Read the Code.

The Copilot Meter Didn't Raise the Price. It Showed You the Bill.

Designing AI for a Teen Discord Server Without Turning It Into a Surveillance Machine

WhatsApp Says No One Can Read Your Messages. A Federal Agent Spent 10 Months Disagreeing.

The Patch Queue Is the New Vulnerability

The Week the Toolchain Became the Kill Chain

ShinyHunters Didn't Breach 9,000 Schools. They Breached One Vendor. Your Institution Inherited the Rest.

The Floor Has Been Hit: Navigating the 2026 Systems Engineering Realignment