The Known Good Updated 29 Jul 2026
Google: Gemma 3 27B vs OpenAI: GPT-5.4 ·Google vs OpenAI

Google: Gemma 3 27B vs OpenAI: GPT-5.4 — head-to-head comparison

Google: Gemma 3 27B profile OpenAI: GPT-5.4 profile All comparisons
Google: Gemma 3 27B — Google
Quality
#46 of 62
46based on 4 of 7 evaluations
Known Good Index
Speed
#278 of 317
20.9
Output tokens per second
Price
#66 of 370
$0.17
Blended per 1M tokens
In $0.08 · Out $0.45
Cache hit price
#74 of 238
$0.040
per 1M tokens (−50%)
Context
#198 of 375
262k
tokens (~400 pages)
OpenAI: GPT-5.4 — OpenAI
Quality
#10 of 62
90based on 5 of 7 evaluations
Known Good Index
Speed
#90 of 317
68.0
Output tokens per second
Price
#328 of 370
$5.62
Blended per 1M tokens
In $2.50 · Out $15.00
Cache hit price
#192 of 238
$0.25
per 1M tokens (−90%)
Context
#17 of 375
1.05M
tokens (~1,600 pages)
Summary

On the Known Good Index, Google: Gemma 3 27B scores 46 and OpenAI: GPT-5.4 scores 90 — a 44-point lead for OpenAI: GPT-5.4. They share 2 published evaluations, with Google: Gemma 3 27B ahead on 0 and OpenAI: GPT-5.4 ahead on 2.

Blended pricing is $0.17 per 1M tokens for Google: Gemma 3 27B and $5.62 for OpenAI: GPT-5.4 — Google: Gemma 3 27B is about 32.6× cheaper.

Throughput is 21 tokens per second for Google: Gemma 3 27B and 68 for OpenAI: GPT-5.4, with time to first token at 1.1s and 2.5s. Google: Gemma 3 27B accepts 262k tokens of context against 1.05M.

Google: Gemma 3 27B is open weights under Open weights (restricted use); OpenAI: GPT-5.4 is proprietary. 7 tracked hosts serve one or both.

Generated from tracked data on 29 Jul 2026 · how these comparisons are computed →

At a glance
MetricGoogle: Gemma 3 27BOpenAI: GPT-5.4
Known Good Index 46 based on 4 of 7 evaluations 90 based on 5 of 7 evaluations
Blended $/1M $0.17 $5.62
Output tok/s 21 68
Context window 262k 1.05M
Providers 5 2
Jump to section

Quality

Known Good Index and the capability indices, side by side

Index scores

Each index is min-max normalised across tracked models and scaled 0–100 from published evaluations. We do not run these evaluations.

MetricGoogle: Gemma 3 27BOpenAI: GPT-5.4
Known Good Index 46 90
Coding Index — 89
Mathematics Index 47 98
Reasoning Index 40 88
Agentic Index — 89
Openness Index 60 5
The Known Good

Quality Breakdown

Every evaluation both models report a published score for

Evaluations both models report

Bar = Google: Gemma 3 27B · marker = OpenAI: GPT-5.4 · scores ingested from Epoch AI (CC-BY 4.0)

GPQA Diamond49 / 93
Mock AIME 2024–2520 / 98

Right-hand figures read Google: Gemma 3 27B / OpenAI: GPT-5.4. Google: Gemma 3 27B leads on 0 of 2, OpenAI: GPT-5.4 on 2.

The Known Good

Arena Elo

Human preference rating from blind pairwise votes

Arena ratings

Elo from blind pairwise votes, ingested from Arena. Higher is better.

MetricGoogle: Gemma 3 27BOpenAI: GPT-5.4
Text Arena — 1,452
Text Arena (style controlled) — 1,465
The Known Good

Price & Cost

Published API pricing by token type

Published pricing

USD per 1M tokens. Blended is a 3:1 input:output mix. Live from OpenRouter, cross-checked against models.dev. Lower is better.

MetricGoogle: Gemma 3 27BOpenAI: GPT-5.4
Input $/1M $0.08 $2.50
Output $/1M $0.45 $15.00
Blended $/1M $0.17 $5.62
Cache read $/1M $0.040 $0.250
The Known Good
We publish blended price rather than cost-per-task. Cost-per-task requires token counts from an evaluation suite we do not run. Blended price is computed from published rates and is fully reproducible — see the pricing methodology.

Context Window

Maximum input accepted and maximum output emitted

Context and output limits

Provider-declared limits, from metadata via models.dev and OpenRouter. Higher is better.

MetricGoogle: Gemma 3 27BOpenAI: GPT-5.4
Context window 262k 1.05M
Max output tokens 131k 128k
The Known Good

Speed & Latency

Median throughput and time to first token across hosting providers

Output speedTime to first token
Throughput and latency

Medians across all tracked providers, derived from OpenRouter endpoint telemetry.

MetricGoogle: Gemma 3 27BOpenAI: GPT-5.4
Output tokens/s 21 68
Time to first token 1.1s 2.5s
The Known Good

Providers

Which hosts serve each model, and at what price and speed

7 hosts
Full provider benchmarks →
Provider Google: Gemma 3 27B blendedGoogle: Gemma 3 27B tok/s OpenAI: GPT-5.4 blendedOpenAI: GPT-5.4 tok/s
Parasail $0.17 37 — —
Phala $0.23 20 — —
DeepInfra $0.10 18 — —
Novita $0.14 17 — —
Nebius $0.15 15 — —
Azure · first party — — $5.62 59
OpenAI · first party — — $2.81 59

Per-host pricing and telemetry from OpenRouter. A dash means that host does not serve that model.

The Known Good
Sources for this page
Pricing & providers — OpenRouter Metadata — models.dev Benchmarks — Epoch AI (CC-BY 4.0)Elo — Arena Full attribution →