Gemma 4 Benchmarks
Every published score for all five Gemma 4 sizes, next to Gemma 3 27B for comparison: reasoning, coding, vision, long context, audio, human preference and on-device speed - plus where the models still fall short.
Scores from Google's model card and technical report · Arena rankings as of October 2, 2026
Reasoning, knowledge and coding
Instruction-tuned models with thinking mode on. Gemma 3 27B, which has no thinking mode, is shown for scale.
AIME 2026
MMLU Pro
GPQA Diamond
LiveCodeBench v6
Complete text table
| Benchmark | 31B | 26B A4B | 12B | E4B | E2B | Gemma 3 27B |
|---|---|---|---|---|---|---|
| MMLU Pro | 85.2 | 82.6 | 77.2 | 69.4 | 60.0 | 67.6 |
| AIME 2026 (no tools) | 89.2 | 88.3 | 77.5 | 42.5 | 37.5 | 20.8 |
| GPQA Diamond | 84.3 | 82.3 | 78.8 | 58.6 | 43.4 | 42.4 |
| LiveCodeBench v6 | 80.0 | 77.1 | 72.0 | 52.0 | 44.0 | 29.1 |
| Codeforces Elo | 2150 | 1718 | 1659 | 940 | 633 | 110 |
| BigBench Extra Hard | 74.4 | 64.8 | 53.0 | 33.1 | 21.9 | 19.3 |
| SciCode | 43.0 | 40.0 | 38.0 | 24.0 | 21.0 | 21.0 |
| IFEval (instruction following) | 98.9 | 98.5 | 97.2 | 96.7 | 94.6 | 90.4 |
| IFBench | 76.0 | 72.0 | 74.0 | 44.0 | 38.0 | 32.0 |
| Tau2 (agentic, average of 3) | 76.9 | 68.2 | 69.0 | 42.2 | 24.5 | 16.2 |
| MMMLU (multilingual) | 88.4 | 86.3 | 83.4 | 76.6 | 67.4 | 70.7 |
| HLE (no tools) | 19.5 | 8.7 | 5.2 | - | - | - |
| HLE with search | 26.5 | 17.2 | - | - | - | - |
Sources: Google's Gemma 4 model card and technical report. Percentages except Codeforces Elo. HLE is Humanity's Last Exam. For what these mean in practice see capabilities; for coding specifically see the coding section and Claude Code guide.
Images, documents and charts
Vision scores depend on how many tokens each image is given. These use the maximum budget of 1,120; the default is 280, and document-reading tasks in particular score lower there.
| Benchmark | 31B | 26B A4B | 12B | E4B | E2B | Gemma 3 27B |
|---|---|---|---|---|---|---|
| MMMU Pro | 76.9 | 73.8 | 69.1 | 52.6 | 44.2 | 49.7 |
| MATH-Vision | 85.6 | 82.4 | 79.7 | 59.5 | 52.4 | 46.0 |
| InfographicVQA | 92.0 | 89.3 | 88.4 | 70.0 | 63.9 | 70.6 |
| MedXpertQA MM | 61.3 | 58.1 | 48.7 | 28.7 | 23.5 | - |
| OmniDocBench 1.5 (lower is better) | 0.131 | 0.149 | 0.164 | 0.181 | 0.290 | 0.365 |
Sources: Google's model card and technical report. Image token budgets are explained in prompting.
Holding up over long inputs
A large context window only matters if the model can still find and reason over what is in it.
| Benchmark | Context | 31B | 26B A4B | 12B | E4B | E2B | Gemma 3 27B |
|---|---|---|---|---|---|---|---|
| RULER | 32K | 96.8 | 97.3 | 96.4 | 95.2 | 83.0 | 91.1 |
| RULER | 128K | 96.4 | 89.8 | 91.2 | 86.6 | 70.4 | 66.0 |
| LOFT retrieval | 128K | 79.5 | 66.3 | 66.4 | 58.5 | 50.5 | 8.6 |
| GraphWalks (F1) | under 128K | 82.3 | 72.6 | 71.0 | 50.9 | 4.1 | 32.8 |
| MRCR v2, 8 needles | 128K | 66.4 | 44.1 | 43.4 | 25.4 | 19.1 | 13.5 |
RULER, LOFT and GraphWalks from the technical report; MRCR v2 from the model card. The report also gives the 31B 52.9 on MTOB (English to Kalamang) at 128K and 54.3 at 256K. Note E2B's 4.1 on GraphWalks: its 128K window works for lookup, not for reasoning across documents. Context sizes per model are in technical specs.
Speech recognition and translation
Only E2B, E4B and 12B accept audio. They are compared here with the previous on-device generation, Gemma 3n.
| Benchmark | Gemma 4 12B | Gemma 4 E4B | Gemma 4 E2B | Gemma 3n E4B | Gemma 3n E2B |
|---|---|---|---|---|---|
| FLEURS transcription (word error rate, lower is better) | 0.063 | 0.075 | 0.090 | 0.085 | 0.108 |
| CoVoST speech translation (BLEU, higher is better) | - | 38.2 | 35.4 | 34.7 | 31.6 |
From the technical report, which also notes the audio encoder shrank from 680M parameters in Gemma 3n to 305M. Google's model card reports slightly different figures from a separate run, including 38.5 on CoVoST and 0.069 on FLEURS for the 12B (excluding Chinese). Audio clips are limited to 30 seconds - see limitations.
Arena rankings
Arena ranks models by how often people prefer their answers in blind comparisons. Scores barely move; ranks slide as newer models arrive.
| Model | Score, Oct 2026 | Rank, Oct 2026 | Rank, June 2026 | At launch, April 2026 |
|---|---|---|---|---|
| Gemma 4 31B | 1453 ± 7 | 77 | 43 | #3 among open models, #27 overall |
| Gemma 4 26B A4B | 1437 ± 7 | 100 | 61 | #6 among open models |
| Gemma 3 27B | 1365 ± 4 | 197 | 157 | Previous generation |
Still about 90 points ahead of Gemma 3
The gap to the previous generation has held since launch: 1453 against 1365.
No longer near the top of open models
The best open-weight models on the board now score around 1473 to 1480 - all of them far larger than 31B.
Strong for its size
At launch Arena rated it on par with Kimi-K2.5 and Qwen-3.5-397B, at roughly a tenth of the size.
October figures from Arena's text leaderboard, last updated October 2, 2026; June figures from the technical report. For head-to-head pages see the model comparison and Gemma 4 vs Qwen 3.5.
On-device speed
Quality scores say nothing about how fast a model feels. These are Google's measurements for Gemma 4 E2B on real hardware, GPU backend.
| Device | Reading speed | Writing speed | First word | Memory |
|---|---|---|---|---|
| Linux PC, NVIDIA RTX 4090 | 11,234 t/s | 143 t/s | 0.1 s | 913 MB |
| MacBook Pro M4 Max | 7,835 t/s | 160 t/s | 0.1 s | 1,623 MB |
| Samsung Galaxy S26 Ultra | 3,808 t/s | 52 t/s | 0.3 s | 676 MB |
| iPhone 17 Pro | 2,878 t/s | 56 t/s | 0.3 s | 1,450 MB |
| Raspberry Pi 5 (CPU) | 133 t/s | 8 t/s | 7.8 s | 1,546 MB |
Tokens per second for prefill ("reading") and decode ("writing"), from Google's LiteRT-LM overview. Phone details, including E4B and CPU-versus-GPU, are on the Android page and mobile downloads; for desktops see Mac setup, the VRAM guide and QAT builds.
How far it moved from Gemma 3
Gemma 3 27B against Gemma 4 31B. Part of the gap is thinking mode, which Gemma 3 does not have.
Competition maths
AIME 2026
Coding
LiveCodeBench v6
Agentic tool use
Tau2
Long-context retrieval
LOFT at 128K
Even the 12B beats Gemma 3 27B on every benchmark in the text table. More on the older family on the Gemma 3 page; feature differences are in features.
EmbeddingGemma 2 scores
The new embedding model built on Gemma 4 is measured on retrieval benchmarks, not chat ones. At the full 768 dimensions:
From Google's EmbeddingGemma 2 model card. Code retrieval rose from 68.76 on version 1. Full details in the EmbeddingGemma 2 guide.
Where the numbers are less flattering
Frontier-difficulty reasoning
On Humanity's Last Exam the 31B scores 19.5, the 26B A4B 8.7 and the 12B 5.2. With search grounding the 31B reaches 26.5. This is the widest gap to the largest hosted models.
Agents on the small models
Tau2 drops from 76.9 on the 31B to 42.2 on E4B and 24.5 on E2B. The edge models can call a tool; they are not dependable for long multi-step work.
Small models and multi-hop context
E2B scores 4.1 on GraphWalks and 21.9 on BigBench Extra Hard. A long window does not make a small model reason across it.
Standing among open models
Gemma 4 31B was #3 among open models on Arena at launch and sits at rank 77 overall six months on, as larger open models have passed it.
The full list of caveats is on the limitations page; responsible-use notes are under safety.
How to read these numbers
Thinking mode is on
Text and vision scores assume thinking is enabled; the long-context RULER results are reported without it. Turn thinking off and expect lower scores.
Vision uses the top resolution
Headline vision numbers use 1,120 tokens per image. The default is 280.
Most numbers are Google's own
They come from the model card and technical report. Arena is the independent check on this page.
Contamination varies
Training data ends in January 2025. AIME 2026 postdates that; older benchmarks carry more risk of having been seen.
Two metrics run backwards
For OmniDocBench and FLEURS word error rate, lower is better.
The weights changed mid-year
The July 2026 refresh altered tool calling, the chat template and vision defaults without a new version number.
Which Gemma 4 model scores highest?
The 31B, on almost every benchmark. The 26B A4B is close behind on most text tasks and edges it on RULER at 32K; the 12B beats the 26B A4B on Tau2 and IFBench. See model variants to choose.
How much better is Gemma 4 than Gemma 3?
Much. Against Gemma 3 27B, the 31B goes from 20.8 to 89.2 on AIME 2026, 29.1 to 80.0 on LiveCodeBench v6 and 67.6 to 85.2 on MMLU Pro, and sits about 90 points higher on Arena.
Is the 12B good enough?
For most everyday work, yes: 77.2 on MMLU Pro and 72.0 on LiveCodeBench v6, ahead of Gemma 3 27B throughout, in a model that fits a 16 GB laptop. See the 12B guide.
How good are E2B and E4B?
Good for their size, clearly weaker than the larger models: E4B scores 69.4 and E2B 60.0 on MMLU Pro. Use them where the device leaves no alternative - see the Android page.
Where does Gemma 4 rank on Arena now?
On October 2, 2026 the 31B was rank 77 with a score of 1453 and the 26B A4B rank 100 with 1437. At launch in April the 31B was #3 among open models.
Are these scores independent?
Mostly not. Apart from Arena, they are Google's own published results. Treat them as a guide and test on your own task.
Do quantised builds score the same?
These results are for the full-precision instruction-tuned models. Google says its QAT 4-bit builds stay close to that quality; ordinary 4-bit quantisation loses more. See QAT builds.
How does it compare on cost?
Benchmarks ignore price. Hosted Gemma 4 costs a few cents per million tokens and running it yourself has no per-token cost - see pricing and self-hosting.
Test it on your own task
Published scores are a starting point. Your prompts are the real benchmark.