Gemma 4 12B
The only Gemma 4 model that takes text, images, audio and video and it fits on a 16 GB laptop. Not the biggest model in the family, but on several axes the most capable one.
11.95B parameters · 256K context · Apache 2.0 · 6.7 GB at 4-bit
Three reasons to pick this one
Encoder-free, and why that matters
Every other multimodal model in the family bolts encoders onto a language backbone. The 12B doesn't and the consequences run further than memory savings.
Vision: 550M → 35M
Instead of the 550M-parameter ViT the 26B and 31B carry, the 12B pushes 48×48×3 RGB patches through a single 35M-parameter projection with 2D positional embeddings roughly one fifteenth the size.
Audio: no encoder at all
Raw 16 kHz audio is sliced into 40 ms frames of 640 floats and projected linearly straight into the language model's input space. There is no intermediate audio representation and no encoder stage.
Where the 12B is the right call
Speech recognition
The family's best transcription, running locally. Meeting notes, voice memos and interview audio that never leaves the machine.
Diarization
Separating who said what, which is a stated strength. Combined with transcription, that covers most of what people actually want from meeting audio.
Video understanding
Google's own example processes five minutes of video at 1 FPS with the audio track alongside it frames and speech understood together rather than as separate pipelines.
Document and screenshot work
88.4 on InfographicVQA at full resolution. Raise the vision token budget for dense documents the default is deliberately conservative.
Coding
72.0 on LiveCodeBench v6 respectable for a model this size, and enough for scoped single-file work on a laptop.
Anything confidential
The strongest argument overall: multimodal understanding with no network involved. Audio, images and documents processed entirely on your own hardware.
Best in the family, by a clear margin
Removing the encoder didn't cost accuracy it improved it.
FLEURS transcription
Average word error rate across languages. Lower is better.
- Gemma 4 12B0.063
- Gemma 4 E4B0.075
- Gemma 4 E2B0.090
- Gemma 3n E4B0.085
- Gemma 4 31Bno audio
CoVoST speech translation
CorpusBLEU into English. Higher is better.
- Gemma 4 12B42.3
- Gemma 4 E4B38.2
- Gemma 4 E2B35.4
- Gemma 3n E4B34.7
- Gemma 4 31Bno audio
How it scores
Against its larger siblings, and against the previous generation's flagship. Thinking mode enabled, vision at maximum resolution.
| Benchmark | 12B | 26B A4B | 31B | Gemma 3 27B |
|---|---|---|---|---|
| MMLU Pro | 77.2 | 82.6 | 85.2 | 67.6 |
| AIME 2026 | 77.5 | 88.3 | 89.2 | 20.8 |
| GPQA Diamond | 78.8 | 82.3 | 84.3 | 42.4 |
| LiveCodeBench v6 | 72.0 | 77.1 | 80.0 | 29.1 |
| MMMU Pro | 69.1 | 73.8 | 76.9 | 49.7 |
| MATH-Vision | 79.7 | 82.4 | 85.6 | 46.0 |
| InfographicVQA | 88.4 | 89.3 | 92.0 | 70.6 |
| RULER @128K | 91.2 | 89.8 | 96.4 | 66.0 |
| Codeforces Elo | 1659 | 1718 | 2150 | 110 |
It beats the 26B on long context
RULER at 128K: the 12B scores 91.2 against the 26B A4B's 89.8. One of the few places a smaller model in this family comes out ahead, and useful if long documents are your workload.
It comfortably beats last generation's flagship
Against Gemma 3 27B on less than half the memory: 77.2 versus 67.6 on MMLU Pro, 77.5 versus 20.8 on AIME. If you're running Gemma 3 27B today, moving to the 12B is an upgrade and a cost reduction.
Full tables for every model size are on the benchmarks page, including the methodology caveats that apply to all of these numbers.
Download and run
Repository IDs are case-sensitive it's gemma-4-12B-it with a capital B, which
is a common copy-paste failure.
# Ollama 7.6 GB download ollama pull gemma4:12b ollama run gemma4:12b # Better 4-bit quality, slightly smaller ollama pull gemma4:12b-it-qat # Apple silicon ollama pull gemma4:12b-mlx
Ollama tags
| Tag | Size | Notes |
|---|---|---|
gemma4:12b | 7.6 GB | Default |
gemma4:12b-it-qat | 7.2 GB | Quantisation-aware 4-bit the one to use |
gemma4:12b-mlx | 7.7 GB | Apple silicon |
gemma4:12b-it-q8_0 | 13 GB | 8-bit, near-lossless |
gemma4:12b-mxfp8 | 13 GB | FP8 on modern NVIDIA |
gemma4:12b-nvfp4 | 7.7 GB | FP4 on Blackwell |
gemma4:12b-it-bf16 | 24 GB | Full precision |
Python
# Gemma 4 needs transformers 5.5.0 or newer older versions # don't recognise the architecture and loading will fail. pip install -U "transformers>=5.5.0" torch accelerate from transformers import pipeline import torch pipe = pipeline( "text-generation", model="google/gemma-4-12B-it", torch_dtype=torch.bfloat16, device_map="auto", ) out = pipe([{"role": "user", "content": "Summarise this recording."}], max_new_tokens=512) print(out[0]["generated_text"][-1]["content"]) # Audio and image inputs go in the same messages list as content # blocks no separate processor pipeline to set up.
Full install guidance, including the vLLM dependency conflict, is on the install page.
One pass, all modalities
The encoder-free design makes this materially simpler than fine-tuning a conventional multimodal model.
What you don't have to do
- ✅ No separate vision encoder to co-tune or freeze
- ✅ No decision about which encoder layers to unfreeze
- ✅ No separate audio encoder training stage
- ✅ One LoRA adapter covers text, vision and audio together
- ✅ The whole multimodal token loop updates in a single pass
Because all three modalities share the same weights, adapting one adapts the pathway they all use.
Ordinary fine-tuning realities
- ⚠️ QLoRA needs far less memory than a full fine-tune start there
- ⚠️ Unsloth and TRL/PEFT are the practical toolchains
- ⚠️ Safety behaviour can erode; re-evaluate before shipping
- ⚠️ Apache 2.0 means the result is yours to redistribute
- ⚠️ Budget for evaluation, not just training
What the 12B isn't
Four things worth knowing before you commit to it.
The larger models do beat it on reasoning
77.5 on AIME against the 31B's 89.2, and 72.0 on LiveCodeBench against 80.0. Real gaps on hard maths and code. If quality decides the outcome and you have the hardware, the 26B or 31B are better they just can't hear or watch anything.
No Multi-Token Prediction checkpoint
MTP variants shipped in April 2026 for E2B, E4B, 26B A4B and 31B before the 12B existed. So the 3.11× speculative-decoding speedup available to the 31B has no 12B equivalent published.
Not in Google's documented hosted list
The Gemini API documents gemma-4-31b-it and gemma-4-26b-a4b-it. Some
third-party sites list a hosted 12B, but Google's own docs don't so if you need 12B audio over an
API, plan on a third-party host or self-hosting.
Multi-step agent work is a family weakness
Gemma 4 measures poorly on agentic composites generally, and the 12B is smaller than the models that were tested. Keep tool-calling chains short and verify each step rather than trusting long autonomous runs.
Common questions
Should I use the 12B or the 26B A4B?
If you need audio or video, the 12B the 26B doesn't accept them. If you're serving text and images at volume, the 26B, since its MoE routing makes it far faster and it scores a few points higher.
For a single user on one consumer GPU, the 12B is usually the better fit simply because 6.7 GB fits where 14.4 GB doesn't.
Does it really run on a 16 GB laptop?
Yes, at 4-bit 6.7 GB of weights leaves headroom for the KV cache and your other applications. Use the official QAT build rather than a community quant, and don't set the context to 256K on a 16 GB machine or you'll push layers onto the CPU.
What exactly does "encoder-free" mean?
For audio it's literal: raw 16 kHz signal in 40 ms frames of 640 floats, projected straight into the model's input space with no encoder.
For vision there is still a learned projection 35M parameters over 48×48×3 patches so it's radically smaller rather than absent, against the 550M ViT the larger models carry.
How do I send audio or images to it?
As content blocks in the same messages list you'd use for text there's no separate processor pipeline to configure. Google's video example runs five minutes at 1 FPS with the audio track alongside. Note that vision has a token budget you can raise for dense documents; the default is conservative.
Why is its transcription better than the edge models'?
Partly size, and partly the architecture. E2B and E4B still use a 305M audio encoder, so the language model sees the encoder's compressed representation. The 12B projects the raw signal directly, so nothing is discarded before the model sees it 0.063 word error rate against E4B's 0.075.
Can I fine-tune it on my own data?
Yes, and it's simpler than most multimodal models because there are no separate encoders to co-tune a single LoRA pass updates the whole multimodal loop. Apache 2.0 means you can keep and redistribute the result. QLoRA via Unsloth or TRL is the practical route on consumer hardware.
Is it better than Gemma 3 27B?
Comfortably, on under half the memory 77.2 against 67.6 on MMLU Pro, and 77.5 against 20.8 on AIME. It also adds audio and video input, doubles the context to 256K, and moves to Apache 2.0 from the more restrictive Gemma Terms. If you're on Gemma 3 27B, this is the obvious migration.
Multimodal, local, and it fits.
6.7 GB at 4-bit, 256K of context, and the only Gemma 4 that can hear you.