Gemma 4 FAQ
Everything asked about Gemma 4, in one searchable place — from "which model should I use" to "why does my download 404". Each answer is short; the links go to the page with the full working.
Basics
What is Gemma 4?
Google DeepMind's open-weight model family, released 31 March 2026 and built from the same research as Gemini 3. Five sizes span a phone to a datacentre, all under Apache 2.0. Overview →
Is Gemma 4 free?
Yes. The weights are Apache 2.0 with no cost, and Google also serves them through the Gemini API without per-token charges. You pay only for compute if you self-host. API details →
How is Gemma different from Gemini?
Gemini is closed and hosted; Gemma is open weights you download and run yourself. They share research lineage — Gemma 4 derives from Gemini 3 — but Gemini remains stronger on absolute capability.
What sizes does Gemma 4 come in?
E2B (2.3B), E4B (4.5B), 12B, 26B A4B (a mixture of experts with ~3.8B active per token) and 31B dense. Compare them →
When was Gemma 4 released?
The core models on 31 March 2026, announced 2 April. The 12B followed on 3 June, and a weights refresh landed 15 July. Full timeline →
What is Gemma 4's knowledge cutoff?
January 2025. Anything later isn't in the weights, so use retrieval or tool calling for current information.
How many languages does it support?
Over 140, backed by a 262,144-entry SentencePiece vocabulary. The edge models also transcribe and translate speech.
Can I use Gemma 4 commercially?
Yes, without conditions. Apache 2.0 permits commercial use, modification and redistribution with no revenue thresholds or user caps. Licensing →
Do I need an account to use it?
Not to run it locally. Hugging Face asks you to accept terms once before downloading, and the hosted API needs a free key.
Is Gemma 4 better than Gemma 3?
By a wide margin. Gemma 3 27B scored 20.8 on AIME 2026 where Gemma 4 31B scores 89.2, and Codeforces Elo went from 110 to 2150. Gemma 3 page →
Is there a Gemma 3.5 or 3.6?
No. The line runs Gemma 1, 1.1, 2, 3, 3n, then 4. If you've seen "3.6" it's most likely Gemini 3.6 Flash or Alibaba's Qwen 3.6.
Who made Gemma 4?
Google DeepMind. It's governed under Google's Frontier Safety Framework and trained on TPUv5p and TPUv6e hardware.
What does the E in E2B and E4B mean?
Effective parameters — E2B has 2.3B raw parameters but is engineered to behave like a 2B-class model on device, with mixed-precision weights bringing its text-only footprint to roughly 0.8 GB.
What does A4B mean in 26B A4B?
Approximately 4 billion active parameters per token. The full 26B must be resident in memory, but only about 3.8B are read for each token.
Models & choosing
Which Gemma 4 model should I use?
26B A4B for serving, 31B when quality decides the outcome, 12B for one consumer GPU or anything with audio, and E2B or E4B for phones. Runtime selector →
Is the 31B worth it over the 26B?
Often not. The accuracy gap is two to three points while the 31B runs at roughly a quarter of the speed at concurrency 1 and needs four times the KV cache. VRAM comparison →
Which model is best for coding?
The 31B scores 80.0 on LiveCodeBench v6, but that measures self-contained problems. On real repository patching it drops to about 41.6%, so scope work to single files. Limitations →
Which model handles audio?
The 12B, E4B and E2B. The 31B and 26B A4B accept text and images only — so the largest model is not the most multimodal. Capabilities →
What's special about the 12B?
It's encoder-free: image patches go through a 35M projection instead of a 550M vision encoder, and raw audio projects straight into the token space with no encoder at all. 12B page →
What is a mixture of experts?
The model routes each token to a subset of its parameters. The 26B holds all 26B in memory but reads only ~3.8B per token, which is why it's about 4.4× faster than the dense 31B.
Does MoE mean it uses less memory?
No — all 26B must be resident, at 57.7 GB in BF16. What you save is memory bandwidth, which is what makes it fast.
Which model for a 16 GB laptop?
The 12B at 4-bit, which needs 6.7 GB of weights. Older guides recommending the 26B for 16 GB predate the 12B's June 2026 release. Mac guide →
Which model for a phone?
E2B, with a mobile build around 1.1 GB. E4B at 2.5 GB works on higher-end devices. Mobile guide →
Is there a model smaller than E2B?
Not in Gemma 4. If you need something genuinely tiny, Gemma 3 270M remains the smallest Gemma — though it's on the older licence. Gemma 3 →
What are the specialised Gemma models?
TranslateGemma for translation, MedGemma for medical, EmbeddingGemma for retrieval, ShieldGemma for safety classification, and DiffusionGemma for parallel text generation. Variants →
What is DiffusionGemma?
A 26B MoE released June 2026 that generates 256 tokens per forward pass in parallel rather than autoregressively — up to 4× faster output, and good at code infilling.
Should I use the base or instruction-tuned model?
Instruction-tuned (-it) unless you're doing your own post-training. Base models continue text rather than answering questions. Variants →
Downloading
Where do I download Gemma 4?
Hugging Face is canonical, with mirrors on Kaggle, Ollama and LM Studio. Google publishes everything under the google/ namespace. All downloads →
What are the exact repository names?
google/gemma-4-31B-it, gemma-4-26B-A4B-it, gemma-4-12B-it, gemma-4-E4B-it and gemma-4-E2B-it. Full list →
Why does my download 404?
Almost certainly the capital B. Gemma 4 uses gemma-4-12B-it; Gemma 3 used lowercase, which is why the mistake is so easy.
What is the -it-assistant variant?
A ~0.4B Multi-Token Prediction drafter for speculative decoding — not a chat model. Load it alongside the main checkpoint for up to 3× faster decoding. Variants explained →
Should I use the QAT GGUF builds?
Yes, where one exists. Quantisation-aware training exposes the model to rounding error during training, so it loses less quality than post-hoc community quants at the same bit depth.
What's the unquantized QAT variant for?
It holds QAT-trained weights at full precision, so you can apply your own quantisation scheme and still get the benefit. If you just want 4-bit, take the GGUF.
How big are the downloads?
From 2.9 GB for E2B at 4-bit to about 70 GB for the 31B at BF16. Use hf download rather than git clone — git-lfs silently stores a second copy. Sizes →
Do I need a Hugging Face token?
For gated repositories, yes. Accept the licence on the model page, then run hf auth login.
How do I make downloads faster?
Install hf_transfer and set HF_HUB_ENABLE_HF_TRANSFER=1. It parallelises chunks and will usually saturate a gigabit connection.
Are community re-uploads safe?
Usually fine, but occasionally stale, mislabelled, or quantised more aggressively than advertised. Prefer official google/ repositories where they exist.
What's the difference between GGUF and Safetensors?
Safetensors is the reference full-precision format for Transformers and vLLM. GGUF is the quantised format for llama.cpp, Ollama and LM Studio. Formats →
Do I need to re-download after the July update?
Yes, if you cached weights before mid-July 2026. The refresh changed the chat template and fixed tool calling without changing the version number.
Installing
What's the easiest way to install Gemma 4?
Ollama. One installer, one command, and it exposes an OpenAI-compatible API for free. Install guide →
Which Ollama version do I need?
0.22.0 or newer. The 0.21.x builds have known correctness problems with tool calling.
Do I need to install CUDA?
Not for Ollama or LM Studio — they bundle what they need, and you only need a current NVIDIA driver. vLLM and PyTorch want a CUDA-enabled build.
Can I install without a terminal?
Yes. LM Studio is a normal desktop application with a model browser that estimates memory and warns you if a model won't fit.
Does it work on Windows?
Yes, natively. Ollama and LM Studio both ship Windows installers, and Ollama runs as a background service on localhost:11434. WSL is only needed for vLLM.
What Python version do I need?
3.10 or newer for vLLM. And transformers >= 5.5.0 for Gemma 4 specifically — older versions don't recognise the architecture.
Why does my model forget the conversation?
Ollama defaults to a 4,096-token context regardless of what the model supports. Set OLLAMA_CONTEXT_LENGTH=32768 or use a Modelfile. Install guide →
How do I check the GPU is being used?
Run ollama ps — the GPU column should name your device. If it's empty or shows CPU, add PARAMETER num_gpu 99 and check your driver.
Is Docker supported?
Yes. vLLM publishes an official image. Remember --ipc=host and mount the Hugging Face cache, or every restart re-downloads the weights. Self-hosting →
Can I run Gemma 4 offline?
Completely, once the weights are downloaded. Inference is local — you can verify by disconnecting the network.
How do I uninstall it?
Remove models with ollama rm, then uninstall the application normally. Hugging Face downloads live in ~/.cache/huggingface and won't be removed automatically.
What is the --jinja flag?
It applies the chat template on llama.cpp. Without it tool calls never format correctly and you'll see raw <unused24> tokens in the output.
Hardware & VRAM
How much VRAM does Gemma 4 need?
At 4-bit: 2.9 GB for E2B, 4.5 GB for E4B, 6.7 GB for the 12B, 14.4 GB for the 26B A4B and 17.5 GB for the 31B — plus KV cache. VRAM calculator →
Can I run Gemma 4 without a GPU?
Yes, but slowly above E4B — expect single-digit tokens per second on a 12B. The edge models are genuinely usable on CPU.
What is the KV cache and why does it matter?
It stores attention keys and values for every token in context. It scales with context length and concurrency, and on the 31B at full context it grows larger than the weights. How it works →
How much KV cache does the 31B need?
20.78 GiB at 256K context, 3.28 GiB at 32K. About 96% of it comes from the ten full-attention layers; the fifty sliding layers cap at a 1,024-token window.
Why does the 26B need so much less context memory?
It has 30 layers to the 31B's 60, five global layers to ten, and two global KV heads to four. Those compound to roughly 4× less KV cache. The numbers →
Can I run the 31B on a 24 GB card?
The 4-bit weights fit at 20.5 GiB, but that leaves about 2 GiB for cache — around 8K tokens. Workable for short prompts, not long documents.
Why doesn't 4-bit quantisation solve my memory problem?
Because it only compresses weights. The KV cache stays at fp16, so at long context you can halve the weights and barely move the total. The 4-bit trap →
How much RAM do I need on a Mac?
16 GB runs the 12B comfortably. 32 GB handles the 26B, and 48 GB or more suits the 31B. Unified memory is shared with everything else you have open. Mac guide →
Does more GPU memory make it faster?
Not directly — it lets you hold more context and more concurrent requests. Speed comes from bandwidth, the model's architecture, and MTP drafters.
Is multi-GPU worth it?
For capacity and latency, yes. For throughput it scales poorly — eight H100s give about 1.75× a single card. More replicas of a smaller model usually beats one large one. Self-hosting →
What GPU should I buy?
A 24 GB card runs the 12B comfortably and the 31B at 4-bit with short context. 48 GB or more if you want long context on a large model.
Does AMD work?
Yes, with ROCm 6.x on PATH. Check rocminfo before troubleshooting anything else.
Why do Google's memory figures differ from raw weight arithmetic?
Google's published table is an inference requirement that includes working memory and encoders. Raw parameters × 2 bytes gives a smaller number. Both are correct for different definitions. Both figures →
Performance
How fast is Gemma 4?
On an H100 at concurrency 1: about 40 tokens/sec for the dense 31B and 177 for the 26B A4B. With MTP drafters those rise to 125 and 264. Throughput →
What is Multi-Token Prediction?
A small drafter model proposes several tokens ahead which the full model verifies in parallel. It gives up to 3.11× faster decoding on the 31B at no quality cost. How it works →
Why is the MoE faster than the dense model?
Single-stream decoding is limited by how fast weights can be read from memory. The MoE reads about 3.8B parameters per token instead of 31B.
What is FlashAttention 4?
An attention implementation shipped for NVIDIA Hopper GPUs in July 2026: prefill throughput up 25–70% and time-to-first-token down as much as 31%.
Why does my model get slower during long sessions?
Usually a missing prompt cache — some runtimes reprocess the whole prefix every turn. Clear context between tasks and prefer the MoE, which prefills faster.
Does thinking mode slow things down?
Yes, substantially — reasoning traces are real generated tokens. Turn it down for classification and extraction, keep it high for maths and code.
What's a realistic time to first token?
279 ms for the 31B on one H100 at concurrency 1, dropping to 85 ms across eight GPUs. It stays stable to about concurrency 8, then climbs.
How fast is it on a phone?
E2B on a Galaxy S26 Ultra GPU reaches first token in 0.3 seconds and decodes at 52 tokens/sec — faster to first token than the 31B on an H100. Mobile →
Does quantisation affect speed?
Yes, usually favourably — smaller weights mean less memory traffic. FP8 roughly halves weight bandwidth with near-identical quality.
What's the fastest configuration overall?
26B A4B with MTP checkpoints, FlashAttention 4 on a Hopper GPU, thinking minimised and the smallest vision token budget your task tolerates.
Is MLX faster than GGUF on a Mac?
For Gemma 4, no — GGUF measures about 6–9% faster, attributed to llama.cpp's better K-quant scheme. That may change. Mac detail →
Capabilities
What is thinking mode?
A configurable setting where the model emits reasoning traces before answering. Every published accuracy figure assumes it's enabled.
Does Gemma 4 support function calling?
Yes, natively, with constrained decoding that keeps output schema-valid. Reliability improved substantially in the July 2026 refresh.
Can Gemma 4 generate images?
No. Every model in the family outputs text only. Images and audio are inputs on the models that support them.
How long is the context window?
256K tokens on the 12B, 26B A4B and 31B; 128K on E2B and E4B.
Does long context actually work?
On the large models, yes for retrieval — the 31B's RULER score barely moves from 32K to 128K. On E2B, multi-hop reasoning across long context fails badly. Long context data →
Can it read documents and screenshots?
Yes, but raise the vision token budget. The default is 280; at the 1120 maximum, InfographicVQA rises from 82.8 to 92.0 on the 31B.
What are Agent Skills?
Callable capabilities the model composes into workflows — querying knowledge sources, transforming content, handing off to other models. They run on-device.
Does it support structured JSON output?
Yes, constrained to a schema. That's a stronger guarantee than prompting a model to "respond in JSON".
Can Gemma 4 transcribe audio?
The 12B and edge models can. The 12B posts the family's best result at 0.063 average word error rate on FLEURS.
Does it support system prompts?
Yes, natively via system_instruction. Set a role and output format once for the whole conversation.
Can I fine-tune Gemma 4?
Yes, and Apache 2.0 lets you keep and redistribute the result. LoRA and QLoRA work through TRL, PEFT and Unsloth; full fine-tuning via JAX or Keras.
API & hosted
Is there a Gemma 4 API?
Not a separate one. Google serves Gemma 4 through the Gemini API using the same SDK and endpoint, with a different model ID. API guide →
How much does the API cost?
Google charges nothing per token for Gemma models, and the free tier needs no billing account. What limits you is rate, not spend.
How do I get an API key?
From Google AI Studio at aistudio.google.com/apikey. New keys are issued as auth keys, restricted to the Gemini API by default. Step by step →
Why did my API key stop working?
Most likely the auth-key migration. Unrestricted legacy keys are already rejected, and all standard keys stop working in September 2026. Migration →
Which models are available on the hosted API?
Google documents gemma-4-31b-it and gemma-4-26b-a4b-it. The 12B — the only large model with audio — isn't in that list.
Can I call the API from my front-end?
Not with the key in it. Anything shipped to a browser can be extracted. Put a thin server in between that holds the key and rate-limits your users. Security →
What are third-party options?
OpenRouter, Fireworks, DeepInfra, Together and others serve the same weights behind OpenAI-compatible endpoints. OpenRouter lists the 31B at $0.08 in / $0.35 out per million.
Is my data used for training?
On the free tier, Google may use prompts to improve its products. Paid tiers don't. Self-hosting avoids the question entirely.
Can I use the OpenAI SDK?
With third-party hosts and your own vLLM or Ollama server, yes — just change base_url. For Google's API use the google-genai SDK.
Is self-hosting cheaper than the API?
Usually not. Even at 100% utilisation the cheapest rented H100 costs about 2.8× what OpenRouter charges, and Google's API is free of token charges. The arithmetic →
Licensing
What licence is Gemma 4 under?
Apache 2.0, across the entire family. No use restrictions in the licence, no revenue thresholds, no downstream obligations.
Can I redistribute a fine-tuned Gemma 4?
Yes, freely. Apache 2.0 imposes no conditions on derivatives beyond the standard notice requirements.
Is Gemma 3 also Apache 2.0?
No. Gemma 1, 1.1, 2, 3 and 3n remain under the Gemma Terms of Use, whose restrictions propagate to anyone you redistribute to. The change was not retroactive. Licence divide →
Which specialised variants are Apache 2.0?
Only Gemma 4 itself and DiffusionGemma. TranslateGemma, MedGemma, EmbeddingGemma, ShieldGemma, PaliGemma and CodeGemma stay on the older terms.
Does Google restrict how I use Gemma 4?
The licence itself imposes no use restrictions, unlike the Gemma Terms. Google still publishes prohibited-use and intended-use guidance alongside it. Ethics page →
Do I need to credit Google?
Apache 2.0 requires you to retain copyright and licence notices in redistributed copies. It doesn't require user-facing attribution.
Can I use Gemma 4 in a closed-source product?
Yes. Apache 2.0 permits proprietary derivatives with no obligation to open your own code.
Is ShieldGemma covered by Apache 2.0?
No — it's still on the Gemma Terms. So adding Google's recommended safety layer to an Apache 2.0 deployment reintroduces downstream obligations.
Limitations & safety
What is Gemma 4 worst at?
Agentic, multi-step tool use. Independent testing ranks the 31B #129 of 134 on agentic composites despite strong knowledge and reasoning scores. Limitations →
Does Gemma 4 hallucinate?
Google's model card states plainly that it "may generate incorrect or outdated factual statements". The January 2025 cutoff widens the gap it fills with plausible text.
Is it safe to deploy publicly?
With your own safeguards, for many applications yes. Open weights mean no provider filters inputs or outputs — that layer is yours to add. Safety →
Does fine-tuning break safety training?
It can degrade it, particularly on unfiltered data. Treat a fine-tune as a different model for safety purposes and re-evaluate.
How does it compare to frontier closed models?
Behind on absolute capability. HLE is the clearest illustration — 19.5 for the 31B. The argument for Gemma is that it's the strongest thing you can download and own. vs Qwen3.8-Max →
Is it good at maths?
Very — 89.2 on AIME 2026 for the 31B, against 20.8 for Gemma 3 27B. That's with thinking mode enabled.
Can it handle sarcasm and nuance?
Google's model card says it "might struggle to grasp subtle nuances, sarcasm, or figurative language" — relevant for sentiment work and moderation.
Does it have a content filter?
Not in the weights you download. Safety behaviour is trained in, but there's no server-side moderation. ShieldGemma exists for classification.
Are the published benchmarks trustworthy?
They're self-reported by Google and generally reliable, but assume thinking mode on and vision at maximum resolution. Independent agentic and SWE results are less flattering. Provenance →
Should I use it for medical or legal work?
Only with a qualified human in the loop. Fluent confident output in a high-stakes domain reads as authoritative to exactly the users least able to check it.
Mobile & Mac
Can I run Gemma 4 on my phone?
Yes. E2B has a mobile build around 1.1 GB and runs fully offline. Google AI Edge Gallery on the Play Store is the easiest route. Mobile guide →
What are the Android requirements?
Android 12 or newer, roughly 4 GB of RAM for a comfortable E2B experience, and about 6 GB of free storage.
Does it work on iPhone?
Yes — the same LiteRT builds run on iOS 17 and up. An iPhone 17 Pro reaches 25.1 tokens/sec on E4B via GPU.
Should I use CPU or GPU on mobile?
GPU, decisively. On the same phone and model it cut time-to-first-token from 1.8 s to 0.3 s and memory from 1,733 MB to 676 MB.
Will it drain my battery?
Sustained generation is demanding and warms the device. Idle cost is zero since the model only runs when asked.
How do I raise the GPU memory limit on a Mac?
sudo sysctl iogpu.wired_limit_mb=28000. It resets on reboot by design, and overshooting causes a kernel-level stall rather than a clean error. Mac guide →
Is a MacBook Air good enough?
For occasional use, yes. It's fanless, so it throttles during long sessions. Memory capacity matters far more than the chip tier.
Do Intel Macs work?
Barely, and only for the smallest models. Everything useful here assumes Apple silicon and unified memory.
Troubleshooting
"Transformers does not recognize this architecture"
Your transformers version is older than 5.5.0. Run pip install -U "transformers>=5.5.0".
Tool calls never execute
On vLLM you need both --enable-auto-tool-choice and a matching --tool-call-parser. On llama.cpp you need --jinja.
Raw <unused24> tokens in the output
The chat template isn't being applied. Add --jinja on llama.cpp, and update your runtime.
Generation is very slow
Check ollama ps first — if the GPU column is empty you're on CPU. Then check your driver with nvidia-smi.
Out of memory under load
Lower --max-model-len, reduce concurrency, or drop to a 4-bit QAT build. Context costs memory linearly. VRAM calculator →
pip reports a vllm/transformers conflict
vLLM 0.19.0 pins transformers below 5, but Gemma 4 needs 5.5.0+. Use a nightly wheel or build from source, or accept the pip warning.
403 or "gated repo" when downloading
Accept the licence on the model page, then authenticate with hf auth login.
Nothing responds on localhost:11434
The service isn't running. Start it with ollama serve and test with curl http://localhost:11434.
The whole machine stalls when loading on a Mac
A wired-memory shortfall — a kernel-level fault rather than a clean error. Lower iogpu.wired_limit_mb or use a smaller model.
First response is slow every time
The model is being unloaded between requests. Set OLLAMA_KEEP_ALIVE=-1 to keep it resident.
Results don't match the published benchmarks
Check thinking mode is on and the vision token budget is raised — the defaults are lower than the benchmark configuration. Also re-pull post-July weights.
The model gives short or truncated answers
Usually the context limit. Ollama defaults to 4,096 tokens; raise it with OLLAMA_CONTEXT_LENGTH or a Modelfile.
Still stuck?
The troubleshooting tables and the calculators answer most of what isn't here.