Features & Technical Capabilities
Everything Gemma 4 offers, then how it is built: the feature list first, the engineering underneath it second, and the new EmbeddingGemma 2 model that extends the family into search.
From Google's model card and technical report ·
The essentials
E2B, E4B, 12B, 26B A4B and 31B.
On the three larger models; 128K on the edge models.
With a vocabulary of about 262,000 entries.
Open weights, commercial use allowed.
Released March 31, 2026 by Google DeepMind. Training data ends in January 2025. Sizes and files are on the models page.
What you get in every size
The interface is the same across the family, so a prompt or tool definition written for one size works on another.
Configurable thinking
Reasons step by step before answering, or skips it when you want speed.
Function calling
Built-in tokens for declaring tools, calling them and reading results.
Image understanding
Reads photos, documents and charts at their natural shape, with adjustable detail.
Video as frames
Up to 60 seconds at one frame per second, on every size.
Audio input
Speech recognition and translation on E2B, E4B and 12B, in clips of up to 30 seconds.
Long context
Whole documents and codebases in a single prompt.
System role
Native system instructions for steadier, more controllable behaviour.
Faster decoding
A matching drafter model for every size speeds up generation with no quality loss.
Official 4-bit builds
Quantisation-aware builds that keep quality close to full precision.
On-device
E2B and E4B run offline on phones through Google's LiteRT-LM.
Fine-tuning
Open weights you can adapt and redistribute.
Text output
It reads several formats but writes only text.
How to use each one - settings, code and limits - is on the capabilities page. Prompt advice is in prompting; coding in Gemma 4 coding; phones in mobile downloads and the Android page.
Which model has which feature
| Feature | 31B | 26B A4B | 12B | E4B | E2B |
|---|---|---|---|---|---|
| Text and images | ✓ | ✓ | ✓ | ✓ | ✓ |
| Video as frames | ✓ | ✓ | ✓ | ✓ | ✓ |
| Audio input | - | - | ✓ | ✓ | ✓ |
| Thinking mode | ✓ | ✓ | ✓ | ✓ | ✓ |
| Function calling | ✓ | ✓ | ✓ | ✓ | ✓ |
| MTP drafter | ✓ | ✓ | ✓ | ✓ | ✓ |
| Official QAT 4-bit build | ✓ | ✓ | ✓ | ✓ | ✓ |
| Phone build | - | - | - | ✓ | ✓ |
| Context window | 256K | 256K | 256K | 128K | 128K |
| Memory at 4-bit | 17.5 GB | 14.4 GB | 6.7 GB | 4.5 GB | 2.9 GB |
From Google's model card. Choose a size with the model comparison, the VRAM guide and the 12B guide; scores are on the benchmarks page.
How Gemma 4 handles long context
Gemma 4 is a decoder-only Transformer. Most of its layers only look at nearby text, which is cheap. Every so often a global layer looks at the whole context. That mix is what makes a 256K window affordable.
Local and global layers
Five local attention blocks for each global one; four to one on E2B, which gives the smallest model proportionally more global layers.
p-RoPE positions
Global layers encode position with p-RoPE at p = 0.25; local layers use standard RoPE. Base frequencies are 1M for global and 10k for local.
Keys reused as values
In global layers the stored keys double as values, which the report says cuts the global KV cache by 37.5%.
Cache sharing on edge models
E2B and E4B share the KV cache across layers, at ratios of 20/35 and 18/42, to fit in phone memory.
From the Gemma 4 technical report, which also specifies RMSNorm before and after each block and QK-norm. The report does not publish the local window size. What long context is like in use is covered under capabilities; raw figures in technical specs.
How it sees and hears
Three different designs across the family. The 12B is the unusual one: it has no separate vision or audio encoder at all.
| Model | Vision path | Audio path |
|---|---|---|
| 31B · 26B A4B | A 550M-parameter vision encoder | No audio input |
| 12B | No encoder: 48×48 image patches pass through a single 35M-parameter projection | No encoder: raw 40 ms chunks of 16 kHz audio are projected straight in |
| E4B · E2B | A 150M-parameter vision encoder | A 305M-parameter audio encoder, down from 680M in Gemma 3n |
Adjustable image detail
Each image costs 70, 140, 280, 560 or 1,120 tokens. The default is 280; use more for small text and dense documents.
Audio costs 25 tokens a second
Mono, 16 kHz, MP3 or WAV, up to 30 seconds a clip.
Order matters
Google recommends placing images before your text and audio after it.
Encoder sizes from the technical report; token budgets and limits from the model card.
Mixture of Experts and the drafter
Two ideas that make the larger models quicker than their size suggests.
Mixture of Experts - the 26B A4B
The model holds about 26B parameters but activates about 3.8B for each token. Less computation per token means faster output.
| Total parameters | 25.2B |
| Active per token | About 3.8B |
| Memory at 4-bit | 14.4 GB - all of it must be loaded |
Multi-Token Prediction drafter
A small 4-layer model reads the main model's cache and proposes several tokens at once. The main model checks them in one pass.
| Drafter size | About 78M (edge) to 0.5B (31B) |
| Width | 256 on E2B and E4B; 1,024 on 26B A4B and 31B |
| Output shortcut | Picks from 4,096 clusters instead of all 262k tokens |
From the technical report and Google's MTP overview, which describes the drafter as faster with no quality loss and notes the 26B A4B drafter may not help at batch size 1.
The report gives no speed-up figures, so none are quoted here. The drafter files are the -it-assistant downloads in the download hub.
Trained to run small
Ordinary quantisation rounds a finished model's weights. Quantisation-aware training lets the model learn to tolerate the rounding, so less quality is lost.
| Component | Precision | Result |
|---|---|---|
| Q4_0 builds, all sizes | 4-bit weights | 2.9 GB for E2B up to 17.5 GB for 31B |
| Mobile weights | A mix of 2-bit and 4-bit | About 1.1 GB for E2B and 2.5 GB for E4B |
| Edge vision encoder | 8-bit weights and activations | Half the size |
| Edge audio encoder | Mixed precision | 390 MB down to 87 MB, a 78% reduction |
From the technical report and model card. Which build to download is explained on the QAT page; install steps in how to download and the install guide.
EmbeddingGemma 2
The newest member of the family is not a chat model. It turns text, code, images, video and audio into vectors in one shared space, so a typed question can find a photo, a clip or a recording. It is built on the Gemma 4 architecture and shares its tokenizer and audio encoder.
Or 270M for text and code only.
Can be shortened to 512, 256 or 128.
Tokens, shared across all inputs.
Free for commercial use.
| Configuration | Parameters | Ollama download |
|---|---|---|
| Text and code only | 270M | 378 MB |
| Text and images | 440M | - |
| Text and audio | 570M | - |
| Everything | 740M | 1.3 GB |
| Get it from | Name |
|---|---|
| Hugging Face | google/embeddinggemma-2 |
| Ollama | ollama pull embeddinggemma-2 |
| Kaggle | Kaggle model page |
| Phones | LiteRT builds from 165 MB to 485 MB |
# pip install -U sentence-transformers transformers
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("google/embeddinggemma-2")
query = "What causes the northern lights?"
document = "The northern lights are caused by charged particles from the sun."
query_emb = model.encode(query, prompt_name="SearchQuery")
doc_emb = model.encode(document, prompt_name="Document")
print(model.similarity(query_emb, doc_emb))
From Google's model card. Use bfloat16 or float32; Google warns the model is not compatible with float16.
What it is for
Semantic search, retrieval for RAG, recommendations, clustering and classification - across text, code and media.
Pair it with Gemma 4
EmbeddingGemma 2 finds the right material; Gemma 4 reads it and writes the answer.
How it scores
78.68 on MTEB Code, up from 68.76 for version 1, and 61.36 on MTEB Multilingual v2, at 768 dimensions.
From Google's EmbeddingGemma 2 developer guide, model card and launch post. Vectors from version 1 and version 2 are not interchangeable, so re-embed your data if you upgrade. Task prompts, limits and more code are in the EmbeddingGemma 2 guide; serving it locally is on the API page.
Complete technical specification
| Specification | 31B | 26B A4B | 12B | E4B | E2B |
|---|---|---|---|---|---|
| Design | Dense | Mixture of Experts | Unified, encoder-free | Edge | Edge |
| Parameters | 30.7B | 25.2B | 11.95B | 4.5B effective (8B) | 2.3B effective (5.1B) |
| Active per token | All | About 3.8B | All | All | All |
| Layers | 60 | 30 | 48 | 42 | 35 |
| Context window | 256K | 256K | 256K | 128K | 128K |
| Local to global layers | 5 to 1 | 5 to 1 | 5 to 1 | 5 to 1 | 4 to 1 |
| KV cache sharing | - | - | - | 18/42 | 20/35 |
| Vision | 550M encoder | 550M encoder | 35M projection | 150M encoder | 150M encoder |
| Audio | - | - | Raw projection | 305M encoder | 305M encoder |
| Drafter size | 0.5B | 0.43B | 0.4B | About 78M | About 78M |
| Memory, full precision | 69.9 GB | 57.7 GB | 26.7 GB | 17.9 GB | 11.4 GB |
| Memory, 8-bit | 34.9 GB | 28.8 GB | 13.4 GB | 8.9 GB | 5.7 GB |
| Memory, 4-bit | 17.5 GB | 14.4 GB | 6.7 GB | 4.5 GB | 2.9 GB |
| Mobile build | - | - | - | 2.5 GB | 1.1 GB |
| Positions | p-RoPE (p = 0.25) on global layers, RoPE on local layers | ||||
| Normalisation | RMSNorm before and after each block, with QK-norm | ||||
| Vocabulary | About 262,000 entries | ||||
| Recommended sampling | Temperature 1.0, top-p 0.95, top-k 64 | ||||
| Training data cutoff | January 2025 | ||||
| Licence | Apache 2.0 | ||||
Parameters, layers, memory and sampling from Google's model card; attention, cache, encoder and normalisation details from the technical report; drafter sizes as listed on Hugging Face. The report does not publish hidden sizes for the main models. A plainer version is on the technical specs page.
How it was trained and evaluated
Infrastructure
Trained on Google's TPUs using JAX, with the Pathways approach and the GSPMD partitioner, according to the technical report.
Data
Web documents, code, images and audio, with a cutoff of January 2025. Because the cutoff is early, tests such as AIME 2026 postdate the training data.
Safety evaluation
Evaluated against policies on child safety, dangerous content, sexually explicit content, hate speech and harassment, under Google's Frontier Safety Framework.
What are Gemma 4's main features?
A thinking mode, built-in function calling, image and video understanding on every size, audio input on three sizes, a context window of up to 256K tokens, support for more than 140 languages, and an Apache 2.0 licence.
What is p-RoPE, and why p = 0.25?
A variant of rotary position encoding that Gemma 4 applies on its global attention layers; local layers keep standard RoPE. The technical report gives the setting but does not explain how 0.25 was chosen.
How does reusing keys as values save memory?
A model normally caches a key and a value for every token in every layer. In Gemma 4's global layers the key also serves as the value, so less is stored. The report puts the saving at 37.5% of the global cache.
Is the 12B really encoder-free?
For audio, yes: raw audio chunks are projected straight into the model. For images there is a single 35M-parameter projection in place of the 550M encoder the larger models use - far smaller, though not nothing.
Why does the 26B A4B need all its parameters in memory?
Any part of the model may be needed for the next token, so everything has to be loaded. What you save is computation per token, not memory.
How many layers do the models have?
Google's model card lists 60 for the 31B, 30 for the 26B A4B, 48 for the 12B, 42 for E4B and 35 for E2B.
How is EmbeddingGemma 2 different from Gemma 4?
Gemma 4 writes text. EmbeddingGemma 2 does not write anything: it converts content into vectors for search and comparison. They are designed to be used together. See the EmbeddingGemma 2 guide.
Where can I try these features?
In Google AI Studio, or see chat and the playground. To run it yourself, start with the runtime selector; Mac users see Mac setup; teams see self-hosting. Costs are on the pricing page.
See the features in action
Read how to use each one, or download a model and try them yourself.