Gemma 4 QAT
QAT builds are Google's official small versions of Gemma 4. They take about a quarter of the memory of the full model, and because the model was trained for the smaller format, Google says quality stays close to the original. For most people running Gemma 4 at home, these are the ones to download.
QAT stands for quantization-aware training ·
What QAT is, in plain terms
A model is billions of numbers. Storing each one less precisely makes the file smaller, a step called quantization. The question is when you do the shrinking.
Ordinary quantization
Train the model at full precision, then round every number down afterwards. Quick and common, but the model never had a chance to adjust, so some quality is lost.
Quantization-aware training
Simulate the rounding while the model is still being trained, so it learns numbers that survive being compressed. Only the model's maker can do this, and Google has.
From Google's QAT announcement of June 5, 2026 and the QAT model cards. Google has not published benchmark scores for the QAT builds alongside the standard ones, so "similar quality" is its description, not a measured figure on this page.
How big each QAT build is
Full precision against 4-bit, for every size.
| Model | Memory, full precision | Memory, 4-bit | Google's QAT file | Ollama download | Ollama name |
|---|---|---|---|---|---|
| Gemma 4 E2B | 11.4 GB | 2.9 GB | 3.35 GB | 4.3 GB | gemma4:e2b-it-qat |
| Gemma 4 E4B | 17.9 GB | 4.5 GB | 5.15 GB | 6.1 GB | gemma4:e4b-it-qat |
| Gemma 4 12B | 26.7 GB | 6.7 GB | 6.98 GB | 7.2 GB | gemma4:12b-it-qat |
| Gemma 4 26B A4B | 57.7 GB | 14.4 GB | 14.4 GB | 16 GB | gemma4:26b-a4b-it-qat |
| Gemma 4 31B | 69.9 GB | 17.5 GB | 17.7 GB | 19 GB | gemma4:31b-it-qat |
Memory figures are from Google's Gemma 4 model card and cover the model only; a long conversation needs more on top. File sizes are from Google's Hugging Face pages; Ollama sizes from Ollama's tag list. Ollama lists its QAT builds as taking text and images. To work out what your machine can hold, see the VRAM guide.
Which QAT build should I get?
Choose your memory and your app to get a suggested build and the command for it.
This is this site's rule of thumb: the largest build whose download is under about 70% of your memory, leaving room for the conversation. It is a starting point, not a guarantee. For a fuller comparison use the runtime selector and model comparison.
Four formats, and which sizes get each
| Format | What it is for | E2B | E4B | 12B | 26B A4B | 31B |
|---|---|---|---|---|---|---|
| GGUF, Q4_0 | Ready to run in Ollama, llama.cpp and LM Studio | ✓ | ✓ | ✓ | ✓ | ✓ |
| Unquantized QAT | Half-precision weights for your own conversion or research; drafter models included | ✓ | ✓ | ✓ | ✓ | ✓ |
| Compressed tensors, w4a16 | For serving with vLLM | ✓ | ✓ | ✓ | - | ✓ |
| Mobile, wNa8o8 | For phones, through LiteRT-LM | ✓ | ✓ | - | - | - |
Most people want GGUF
It is the format desktop apps read. If you use Ollama or LM Studio you never handle the file directly.
No vLLM build for the 26B A4B
Google's compressed-tensors list covers E2B, E4B, 12B and 31B only.
Fine-tuning or converting
Start from the unquantized QAT weights. They are also published for the small drafter models.
From Google's QAT model cards and its Gemma 4 QAT Q4_0 collection of 19 repositories. Repository names follow the pattern google/gemma-4-12B-it-qat-q4_0-gguf, …-qat-q4_0-unquantized and …-qat-w4a16-ct.
How to download a QAT build
Examples use the 12B. Swap in another size from the table above.
ollama run gemma4:12b-it-qat # 7.2 GB
# other sizes
ollama run gemma4:e2b-it-qat # 4.3 GB
ollama run gemma4:e4b-it-qat # 6.1 GB
ollama run gemma4:26b-a4b-it-qat # 16 GB
ollama run gemma4:31b-it-qat # 19 GB# install llama.cpp (macOS or Linux), then serve with a web page curl -LsSf https://llama.app/install.sh | sh llama serve -hf google/gemma-4-12B-it-qat-q4_0-gguf:Q4_0 # or chat in the terminal llama cli -hf google/gemma-4-12B-it-qat-q4_0-gguf:Q4_0
# pull Google's GGUF straight from Hugging Face into Ollama
ollama run hf.co/google/gemma-4-12B-it-qat-q4_0-gguf:Q4_0lms get google/gemma-4-12b-qat # download lms load google/gemma-4-12b-qat # load into memory
from transformers import AutoProcessor, AutoModelForMultimodalLM
# the unquantized QAT weights, for your own conversion or research
processor = AutoProcessor.from_pretrained("google/gemma-4-12B-it-qat-q4_0-unquantized")
model = AutoModelForMultimodalLM.from_pretrained(
"google/gemma-4-12B-it-qat-q4_0-unquantized", device_map="auto")llama.cpp, Hugging Face and Transformers commands are from Google's model cards; Ollama names from Ollama's library; LM Studio names from its catalogue.
- Ollama: add
-it-qatto the size. Full walkthrough on the Ollama page. - LM Studio: add
-qatto the model name, or search "qat" in the app. See LM Studio. - Mac: Ollama also offers
-mlxbuilds made for Apple silicon. Compare them on the Mac page. - Serving many users: the vLLM-ready builds are covered under self-hosting.
- Not sure where to start: the install guide and how to download go step by step.
The mobile QAT format
For E2B and E4B, Google trained a separate, more aggressive format aimed at phone hardware. Google says it brings E2B down to about 1 GB of memory, and under 1 GB for a text-only version.
Static activations
Scaling settings are worked out ahead of time, so the phone's processor does less work.
Channel-wise quantization
Data is arranged the way mobile accelerator chips expect, so they can compute on it directly.
Targeted 2-bit layers
The layers that produce tokens are compressed hardest; the layers that do the reasoning keep more precision.
Embedding and cache savings
Extra compression for the vocabulary table and the model's short-term memory.
From Google's QAT announcement. These builds run through LiteRT-LM and the Google AI Edge Gallery app: E2B is a 2.58 GB download and E4B 3.66 GB. Setup is on the Android page and under mobile downloads.
Does QAT make a difference you can notice?
One independent test compared Google's 12B QAT build with a popular community 4-bit build, on an Apple silicon Mac through Ollama.
| Build | File size | Speed, average | Task score |
|---|---|---|---|
| Google QAT, Q4_0 | 7.20 GB | 39.4 tokens/s | 4.5 of 6 |
| Community Q4_K_M, ordinary quantization | 7.60 GB | 30.9 tokens/s | 4.5 of 6 |
| Plain Q4_0, ordinary quantization | about 6.9 GB | 38.4 tokens/s* | - |
Faster, but not because of QAT
The plain Q4_0 build was nearly as fast. The tester's conclusion: the speed comes from the simpler Q4_0 file format, which Google happens to use.
No quality gap in a small test
Both builds scored the same on six tasks. A test that small cannot show subtle differences either way.
Still the sensible default
It is the official build, slightly smaller, at least as fast, and trained for its format. There is little reason to pick another 4-bit build.
From an independent write-up dated June 8, 2026. *Measured on a single 490-token run, where QAT reached 42.9 and Q4_K_M 33.2 tokens/s. This is one person's test on one machine, not a benchmark; the third card is this site's view. Official scores for the standard models are on the benchmarks page.
When QAT is not the answer
You have memory to spare
An 8-bit or full-precision build is the closest to the original. See model variants.
You need audio in Ollama
Ollama lists its QAT builds for text and images. Check capabilities for other routes.
Very long conversations
QAT shrinks the model, not the memory a long conversation uses. Budget for both.
EmbeddingGemma 2 and QAT
EmbeddingGemma 2, Google's new model for search across text, images, audio and video, uses the same technique. Google's on-device release says quantization-aware training compresses its weights to 4-bit and 8-bit, which is how the full model fits in about 567 MB of memory on a phone.
| Build | Handles | Download | Where |
|---|---|---|---|
| Phone, text | Text and code | 165 MB | litert-community/embeddinggemma-2-text-270m-litert-lm |
| Phone, text and vision | Adds images | 388 MB | litert-community/embeddinggemma-2-text-vision-440m-litert-lm |
| Phone, full | Text, images, audio, video | 485 MB | litert-community/embeddinggemma-2-740m-litert-lm |
| Desktop GGUF, 8-bit | Text only | 310 MB | ggml-org/embeddinggemma-2-GGUF |
| Ollama | Text, or text and images | 378 MB - 1.3 GB | ollama pull embeddinggemma-2 |
Built in, not a separate download
We found no separately labelled "QAT" repositories for EmbeddingGemma 2, unlike Gemma 4. The phone builds come pre-quantized.
Do not use float16
Google warns the model's values exceed what float16 can hold. Use bfloat16 or float32 when running the full weights.
A second way to save space
Shorter vectors - 512, 256 or 128 numbers instead of 768 - cut storage for everything you index. See pricing.
QAT statement and memory figures from Google's on-device EmbeddingGemma 2 post (about 191 MB text-only and 567 MB full, on a Pixel 11 Pro); the float16 warning from the model card. The desktop GGUF and Ollama builds are not described by their publishers as QAT. How shorter vectors affect quality is on the limitations page; task prompts are in the prompt guide.
What does QAT mean?
Quantization-aware training. The model is trained while simulating the compression it will later undergo, so it loses less quality when shrunk to 4 bits.
Is a QAT build worse than the full model?
Google says quality is similar to the full-precision version. It has not published side-by-side benchmark scores, so test it on your own tasks if small differences matter.
Which QAT build should I download?
The largest that fits your memory with room to spare. For many laptops and 12-16 GB graphics cards that is the 12B: ollama run gemma4:12b-it-qat. See the 12B page.
Is QAT faster?
Not by itself. In one independent test Google's build was faster than a Q4_K_M build, but an ordinary build in the same Q4_0 format was nearly as fast.
Is there a QAT build of the 31B?
Yes. GGUF, unquantized and vLLM-ready versions exist; the GGUF file is 17.7 GB.
Can I fine-tune a QAT build?
Use the unquantized QAT weights, which Google publishes for that purpose, rather than a GGUF file.
Is it free?
Yes. The QAT builds are Apache 2.0, like the rest of Gemma 4. Get them from the download hub.
Does EmbeddingGemma 2 have QAT builds?
Google says QAT is used to compress it to 4-bit and 8-bit weights for devices. There are no separately named QAT downloads; the phone builds are already quantized.
One command to a quarter-size Gemma 4
The QAT builds are free and run on ordinary laptops and graphics cards.