Gemma 4 Runtime Selector
A runtime is the program that actually runs the model. There are eight sensible ways to run Gemma 4, and the right one depends on your device, your memory and what you want to do. Answer four questions and get a recommendation, a model size and the exact command.
If you are undecided, start with Ollama ·
Find your runtime
Tap an answer to each question. The recommendation updates as you go.
Memory means graphics card memory on a PC, or total memory on an Apple silicon Mac. The model size is this site's rule of thumb: the largest QAT build whose download is under about 70% of your memory. The recommendations are guidance, not guarantees; check the VRAM guide for your exact hardware.
All eight ways to run Gemma 4
| Runtime | Runs on | File format | Best for | Watch for | Guide |
|---|---|---|---|---|---|
| Ollama | Windows, macOS, Linux | GGUF | The quickest start; a local API included | Lists Gemma 4 builds as text and image only | Ollama |
| LM Studio | Windows, macOS, Linux | GGUF, MLX | People who prefer an app to a terminal | No EmbeddingGemma 2 entry in its catalogue on October 7 | LM Studio |
| llama.cpp | Windows, macOS, Linux | GGUF | Fine control and a small footprint | More to set up and manage yourself | Install guide |
| MLX | Apple silicon Macs | MLX | Python work and experiments on a Mac | Apple silicon only | Mac setup |
| vLLM | Linux servers | Safetensors, compressed tensors | Serving many users at once | No QAT build for the 26B A4B | Self-host |
| Transformers | Anywhere Python runs | Safetensors | Research, custom code and fine-tuning | Full-precision weights are large: 26.7 GB of memory for the 12B | How to use |
| LiteRT-LM | Android, iOS | .litertlm | Phones, and apps you build for them | E2B and E4B only; 32K context | Android |
| Hosted API | Any device with internet | None to download | No hardware, no upkeep | Your data leaves your machine; you pay per token | API |
The runtime does not change the model
Quality depends mainly on which size you run and how heavily it is compressed, not on the program running it.
They can share a machine
You can install more than one and switch freely. Each keeps its own copy of the model, so watch your disk space.
All free
The weights are Apache 2.0 and every local runtime here is free to download. Only the hosted route charges for use.
"Best for" is this site's guidance. Format availability is from Google's QAT collection and QAT announcement; Ollama input types from its tag list; memory from the model card. Google's announcement also names SGLang for serving and Transformers.js for the browser.
The first command for each
Examples use the 12B where the runtime supports it.
ollama run gemma4:12b-it-qat # download (7.2 GB) and chat
lms get google/gemma-4-12b-qat # download lms load google/gemma-4-12b-qat # load into memory lms server start # start the local API on port 1234
curl -LsSf https://llama.app/install.sh | sh llama serve -hf google/gemma-4-12B-it-qat-q4_0-gguf:Q4_0
pip install mlx mlx-lm mlx-vlm mlx_lm.generate --model mlx-community/gemma-4-e2b-it-4bit --prompt "Who are you?"
pip install vllm vllm serve google/gemma-4-12B-it-qat-w4a16-ct
from transformers import AutoProcessor, AutoModelForMultimodalLM
processor = AutoProcessor.from_pretrained("google/gemma-4-12B-it")
model = AutoModelForMultimodalLM.from_pretrained(
"google/gemma-4-12B-it", device_map="auto")litert-lm run --from-huggingface-repo=litert-community/gemma-4-E2B-it-litert-lm --prompt="Write me a poem"
llama.cpp and Transformers commands are from Google's model cards; MLX from Google's MLX guide; the LiteRT-LM command from its Hugging Face page. The vLLM line is the standard way to serve a repository and uses Google's vLLM-ready 12B build; check that repository's page for current options. Full walkthroughs are in the install guide, how to download and run locally.
When to switch runtime
More users
Moving from yourself to a team or the public? Go from Ollama to vLLM on a Linux server. See self-hosting.
Fine-tuning
Chat apps cannot train a model. Move to Transformers, or MLX on a Mac, with the full or unquantized QAT weights.
Audio
If your current app only takes text and images, switch to one that accepts audio, and use E2B, E4B or 12B.
Outgrowing your hardware
If the size you need will not fit, a hosted API runs the 26B A4B or 31B for cents per million tokens. See pricing.
This is this site's guidance. What each size can do is under capabilities and features; the small builds are explained on the QAT page.
And for EmbeddingGemma 2?
Google's new search model is a day old, so runtime support is still uneven. This is where it stood on October 7.
| Runtime | Handles | Notes |
|---|---|---|
| sentence-transformers (Python) | Text, code, images, audio, video | The route Google documents; the only one here covering every input type |
| Ollama | Text; text and images | ollama pull embeddinggemma-2, 378 MB to 1.3 GB |
| llama.cpp (GGUF) | Text only | ggml-org/embeddinggemma-2-GGUF, 310 MB at 8-bit |
| LiteRT | Text; text and vision; everything | Phone builds of 165, 388 and 485 MB |
| LM Studio | - | No catalogue entry found on October 7 |
From Google's EmbeddingGemma 2 developer guide, Ollama's library and the Hugging Face pages. Details and code are on the EmbeddingGemma 2 page.
What is a runtime?
The program that loads the model and runs it. The model is a file of numbers; the runtime turns your prompt into an answer using that file.
Which runtime should a beginner use?
Ollama if you are happy typing one command, LM Studio if you would rather click. On a phone, Google AI Edge Gallery.
Does the runtime affect answer quality?
Hardly. Quality comes mainly from the model size and how much it has been compressed. See the benchmarks.
What is the best runtime on a Mac?
Ollama or LM Studio for everyday use; MLX if you work in Python. The Mac page compares them.
What should I use to serve many users?
vLLM on a Linux server. Note there is no vLLM-ready QAT build of the 26B A4B.
Can I use more than one?
Yes. They install side by side. Each downloads its own copy of the model.
Do I have to run it myself at all?
No. You can chat with Gemma 4 online or use a hosted API. Running it yourself is for privacy, offline use and cost control.
Got your runtime?
Download a model and try your first prompt.