Gemma 4 Runtime Selector
Ollama, llama.cpp, vLLM, LM Studio, MLX, LiteRT or a hosted API - seven ways to run the same weights, and the right one depends entirely on your machine and what you're building. Answer four questions and get the runtime, the model size and the exact commands.
Questions that don't apply to your answers hide themselves - picking a phone removes the memory question, for instance.
All seven runtimes
The same weights run under all of these. What differs is setup effort, control and what they're optimised for.
| Runtime | Platforms | Format | Best for | Watch out for |
|---|---|---|---|---|
| Ollama | Mac · Win · Linux | GGUF |
Almost everyone. One command, serving included | Context defaults to 4,096 - set it explicitly |
| LM Studio | Mac · Win · Linux | GGUF · MLX |
No terminal; comparing quantisations visually | Raise context and GPU offload in load settings |
| llama.cpp | Mac · Win · Linux | GGUF |
Maximum control; leanest footprint | --jinja is mandatory or tool calls break |
| vLLM | Linux | Safetensors |
Production serving, many concurrent users | Needs transformers 5.5+; both tool flags required |
| Transformers | Any | Safetensors |
Writing Python against the model; fine-tuning | Version 5.5.0 or newer, or it won't load Gemma 4 |
| MLX | Apple silicon | MLX |
Native Mac Python and fine-tuning | GGUF currently measures faster for Gemma 4 |
| LiteRT-LM | Android · iOS | .litertlm |
Shipping a model inside a mobile app | E2B and E4B only; use the GPU backend |
When to switch
Signals that you've outgrown what you started with.
Ollama → vLLM
When more than a handful of people use it at once. vLLM's continuous batching and tensor parallelism are built for concurrency; Ollama is built for one person at a time.
Ollama → Transformers
When you need to touch the model rather than talk to it - custom generation loops, embeddings, or fine-tuning on your own data.
Ollama → llama.cpp
When you want a specific quantisation Ollama doesn't package, or the leanest possible footprint on constrained hardware.
Anything → LiteRT-LM
When the model needs to ship inside a mobile app rather than run on a machine you control.
Local → hosted API
When your hardware can't keep up, or you'd rather not run infrastructure. Gemma 4 costs nothing per token on Google's API.
Hosted → local
When data residency, offline operation or per-token cost starts to matter more than convenience. The weights are Apache 2.0, so nothing stops you.
Common questions
Does the runtime change the model's quality?
Not directly - the weights are the same. What does change quality is quantisation, which the runtime determines. A 4-bit build is measurably weaker than full precision, and Google's QAT checkpoints beat community quants at the same bit depth. So pick the runtime for convenience and the quantisation for quality.
Can I run more than one?
Yes, and it's common - Ollama for interactive use, vLLM for serving, Transformers for experiments. They'll each keep their own copy of the weights, so watch your disk. Ollama and LM Studio both default to port ranges that don't collide.
Which is fastest?
It depends on the shape of your workload. For a single stream, llama.cpp and Ollama are close, and MTP drafter checkpoints matter far more than the runtime - up to 3× on the 31B. For many concurrent users, vLLM wins decisively because continuous batching is what it's built for.
Why doesn't the selector ever recommend llama.cpp?
Because Ollama is built on it and handles the fiddly parts for you. llama.cpp is
the right call when you want a specific quantisation, the smallest possible install, or fine control over
flags - but as a default it's more setup for the same result. It's in the comparison table above with the
one flag that catches people: --jinja.
Is switching runtimes painful?
Usually not. Most expose an OpenAI-compatible endpoint, so changing
base_url is often the whole migration. Prompts and tool schemas carry over. The re-download
is the main cost, since formats differ - GGUF for Ollama and llama.cpp, Safetensors for vLLM and
Transformers.
What if I have less than 8 GB?
E2B at 4-bit needs about 2.9 GB and will run, though slowly on CPU alone. Below that you're better off with the free hosted API than fighting the hardware - or, if it's a phone rather than a PC, the mobile builds are purpose-made for exactly this.
Got your runtime?
Full installation steps, the settings that fail silently, and a troubleshooting table.