How to Install Gemma 4 & EmbeddingGemma 2
Set up Google's open chat model and its new search model on Windows, macOS or Linux. Most people are running in ten minutes; this page also covers the settings and checks that make it run well.
Free · Apache 2.0 · runs offline after the download ·
Install Gemma 4 with Ollama
Ollama is the simplest route on every system: one installer, then one command. Choose your system.
Download and run the installer
Get it from ollama.com/download. Ollama then runs in the background.
Open PowerShell or Command Prompt
Search for either in the Start menu.
Run the model
The first run downloads 7.2 GB, then opens a chat.
# After running the installer from ollama.com/download, # open PowerShell or Command Prompt: ollama --version ollama run gemma4:12b-it-qat
No WSL needed. With an NVIDIA card, update its driver first if the model runs slowly.
Download the app
From ollama.com/download. Drag it to Applications and open it.
Open Terminal
Find it with Spotlight.
Run the model
The first run downloads 7.2 GB, then opens a chat.
# After installing the app from ollama.com/download, # open Terminal: ollama --version ollama run gemma4:12b-it-qat
Apple silicon Macs share memory between the processor and graphics, so there is nothing extra to set up. More in Mac setup.
curl -fsSL https://ollama.com/install.sh | sh ollama --version ollama run gemma4:12b-it-qat
The script installs Ollama and starts it as a service. For an NVIDIA card, install the vendor driver first.
# Start the Ollama server in a container docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama # With an NVIDIA card, add: --gpus=all # Run Gemma 4 inside it docker exec -it ollama ollama run gemma4:12b-it-qat
Models are kept in the ollama volume, so they survive restarts. Team deployments are covered under self-hosting.
The examples use the 12B, which suits a 16 GB machine. For other sizes swap the name: gemma4:e4b-it-qat (8 GB), gemma4:26b-a4b-it-qat or gemma4:31b-it-qat (24 GB graphics card).
Help choosing is in how to download and on the models page.
Five ways to install, in order of effort
| Method | Effort | Choose it when | Section |
|---|---|---|---|
| Ollama | One installer, one command | You want it working, with a local API included | Quick install |
| LM Studio | An app; no typing | You prefer buttons to a terminal | LM Studio |
| Phone app | One app, one download | You want it on an iPhone or Android phone | Mobile downloads |
| Python | A few commands | You are writing code against the model | Python |
| llama.cpp | Build from source | You want fine control over a local server | Python and llama.cpp |
Not sure? The runtime selector asks a few questions and picks one. All of these are free.
Install with LM Studio
A desktop app with a model browser and a chat window. Nothing to type.
Install LM Studio
Download it from lmstudio.ai for Windows, macOS or Linux.
Find Gemma 4
Open the model search with Ctrl + Shift + M (Cmd on Mac) and type "Gemma".
Download a size that fits
The app shows each download's size: 5.90 GB for E4B, 7.40 GB for the 12B, 19.00 GB for the 31B.
Load it and check two settings
Press Ctrl / Cmd + L to load the model. In its settings, raise the context length and make sure GPU offload is turned up.
Chat
Start typing. To use it from your own code, start the local server from the Developer tab; it listens on port 1234.
Sizes from LM Studio's Gemma 4 page. What to do next is on the chat page.
Install for Python, or build llama.cpp
Python and Transformers
# 1. Make a clean environment python -m venv .venv source .venv/bin/activate # Windows: .venv\Scripts\activate # 2. Install the libraries (from Google's model page) pip install -U transformers torch accelerate # For images and video, also: torchvision # For audio, also: torchvision librosa # 3. Check it worked python -c "import transformers; print(transformers.__version__)"
Commands from Google's model page. Use a current version of Transformers; older ones do not know Gemma 4. Loading code is on the 12B page.
llama.cpp
git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
cmake -B build && cmake --build build --config Release -j
# Downloads Google's 4-bit build on first run, then serves it
./build/bin/llama-server \
-hf google/gemma-4-12B-it-qat-q4_0-gguf \
--ctx-size 32768 --n-gpu-layers 99 --port 8080
You need git, CMake and a C++ compiler. The server speaks the OpenAI format on port 8080 - see the API page.
Full precision in Python needs far more memory than the 4-bit builds: about 27 GB for the 12B. Check the VRAM guide and QAT builds first.
Two settings most people miss
The model will run without these, just not as well as it could.
1. Context length
Gemma 4 can hold 128K to 256K tokens, but local apps usually start with a much smaller window. When a conversation or document outgrows it, the earliest part is dropped without any error.
2. Graphics card use
If the model does not fit in your graphics memory, part of it runs on the processor. You still get correct answers, at a fraction of the speed.
# Inside a chat, for this session only
/set parameter num_ctx 32768
# For every model, set before starting Ollama
export OLLAMA_CONTEXT_LENGTH=32768 # macOS, Linux
setx OLLAMA_CONTEXT_LENGTH 32768 # Windows, then restart Ollama
# A saved version of the model with a longer memory # Put these two lines in a file called Modelfile: FROM gemma4:12b-it-qat PARAMETER num_ctx 32768 # Then build and run it: ollama create gemma4-32k -f Modelfile ollama run gemma4-32k
These are standard Ollama settings; the 32K suggestion is this site's guidance. How to use the model once it is set up is in how to use Gemma 4.
Confirm your graphics card is being used
One command tells you. Start a chat, then in a second terminal run ollama ps and read the processor column.
ollama ps # which models are loaded, and on what nvidia-smi # NVIDIA: is the card and driver visible?
If it says CPU, or a split
The model is too large for your graphics memory at this context length. Lower the context, use the 4-bit build, or drop a size.
NVIDIA, AMD and Apple
NVIDIA needs a current driver. AMD needs its own driver stack installed. Apple silicon needs nothing: acceleration is built in.
On a suitable graphics card the 12B at 4-bit should write faster than you can read. If words arrive one at a time with pauses, it is running on the processor. Measured speeds for other devices are on the benchmarks page.
Install EmbeddingGemma 2
Google's open model for search and retrieval. It is small - 1.3 GB at most - and installs alongside Gemma 4 with the same tools. It does not chat; it turns text, code, images, video and audio into vectors you can search.
With Ollama
ollama pull embeddinggemma-2 # full model, 1.3 GB
ollama pull embeddinggemma-2:270m # text and code only, 378 MB
# Test it: this should print a long list of numbers
curl http://localhost:11434/api/embed -d '{
"model": "embeddinggemma-2",
"input": "hello"
}'
If Ollama is already installed for Gemma 4, this is the only step.
With Python
pip install -U "sentence-transformers[image,audio,video]" transformers
# Test it: downloads the model on first run, then prints (768,)
python -c "from sentence_transformers import SentenceTransformer as S; print(S('google/embeddinggemma-2').encode('hello').shape)"
Google's guide asks for sentence-transformers version 6.1.0 or later. The extras in brackets add image, audio and video support; leave them out for text only.
Pick a size
:270m for text and code, :440m with images, :570m with audio, or the full 740M model.
What it needs
Very little. Google reports about 567 MB of memory for the full model on a Pixel 11 Pro with quantisation.
One setting to avoid
Google's model card warns against float16. Use bfloat16 or float32.
Other runtimes
Google's guide also lists LM Studio, MLX, vLLM, SGLang and LiteRT. Check each one for an EmbeddingGemma 2 build.
Install command and version from Google's EmbeddingGemma 2 developer guide; sizes from Ollama's tag list; the float16 note from the model card. How to use it once installed is in how to use and the EmbeddingGemma 2 guide.
What your machine needs
Match the model to your memory, not your disk. Leave headroom on top of these figures for the conversation itself.
| Model | Memory, 4-bit | Memory, full precision | Ollama download | Realistic hardware |
|---|---|---|---|---|
| EmbeddingGemma 2 | - | - | 378 MB - 1.3 GB | Almost anything, including phones |
| Gemma 4 E2B | 2.9 GB | 11.4 GB | 4.3 GB | Any recent laptop; phones with the mobile build |
| Gemma 4 E4B | 4.5 GB | 17.9 GB | 6.1 GB | 8 GB of memory |
| Gemma 4 12B | 6.7 GB | 26.7 GB | 7.2 GB | 16 GB of memory, or a graphics card with 8 GB or more |
| Gemma 4 26B A4B | 14.4 GB | 57.7 GB | 16 GB | A 24 GB graphics card, or a 32 GB Mac |
| Gemma 4 31B | 17.5 GB | 69.9 GB | 19 GB | A 24 GB graphics card, or a 32 GB Mac |
Memory figures are Google's estimates for the weights, from the model card; downloads are Ollama's QAT builds; the hardware column is this site's guidance. Without a graphics card the two edge models are usable; larger ones run slowly. Details in the VRAM guide and technical specs.
Update, list and remove
ollama pull gemma4:12b-it-qat # fetch the newest build of a model ollama list # what is installed ollama rm gemma4:12b-it-qat # delete a model and free the space
Keep both up to date
Update the Ollama app from time to time, and pull your models again. Google refreshed Gemma 4's weights in July 2026 without changing their names.
Uninstalling completely
Remove the models first, since they take the most space, then uninstall the app as usual. Python downloads live in ~/.cache/huggingface and must be deleted separately.
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
| "ollama" is not recognised | The terminal was open before the install finished | Close and reopen the terminal |
| Nothing answers at localhost:11434 | Ollama is not running | Open the Ollama app, or run ollama serve |
| Very slow replies | Running on the processor | Check ollama ps; use a smaller size or lower the context |
| Out of memory | The model and context together exceed your memory | Lower the context, use the 4-bit build, or drop a size |
| It forgets earlier messages | The context window is at a small default | Raise it - see the two settings |
| Slow first reply, then fine | The model is being loaded into memory each time | Normal after a pause; keep a chat open to avoid reloading |
| Python says it does not recognise the model | Transformers is out of date | pip install -U transformers |
| Odd symbols or broken tool calls | Weights from before the July 2026 refresh | Remove the model and pull it again |
| EmbeddingGemma 2 gives poor results | Loaded in float16, or no task prompt | Use bfloat16, and pass a prompt name such as SearchQuery |
Causes and fixes are this site's guidance, not an official list. Download problems are covered in how to download; more answers in the FAQ.
Install questions
Which install method should I use?
Ollama, unless you have a reason not to. Choose LM Studio if you would rather not use a terminal, and Python when you are writing code against the model.
Do I need a graphics card?
No, but you will want one for anything above E4B. The two edge models are usable on a processor alone. Apple silicon Macs are a good middle ground.
Can I install it on Windows without WSL?
Yes. Ollama and LM Studio both have ordinary Windows installers.
Do I need to install CUDA myself?
Not for Ollama or LM Studio, which include what they need. You only need a current driver for your graphics card.
Can I install Gemma 4 and EmbeddingGemma 2 together?
Yes. They use the same tools, and EmbeddingGemma 2 adds at most 1.3 GB. Together they can answer questions from your own files - see how to use.
Does it work offline once installed?
Yes. After the download, neither model needs a connection. Your prompts stay on your machine; the app you installed may still check for its own updates.
How do I install it on a phone?
Use Google AI Edge Gallery on iPhone or Android - see mobile downloads and the Android page.
Is it free?
Yes. Both models are Apache 2.0 and the tools on this page are free. Hosted alternatives are on the pricing page; limits and responsible use are under limitations and safety.
Installed? Put it to work
Start a chat, connect your editor, or build search over your own files.