Gemma 4 & EmbeddingGemma 2 on Ollama

1. Install Ollama

   Download it from ollama.com/download. On Linux: curl -fsSL https://ollama.com/install.sh | sh

2. Run a Gemma 4 model

   In a terminal: ollama run gemma4:12b-it-qat. The first run downloads 7.2 GB, then opens a chat. On an 8 GB machine use gemma4:e4b-it-qat.

3. Add the search model if you want it

   ollama pull embeddinggemma-2 adds EmbeddingGemma 2 in 1.3 GB.

Step-by-step help for each system is in the install guide; Mac details in Mac setup. Prefer an app with buttons? See LM Studio.

ollama pull gemma4:12b-it-qat          # download a model
ollama run gemma4:12b-it-qat           # chat in the terminal
ollama run gemma4 "roses are red"      # one-off prompt
ollama list                            # models you have
ollama ps                              # models loaded now, and on what
ollama show gemma4:12b-it-qat          # details of a model
ollama stop gemma4:12b-it-qat          # unload from memory
ollama rm gemma4:12b-it-qat            # delete to free disk

/set system You are a concise assistant.
/set parameter num_ctx 32768     # longer memory for this chat
/set think                       # reason before answering
/set nothink                     # answer straight away
/clear                           # forget the conversation so far
/bye                             # leave

ollama run gemma4:12b-it-qat "What is in this image? ./photo.png"

Thinking controls from Ollama's thinking documentation. More on using the model in how to use Gemma 4, chat and prompting.

# pip install ollama
import ollama

reply = ollama.chat(
    model="gemma4:12b-it-qat",
    messages=[
        {"role": "system", "content": "You are a concise assistant."},
        {"role": "user", "content": "Why is the sky blue?"},
    ],
)
print(reply["message"]["content"])

curl http://localhost:11434/api/chat -d '{
  "model": "gemma4:12b-it-qat",
  "messages": [
    {"role": "system", "content": "You are a concise assistant."},
    {"role": "user", "content": "Why is the sky blue?"}
  ],
  "think": false,
  "stream": false
}'

# pip install openai
from openai import OpenAI

client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")

reply = client.chat.completions.create(
    model="gemma4:12b-it-qat",
    messages=[{"role": "user", "content": "Why is the sky blue?"}],
)
print(reply.choices[0].message.content)

More on endpoints and hosted alternatives on the API page; connecting an editor is in Gemma 4 coding.

# pip install ollama
# ollama pull embeddinggemma-2   ·   ollama pull gemma4:12b-it-qat
import ollama

passages = [
    "Refunds are available within 30 days of purchase.",
    "Support is open Monday to Friday, 9am to 5pm.",
    "Orders over $50 ship free within the country.",
]
question = "Can I get my money back after three weeks?"

# 1. Turn the passages and the question into vectors
p_vecs = ollama.embed(model="embeddinggemma-2", input=passages)["embeddings"]
q_vec = ollama.embed(model="embeddinggemma-2", input=question)["embeddings"][0]

# 2. Find the closest passage (the vectors are unit length, so a dot product is enough)
scores = [sum(a * b for a, b in zip(q_vec, p)) for p in p_vecs]
best = passages[scores.index(max(scores))]

# 3. Let Gemma 4 answer from it
reply = ollama.chat(
    model="gemma4:12b-it-qat",
    messages=[
        {"role": "system", "content": "Answer only from the context. "
         "If the answer is not there, say you do not know."},
        {"role": "user", "content": f"Context: {best}\n\nQuestion: {question}"},
    ],
)
print(reply["message"]["content"])

This sample was written for this page from Ollama's documented calls and has not been run against live models here. The same idea with Google's Python library is in how to use both.

Causes and fixes are this site's guidance, not an official list. More in the install guide and the FAQ.