How to Use Gemma 4 & EmbeddingGemma 2

1. Choose where to run it

   Nothing to install: Google AI Studio or the playground. On your own machine: Ollama or LM Studio. In an app: Transformers or a hosted API. Unsure? Use the runtime selector.

2. Start a chat

   With Ollama it is one command: ollama run gemma4:12b-it-qat. Type a message and you are talking to the model, offline. Other ways in are on the chat page.

3. Give it a system prompt

   Gemma 4 has a real system role. Use it once to set the role, tone and output format, then keep your messages short and specific.

4. Turn thinking on for hard problems

   Thinking makes the model reason before it answers. Worth it for maths, code and multi-step tasks; unnecessary for quick chat.

From Google's model card, thinking guide and function-calling guide. Each setting is covered in depth on the capabilities page; prompt technique is in prompting; the model list is on models.

1. Embed what you want to search

   Run every document, image or clip through the model once and save the vectors.

2. Embed the query

   When someone searches, turn their query into a vector the same way.

3. Compare

   The saved vectors closest to the query's vector are your results.

# pip install -U sentence-transformers transformers
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("google/embeddinggemma-2")

query = "What causes the northern lights?"
document = "The northern lights are caused by charged particles from the sun."

query_emb = model.encode(query, prompt_name="SearchQuery")
doc_emb = model.encode(document, prompt_name="Document")

print(model.similarity(query_emb, doc_emb))   # higher means closer in meaning

# pip install -U "sentence-transformers[image,audio,video]" transformers
query_emb = model.encode("ocean waves at sunset", prompt_name="SearchQuery")
image_emb = model.encode({"image": "sunset_beach.jpg"})
audio_emb = model.encode({"audio": "ocean_waves.wav"})

# Everything lives in one space, so text can be compared with media
print(model.similarity(query_emb, image_emb))
print(model.similarity(query_emb, audio_emb))

# Download once
ollama pull embeddinggemma-2

# Ask the local server for an embedding
curl http://localhost:11434/api/embed -d '{
  "model": "embeddinggemma-2",
  "input": "What causes the northern lights?"
}'

# Shorter vectors: 256 dimensions instead of 768
query_emb = model.encode(
    query,
    prompt_name="SearchQuery",
    truncate_dim=256,
    normalize_embeddings=True,
)

Google's EmbeddingGemma 2 developer guide is the full walkthrough. Specs, scores and downloads are in this site's EmbeddingGemma 2 guide.

# pip install -U sentence-transformers transformers openai
# ollama pull gemma4:12b-it-qat
from sentence_transformers import SentenceTransformer
from openai import OpenAI

embedder = SentenceTransformer("google/embeddinggemma-2")
gemma = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")

# 1. Embed your passages once
passages = [
    "Refunds are available within 30 days of purchase.",
    "Support is open Monday to Friday, 9am to 5pm.",
    "Orders over $50 ship free within the country.",
]
passage_embs = embedder.encode(passages, prompt_name="Document")

# 2. Embed the question and find the closest passages
question = "Can I get my money back after three weeks?"
q_emb = embedder.encode(question, prompt_name="QuestionAnswering")
scores = embedder.similarity(q_emb, passage_embs)[0]
top = [passages[i] for i in scores.argsort(descending=True)[:2]]

# 3. Let Gemma 4 answer from what was found
reply = gemma.chat.completions.create(
    model="gemma4:12b-it-qat",
    messages=[
        {"role": "system", "content": "Answer only from the context. "
         "If the answer is not there, say you do not know."},
        {"role": "user", "content": "Context:\n" + "\n".join(top) +
         "\n\nQuestion: " + question},
    ],
)
print(reply.choices[0].message.content)

This sample is a starting point written for this page and has not been run against live models here; the calls it uses come from Google's guides and Ollama's OpenAI-compatible endpoint. For a shared deployment see self-hosting; for memory planning, the VRAM guide.