Gemma 4 & EmbeddingGemma 2 Limitations
What these two models do not do well, with the numbers behind each claim and a practical workaround. Every model has limits; knowing them before you build saves the most time.
From Google's model cards and technical report · scores are Google's own ·
Five limits to know first
1. Long agent tasks
Multi-step work with tools is the family's weakest area, and it falls quickly on the small models.
2. The hardest reasoning
On Humanity's Last Exam the 31B scores 19.5. Larger hosted models do much better.
3. Knowledge stops at January 2025
Anything newer has to be supplied in the prompt or through a tool.
4. No audio on the two largest
Only E2B, E4B and 12B take audio. No size produces anything but text.
5. Safety is your job
A model you run yourself has no provider-side filter in front of it.
And for EmbeddingGemma 2
It does not generate text, and its smallest vectors lose a lot of quality on images and video. Details →
What Google documents
Quoted from the Gemma 4 model card. These apply to every size.
Training data
"Biases or gaps in the training data can lead to limitations in the model's responses."
What the model knows, and how evenly, depends on what it was trained on.
Complex and open-ended tasks
"Open-ended or highly complex tasks might be challenging."
It does best when the task is framed with clear prompts and instructions.
Nuance
"Models might struggle to grasp subtle nuances, sarcasm, or figurative language."
Worth remembering for sentiment analysis and moderation.
Factual accuracy
"They may generate incorrect or outdated factual statements."
Fluent wording is not evidence that something is true.
Common sense
"They might lack the ability to apply common sense reasoning in certain situations."
Odd failures on things a person would find obvious do happen.
Bias and misuse
"Can reflect socio-cultural biases embedded in the training material."
The card also notes such models can be misused to produce false or harmful text.
From Google's Gemma 4 model card.
Measured limits
Where the published scores show a real gap.
Agent tasks - Tau2
Hardest reasoning - Humanity's Last Exam
Agents and multi-step tool use
The 31B manages 76.9 on Tau2, but E4B scores 42.2 and E2B 24.5. A small model can call a tool; it cannot be relied on to chain many calls.
Frontier-difficulty reasoning
19.5 on Humanity's Last Exam for the 31B, 8.7 for the 26B A4B and 5.2 for the 12B. Giving the 31B a search tool lifts it to 26.5.
Reasoning across long documents on small models
E2B scores 4.1 on GraphWalks and 21.9 on BigBench Extra Hard. Its long context works for looking things up, not for reasoning across what it finds.
Large software projects
80.0 on LiveCodeBench v6 is for self-contained problems. Google has not published results for Gemma 4 on real-repository tests.
From Google's model card and technical report. All results are on the benchmarks page; how the sizes differ is in the model comparison.
What it can and cannot take in
| Limit | Detail | What to do |
|---|---|---|
| Text output only | No size generates images, audio or speech | Pair it with a dedicated generation tool |
| Audio on three sizes only | E2B, E4B and 12B; not the 26B A4B or 31B | Use the 12B when you need audio |
| Audio clips are short | 30 seconds each | Split longer recordings into segments |
| Video is short and sparse | 60 seconds, at one frame a second | Sample key moments; fast action can be missed |
| Image detail has a default | 280 tokens per image unless you raise it; headline scores use 1,120 | Use 560 or 1,120 for documents and small text |
| Counting in busy images | Google notes it may approximate | Ask specific questions; verify counts that matter |
| Knowledge ends January 2025 | Later events, prices and library versions are unknown to it | Paste the facts in, or connect search |
| Phone builds hold less | Google's phone builds list a 32K context, not the full 128K | Keep on-device prompts short |
From Google's model card and capability guides. How to send each input type is on the capabilities page; phone details are in mobile downloads and on the Android page.
Practical limits when you run it yourself
| Limit | Why it matters | What to do |
|---|---|---|
| Thinking costs time | Reasoning is generated text. Published accuracy assumes it is on | Turn it down for sorting and extraction; keep it for maths and code |
| Long context costs memory | A full 256K window needs far more than the weights alone | Set the context to what the task needs |
| Defaults are often too small | Local apps start with a short context, and the start of a long chat is dropped without warning | Raise it - see the install guide |
| Below 4-bit, quality falls | Very small builds load but reason and code noticeably worse | Use Google's QAT 4-bit builds; prefer a smaller model at 4-bit to a larger one at 3-bit |
| Memory sets the ceiling | A model that does not fit your graphics memory runs on the processor, slowly | Size to your hardware with the VRAM guide |
| Some endpoints lack features | Ollama's Anthropic-style endpoint has no prompt caching, PDF input or citations | Convert documents to text; clear long sessions between tasks |
| The weights changed once | A July 2026 refresh fixed the chat template and tool calling without new names | Download again if your copy is older |
This table is this site's guidance, drawn from Google's documentation and Ollama's compatibility notes. Setup help: how to download · runtime selector · Mac setup · self-hosting.
Safety limits
No filter in front
A hosted chatbot usually has moderation around the model. Weights you run yourself have only what was trained in.
Fine-tuning can weaken it
Further training can erode trained-in safety behaviour. Re-test after you fine-tune.
High-stakes domains
Medical, legal and financial output needs a qualified person checking it. Fluent text can be wrong.
Privacy is on you
Google's card asks developers to follow privacy rules and use privacy-preserving techniques.
Practical steps are on the safety page. If you deploy to the public, add your own input and output checks.
EmbeddingGemma 2 limitations
EmbeddingGemma 2 turns text, code, images, video and audio into vectors for search. Its limits are different in kind from a chat model's: they are about what fits in, how short the vectors can go, and settings that quietly reduce quality.
| Vector length | Text | Images | Multimodal |
|---|---|---|---|
| 768 · full | 61.36 | 64.64 | 59.01 |
| 512 | 61.17 | 64.32 | 58.38 |
| 256 | 60.41 | 63.13 | 56.24 |
| 128 | 57.89 | 59.06 | 45.65 |
| Input | Cost | Most that fits |
|---|---|---|
| Text | 1 token per word piece | 8,192 tokens |
| Images | 280 tokens each | About 29 |
| Video | 140 tokens a frame | About 58 frames |
| Audio | 25 tokens a second | About 327 seconds |
Left: scores on MTEB (text), MIEB (images) and MMEB (multimodal) from Google's EmbeddingGemma 2 model card, which calls quality "close to lossless down to 256 dimensions". Right: these are alternatives, not additions - everything in one input shares the same 8,192-token budget.
It does not write
It produces vectors only. It cannot answer a question, summarise or chat.
128-number vectors lose a lot
Text holds up reasonably, but the multimodal score drops from 59.01 to 45.65.
It needs a task label
Google says leaving out the recommended task prefix for text "may lead to sub-optimal embedding quality".
SearchQuery or Document.It breaks in float16
Google's card says "Do not use float16": results can fail or quietly get worse.
bfloat16 or float32.Languages are not equal
It supports 100+ languages, but Google notes it "may not exhibit equal performance across languages".
Version 1 vectors do not mix
Vectors from the original EmbeddingGemma and from version 2 sit in different spaces.
Long media must be split
One input holds about five and a half minutes of audio or 58 video frames. Longer recordings need cutting into segments, each embedded separately.
No hosted API from Google yet
Google says a Model Garden listing is coming soon. Until then you run it yourself; see the API page.
Bias and use policy
Like any model trained on real-world data it can reflect bias, and Google's Gemma Prohibited Use Policy applies to what you build.
From Google's EmbeddingGemma 2 model card and developer guide. Support in tools other than sentence-transformers and Ollama may arrive on its own schedule, so check each one for an EmbeddingGemma 2 build. Specs, prompts and downloads are in the EmbeddingGemma 2 guide.
Gaps in the ecosystem
Not every size is hosted by Google
Google's API lists the 31B and 26B A4B. The 12B - the largest with audio - is not there, so you run it yourself or use another host. See pricing.
Nothing tiny in Gemma 4
The smallest is E2B at 2.3B effective parameters. For something much smaller there is only Gemma 3 270M, under the older licence.
Older variants keep older terms
Only Gemma 4 and models built on it are Apache 2.0. ShieldGemma, TranslateGemma, MedGemma and others remain under the Gemma Terms of Use.
Scores are mostly Google's own
Apart from the public Arena ranking, the figures on this site come from Google's model card and report. Test on your own task.
Larger models exist
Models many times the size score higher on the hardest work. See Gemma 4 vs Qwen3.8-Max and Gemma 4 vs Claude Code.
Tool support varies
Not every app supports every feature on day one. Check your runtime with the runtime selector.
The full line-up is on the models page; raw figures in technical specs.
Seven mitigations, in priority order
Ground factual work in sources
Retrieval or search addresses both the 2025 cutoff and confident wrong answers. EmbeddingGemma 2 is built for exactly this.
Add your own safety checks
If the public will use it, check what goes in and what comes out.
Keep agent chains short, and verify
It is the weakest measured area. Approve steps, and do not leave small models to run unattended.
Pick the model by what it must take in
Audio means the 12B or an edge model, whatever your hardware could otherwise run.
Fix the settings before judging quality
Context length, thinking mode and image detail change results more than most prompt tweaks.
Use the right EmbeddingGemma 2 settings
A task prompt,
bfloat16, and vectors of 256 numbers or more.Test on your own task
Twenty or thirty real examples, including hard ones, tell you more than any leaderboard.
Technique for each is in prompting and how to use; try ideas quickly in the playground or chat.
What is Gemma 4's biggest limitation?
Reliability on long, multi-step agent tasks, especially on the smaller models: Tau2 falls from 76.9 on the 31B to 24.5 on E2B.
Does Gemma 4 make things up?
It can. Google's model card says it "may generate incorrect or outdated factual statements". The January 2025 cutoff makes this more likely for recent topics.
Is Gemma 4 safe to use in a product?
For many uses, yes, with your own safeguards. Open weights have no provider filter, so safety checks are the developer's responsibility.
What can EmbeddingGemma 2 not do?
It cannot generate text. It also cannot take more than about 8,192 tokens in one input, and it should not be loaded in float16.
How short can EmbeddingGemma 2 vectors be?
Down to 256 numbers with little loss. At 128, text holds up fairly well but multimodal quality drops sharply, from 59.01 to 45.65 on MMEB.
Will these limits be fixed?
Some already were: the July 2026 refresh improved tool calling. Others, such as the knowledge cutoff and the ceiling on reasoning, would need a new model.
Should the limits put me off?
Not by themselves. The question is whether they touch your task. Summarising, extraction, translation, scoped coding and on-device work are well within reach - see features.
How do I test the limits myself?
Collect twenty to thirty examples from your real work, including difficult ones, run them on two sizes and read the outputs. Models are in the download hub.
Know the limits, then build around them
See what both models do well, and how to combine them.