How to Install Gemma 4: Complete Cross-Platform Setup

💻 Hardware Requirements by Variant

Successful installation depends on matching your hardware capabilities to the target model variant. Gemma 4 supports multiple parameter scales, each with distinct memory and compute requirements:

• 2B Variant: Minimum 8GB RAM, integrated GPU or CPU-only. Ideal for laptops, low-power devices, and quick prototyping. Supports INT4/INT8 quantization for sub-2GB footprints.
• 9B Variant: 16GB RAM minimum, discrete GPU (RTX 3060+/M2 Pro+) recommended. Balanced capability for desktop development, local assistants, and medium-scale RAG pipelines.
• 27B Variant: 32GB+ RAM, 12GB+ VRAM GPU (RTX 3090/4090, A10G). Required for advanced reasoning, multi-step planning, and enterprise-grade local inference.
• 270B MoE Variant: Multi-GPU cluster or cloud instance (8×A100/H100). Supports tensor parallelism and expert routing for frontier-level capabilities.

Storage Requirements: Use NVMe SSDs exclusively. HDDs or eMMC storage cause severe I/O bottlenecks during model loading, KV cache management, and context window expansion. Allocate 2× the model size for temporary files, checkpoints, and runtime caches.

️ Software Dependencies

Before proceeding, ensure your system meets baseline software requirements:

• Operating System: Windows 10/11 (WSL2 recommended), macOS 13+ (Apple Silicon or Intel), Ubuntu 20.04+/Debian 11+, CentOS/RHEL 8+, Android 12+
• Python Environment: Python 3.10–3.12 with virtual environment isolation (venv, conda, or uv)
• GPU Drivers: NVIDIA CUDA 12.1+ (Windows/Linux), Apple Metal Performance Shaders (macOS), Vulkan/OpenCL (Android/Linux CPU fallback)
• Build Tools: Git, CMake, GCC/Clang, Python development headers (python3-dev or python-devel)
• Network Access: Stable internet connection for weight downloads, package repositories, and license verification

📥 Installation & Verification

Ollama provides the fastest path to running Gemma 4 locally with automatic hardware detection, quantization selection, and REST API exposure. It supports Windows, macOS, and Linux with unified CLI commands.

curl -fsSL https://ollama.com/install.sh | sh

ollama --version

ollama pull gemma4:9b
ollama run gemma4:9b

curl http://localhost:11434/api/generate -d '{
  "model": "gemma4:9b",
  "prompt": "Explain quantum entanglement in simple terms",
  "stream": false
}'

⚙️ Performance Tuning

• Context Window: Set via OLLAMA_NUM_CTX=4096 environment variable. Higher values increase memory usage but enable longer conversations.
• GPU Layers: Override automatic detection: OLLAMA_NUM_GPU=999 forces full GPU offloading; OLLAMA_NUM_GPU=0 forces CPU-only mode.
• Concurrent Requests: Adjust queue depth: OLLAMA_MAX_QUEUE=512 for high-throughput API servers.
• Model Management: List installed models: ollama list. Remove unused variants: ollama rm gemma4:27b

🔧 Environment Setup

The Transformers library provides maximum flexibility for research, fine-tuning, and custom pipeline development. This method requires Python proficiency and GPU driver configuration.

python -m venv gemma4-env
source gemma4-env/bin/activate  # macOS/Linux
# gemma4-env\Scripts\activate  # Windows

pip install --upgrade pip
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
pip install transformers accelerate safetensors bitsandbytes

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "google/gemma-4-9b-it"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto",
    attn_implementation="flash_attention_2"
)

prompt = "Write a Python function to calculate fibonacci numbers"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

🚀 Optimization Flags

• Flash Attention 2: Reduces memory overhead and accelerates training/inference by 2–3×. Requires CUDA 11.8+ and Ampere/Hopper GPUs.
• 4-bit Quantization: Add load_in_4bit=True, bnb_4bit_compute_dtype=torch.float16 to from_pretrained() for consumer GPU deployment.
• Static KV Cache: Enable use_cache=True and pre-allocate cache tensors for deterministic latency in production APIs.
• Memory Mapping: Use device_map="balanced" for multi-GPU setups to distribute layers evenly across available VRAM.

🔨 Build from Source

llama.cpp delivers exceptional CPU inference performance, minimal dependencies, and cross-platform compatibility. Ideal for laptops, Raspberry Pi, Android devices, and air-gapped environments.

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
mkdir build && cd build
cmake .. -DGGML_CUDA=ON  # Enable GPU acceleration (optional)
cmake --build . --config Release

wget https://huggingface.co/google/gemma-4-9b-it/resolve/main/gemma-4-9b-it-Q4_K_M.gguf
sha256sum gemma-4-9b-it-Q4_K_M.gguf  # Compare with official hash

./llama-cli -m ../gemma-4-9b-it-Q4_K_M.gguf \
  -n 512 -t 8 -c 4096 --interactive \
  --prompt "Explain how transformers work"

Mobile & Edge Deployment

llama.cpp powers most Android/iOS LLM applications. For mobile integration:

• Android (Termux): Install build essentials, compile with Vulkan support, and run CLI or wrap in a simple Python/Flask API.
• iOS (Swift Integration): Use pre-compiled llama.framework or integrate via Swift Package Manager. Metal backend provides 3–5× speedup over CPU.
• Raspberry Pi 5: Compile with -DGGML_RKNN=ON for NPU acceleration on supported boards. Expect 8–12 tok/s on 9B INT4.