To run Gemma 4 with Ollama, install Ollama and enter ollama run gemma4:e4b in your terminal. This downloads the E4B library model and opens an interactive chat. The model tag identifies the variant; use the Ollama library listing to check the current download size before choosing E2B, E4B, 26B or 31B.
Choose the library model when you want a quick start. If you already downloaded a GGUF file and need that specific quantization, import it with a Modelfile using FROM /path/to/file.gguf, then run ollama create my-model and ollama run my-model. A GGUF file is not an Ollama library tag. See the download guide for files and the hardware guide before selecting a larger model or longer context.
Download from ollama.com or install via Homebrew:
# Homebrew
brew install ollama
# Or download from https://ollama.com/download/macOne-line install script:
curl -fsSL https://ollama.com/install.sh | shDownload the installer from ollama.com or use winget:
# winget
winget install Ollama.Ollama
# Or download from https://ollama.com/download/windowsVerify installation:
ollama --versionAll Gemma 4 variants are available in the Ollama library. Choose based on your hardware and needs:
Ultra-lightweight for edge devices and basic tasks
Best balance of quality and resource usage
MoE architecture — large model quality at small model cost
Maximum quality — flagship dense model
# Start interactive chat with Gemma 4 E4B
ollama run gemma4:e4b
# Or the flagship 31B model
ollama run gemma4:31bollama run gemma4:e4b "Explain quantum computing in simple terms"# In the interactive chat, use /image to add images
ollama run gemma4:e4b
>>> /image photo.jpg What do you see in this image?Ollama provides an OpenAI-compatible REST API at localhost:11434, making it easy to integrate Gemma 4 into your applications:
curl http://localhost:11434/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "gemma4:e4b",
"messages": [
{"role": "user", "content": "Hello, Gemma 4!"}
]
}'curl http://localhost:11434/api/generate \
-d '{
"model": "gemma4:e4b",
"prompt": "Write a Python function to sort a list"
}'Create a custom Modelfile to adjust model parameters like temperature, context length, and system prompt:
FROM gemma4:e4b
PARAMETER temperature 0.7
PARAMETER num_ctx 32768
SYSTEM """
You are a helpful coding assistant. Always provide code examples.
"""Ollama auto-detects GPUs, but you can control GPU layer offloading:
# Set number of GPU layers
OLLAMA_NUM_GPU=35 ollama run gemma4:31b
# CPU only mode
OLLAMA_NUM_GPU=0 ollama run gemma4:e4bIncrease the default context window for longer conversations:
ollama run gemma4:e4b --num-ctx 65536Ollama downloads from ollama.com CDN. If slow, check your internet connection or try a VPN. Large models (26B, 31B) may take 10-30 minutes depending on bandwidth.
Try a smaller model variant or a quantized version. Use 'ollama run gemma4:e4b' instead of the 31B model. On systems with limited RAM, close other applications before running.
Ensure Ollama is using your GPU: check with 'ollama ps'. On Mac, Ollama uses Metal GPU acceleration automatically. On Linux/Windows, ensure NVIDIA or AMD GPU drivers are properly installed.
Make sure the Ollama service is running: 'ollama serve'. The default API endpoint is http://localhost:11434. Check firewall settings if accessing from another machine.
For most users, gemma4:e4b offers the best balance of quality and performance. If you have a GPU with 16GB+ VRAM, gemma4:26b provides near-flagship quality with efficient MoE inference. The gemma4:31b model requires 24GB+ VRAM but delivers maximum performance.
Yes. Ollama supports CPU-only inference for all Gemma 4 variants. The E2B and E4B models run reasonably fast on CPU. Larger models will be significantly slower without GPU acceleration but still functional.
Run 'ollama pull gemma4:e4b' (or your preferred variant) to download the latest version. Ollama will only download the differences if you already have a previous version installed.
Yes. Ollama's OpenAI-compatible API works with most AI tools and frameworks including LangChain, LlamaIndex, Open WebUI, Continue.dev, and many others. Just point them to http://localhost:11434.
Yes. Ollama supports Gemma 4's multimodal capabilities. You can pass images to the model using the /image command in the interactive chat or via the API's image parameter.
Disk space depends on the variant: E2B (~1.5GB), E4B (~3GB), 26B MoE (~15GB), 31B Dense (~18GB). These are for the default quantization. Models are stored in ~/.ollama/models on macOS/Linux.
Install Ollama and start chatting with Gemma 4 in minutes. Or explore other deployment options.