Ollama Vision Models (2026): How to Actually Run Image Models Locally

History: Ollama, llama.cpp, and Multimodal Support

Which Vision Models Are Actually in Ollama's Library

Pulling and Running a Vision Model: Step by Step

1. 1
   Install Ollama.
   Why it matters: Download [Ollama](https://ollama.com) for macOS, Linux, or Windows. Installation is a standard installer/package and takes under two minutes.
2. 2
   Pull a vision model.
   Why it matters: Run `ollama pull llava` (or `ollama pull qwen2.5vl`, `ollama pull minicpm-v`, etc.) — this downloads the model weights, sized from under 2 GB (Moondream) to tens of gigabytes (Llama 4).
3. 3
   Run it with an image referenced in the prompt.
   Why it matters: Run `ollama run llava "describe this image: ./photo.jpg"`. Ollama detects the `.jpg`/`.png` file path in the prompt text and attaches the image automatically — this pattern has worked since v0.1.15 in December 2023.
4. 4
   Or call the HTTP API directly.
   Why it matters: POST to `http://localhost:11434/api/generate` or `/api/chat` with the image base64-encoded in an `images` array — the exact JSON shape is documented in Ollama's own [docs/api.md](https://github.com/ollama/ollama/blob/main/docs/api.md), and shown in the Usage Examples section below.
5. 5
   (Optional) Use the official client libraries.
   Why it matters: The official `ollama` Python and JavaScript libraries accept an image file path directly and handle base64 encoding for you, avoiding manual encoding in scripts.

Real Usage Examples: CLI and HTTP API

• No separate image flag exists in the CLI. Ollama detects a .jpg/.png/similar file path inside the prompt text itself and attaches it automatically.
• /api/generate uses prompt; /api/chat uses messages.** Both accept an images array of base64-encoded strings; /api/chat supports multi-turn conversations with images attached to individual messages.

# Pull and run via CLI — reference the image path directly in the prompt
ollama pull llava
ollama run llava "describe this image: ./photo.jpg"

# --- HTTP API: /api/generate (documented in Ollama's docs/api.md) ---
curl http://localhost:11434/api/generate -d '{
  "model": "llava",
  "prompt": "What is in this picture?",
  "stream": false,
  "images": ["<base64-encoded image data>"]
}'

# --- HTTP API: /api/chat (multi-turn, also documented in docs/api.md) ---
curl http://localhost:11434/api/chat -d '{
  "model": "llava",
  "messages": [
    { "role": "user", "content": "What is in this image?", "images": ["<base64-encoded image data>"] }
  ]
}'

# --- Python: official ollama library (handles base64 encoding for you) ---
import ollama

response = ollama.chat(
    model="llava",
    messages=[{
        "role": "user",
        "content": "What is in this image?",
        "images": ["photo.jpg"],
    }],
)
print(response["message"]["content"])

# --- Python: raw HTTP API with manual base64 encoding ---
import base64
import requests

def ask_vision_model(image_path: str, prompt: str, model: str = "llava") -> str:
    with open(image_path, "rb") as f:
        image_b64 = base64.b64encode(f.read()).decode("utf-8")
    response = requests.post(
        "http://localhost:11434/api/generate",
        json={"model": model, "prompt": prompt, "images": [image_b64], "stream": False},
    )
    return response.json()["response"]

VRAM and Hardware Guidance

What Ollama Is Not Good For

• Fine-tuning or training a model. Ollama is inference-only — it runs pre-trained model weights, it does not provide a training or fine-tuning pipeline. If you need to fine-tune a vision-language model on your own data, you need a different toolchain (such as the original LLaVA repository's own training scripts, or a framework like Hugging Face Transformers).
• Bleeding-edge proprietary multimodal capability. Ollama's library is built around open-weight models. As of this review, GPT-4o, Claude, and Gemini's cloud vision APIs generally lead open local models on complex scene understanding, handwriting recognition, and ambiguous imagery — Ollama is the right tool for private, self-hosted use at zero marginal cost per image, not for matching the absolute state of the art.
• Precise numeric extraction from charts and graphs. This is a limitation of the underlying vision-language models themselves, not specific to Ollama as a runner — verify any extracted numbers against source data regardless of which model or runner you use.
• A single unified answer to "which model is best." The right vision model via Ollama depends on the task: Qwen2.5-VL for OCR-heavy document work, MiniCPM-V for OCR at lower VRAM, Llama 3.2 Vision for general image Q&A, Moondream for the lightest footprint. See PromptQuorum's local vision models comparison for task-by-task guidance.

Alternatives to Ollama for Vision Models

Frequently Asked Questions

Verdict

Sources

• Ollama on GitHub — repository creation date, license, and release history.
• Ollama v0.1.15 release notes — the original multimodal/LLaVA support announcement, December 12, 2023.
• Ollama's new engine for multimodal models — the May 2026 multimodal engine rebuild, covering Llama 4, Gemma 3, Qwen2.5-VL, and Mistral Small 3.1.
• Ollama API documentation — the documented /api/generate and /api/chat request/response shape for images.
• Ollama model library — the live library used to verify every model listed in this article, including individual pages for llava, llama3.2-vision, qwen2.5vl, minicpm-v, moondream, granite3.2-vision, gemma3, llama4, and mistral-small3.1.

Related Reading

• LLaVA Review (2026) — a dedicated deep dive on LLaVA specifically, including its history and license nuance.
• Local Vision Models 2026: LLaVA, Llama 3.2 Vision, Qwen3-VL & Ollama Multimodal Setup — a broader, task-by-task comparison across current local vision models.
• Ollama Latest Version 2026: v0.33.1 + Best Models by Use Case — Ollama's wider model catalog beyond vision, including which text models are most popular.