How to Run Qwen2.5-VL and Qwen3-VL Locally in 2026: Document OCR & Vision Guide

Which Qwen Vision Model Should You Run?

Qwen3-VL and Qwen2.5-VL Ollama Tags, Sizes and Licences

Hardware Requirements for Qwen Vision Models

• Download sizes are weights only: Qwen3-VL 8B is 6.2 GB and Qwen2.5-VL 7B is 6.0 GB. The context cache and the image encoder need extra memory that grows with context length and image size, so leave headroom above the download size.
• Smaller options: Qwen3-VL 2B (1.9 GB) and 4B (3.3 GB) or Qwen2.5-VL 3B (3.2 GB) suit small GPUs and laptops; expect lower accuracy than the 7B–8B models.
• Larger options: Qwen3-VL 30B-A3B (20 GB), Qwen3-VL 32B and Qwen2.5-VL 32B (21 GB) and Qwen2.5-VL 72B (49 GB) need a large GPU, several GPUs, or Apple Silicon with large unified memory.
• Partial offload: when a model does not fit in VRAM, Ollama can place part of it in system RAM; this works but is noticeably slower.
• Software: Ollama 0.12.7+ for Qwen3-VL and 0.7.0+ for Qwen2.5-VL, as stated on the Ollama library pages.

Setting Up Qwen2.5-VL and Qwen3-VL with Ollama

1. 1
   Install or update Ollama
   Why it matters: Qwen2.5-VL needs Ollama 0.7.0 or newer and Qwen3-VL needs 0.12.7 or newer. Check with `ollama --version` and update if needed. Ollama is available for macOS, Linux, and Windows.
2. 2
   Pull a model with an explicit tag
   Why it matters: Use `qwen3-vl:8b` or `qwen2.5vl:7b`. Do not use `qwen2-vl:7b` — it is not in the official Ollama library, so the pull fails.
3. 3
   Run the model and attach an image
   Why it matters: In interactive mode, type the image file path inside your prompt. Ollama detects the path and loads the image.
4. 4
   Send images through the API
   Why it matters: The /api/generate endpoint accepts a base64-encoded images array without a data: prefix. This is how applications — and PromptQuorum — send images programmatically.
5. 5
   Verify multilingual OCR
   Why it matters: Send a Chinese or Japanese document scan and compare the extracted text with the source before you build on it.

# Step 1 — Install or update Ollama (Qwen3-VL needs 0.12.7+, Qwen2.5-VL needs 0.7.0+)
# macOS
brew install ollama

# Linux
curl -fsSL https://ollama.com/install.sh | sh

# Windows — download from https://ollama.com/download
ollama --version

# Step 2 — Pull a Qwen vision model (explicit tag)
ollama pull qwen3-vl:8b          # 6.2 GB
# or: ollama pull qwen2.5vl:7b   # 6.0 GB

# Step 3 — Run and attach an image (interactive)
ollama run qwen3-vl:8b
>>> Extract every line of text from ./invoice-jp.png

# Step 4 — Send an image through the API
# Encode the image first:  base64 -i scan.png   (macOS)
curl http://localhost:11434/api/generate -d '{
  "model": "qwen3-vl:8b",
  "prompt": "Extract every line of text from this document. Preserve line breaks.",
  "images": ["<base64-encoded-image>"],
  "stream": false
}'

# Step 5 — Verify multilingual OCR
ollama run qwen3-vl:8b
>>> Extract all text from this image: ./contract-zh.png

Setting Up Qwen Vision Models with LM Studio

1. 1
   Download and install LM Studio
   Why it matters: A cross-platform GUI for local model inference. No terminal required.
2. 2
   Search for Qwen vision models
   Why it matters: Search "Qwen3-VL" or "Qwen2.5-VL" in the model browser and choose a build whose model card lists vision support. Match the quantization to your memory.
3. 3
   Load the model and attach an image
   Why it matters: Use the image button in the chat input to upload a photo or scan. LM Studio passes it to the vision encoder.
4. 4
   Start the local server
   Why it matters: The "Start Server" option exposes an OpenAI-compatible API at localhost:1234. Vision requests use the standard image_url content format.

// LM Studio — OpenAI-compatible vision request (localhost:1234)
{
  "model": "<model-id-shown-in-LM-Studio>",
  "messages": [
    {
      "role": "user",
      "content": [
        { "type": "text", "text": "Extract all text from this document." },
        {
          "type": "image_url",
          "image_url": { "url": "data:image/png;base64,<base64-encoded-image>" }
        }
      ]
    }
  ]
}

Document OCR for Chinese, Japanese, and Mixed-Language Files

• Plain text extraction: "Extract every line of text from this image. Preserve line breaks and reading order."
• Structured fields: "This is a Japanese invoice. Return vendor, date, subtotal, tax, and total as key: value pairs."
• Table extraction: "Extract this table as CSV. Treat the first row as the header."
• Extract and translate in one pass: "Extract the Chinese text from this image, then translate it to English. Show both."

# Japanese invoice -> structured fields
ollama run qwen3-vl:8b
>>> This is a Japanese invoice. Extract vendor name, invoice date,
    subtotal, consumption tax, and total. Return as key: value pairs.
    ./invoice-jp.png

# Illustrative output format (not a measured result):
# vendor: <vendor name>
# date: <invoice date>
# subtotal: <amount> JPY
# tax: <amount> JPY
# total: <amount> JPY

Image Q&A, Screenshot Analysis, and Chart Reading

• Image Q&A: ask open questions about a photo — "What is in this image?" — and check the answers against the image.
• Screenshot and UI analysis: start with reading UI screenshots, error dialogs, and app states. Both model cards also describe GUI-agent abilities, which this guide does not cover.
• Chart and graph reading: describing chart structure and trends works better than extracting exact numbers; confirm exact figures against the underlying data.
• Multi-image comparison: send several images in one request to compare versions or batch-describe a set; how many fit depends on your memory and context size.

Qwen2-VL, Qwen2.5-VL and Qwen3-VL Compared

Connecting a Local Qwen Vision Model to PromptQuorum

Troubleshooting Qwen Vision Models

• "pull model manifest: file does not exist" or an unknown model: the tag is not in the official library. `qwen2-vl:7b` is not available — use `ollama pull qwen3-vl:8b` or `ollama pull qwen2.5vl:7b`.
• The pull succeeds but the model fails to load: update Ollama. Qwen2.5-VL needs 0.7.0+ and Qwen3-VL needs 0.12.7+.
• The image is ignored and the model answers as if no image was sent: confirm the file path is correct and readable. In the Ollama API, the `images` array must contain raw base64 *without* the `data:` prefix — the `data:` prefix is LM Studio and OpenAI format only.
• Garbled or missing CJK characters: the scan is probably too low-resolution. Re-scan at 150–300 DPI and compare the result.
• Out of memory: choose a smaller tag (for example `qwen3-vl:4b` or `qwen2.5vl:3b`), reduce the context size or the image resolution, or run on Apple Silicon, which shares unified memory between CPU and GPU.
• Wrong numbers extracted from an invoice or chart: local vision models can misread digits on noisy input. Increase scan quality and always verify numeric output against the source.
• A PDF will not load: Ollama's API takes images, not PDF files. Convert pages to PNG or JPEG first (with pdf2image or pypdfium2), then send each page as a separate image.
• LM Studio shows "failed to load model": either insufficient memory, or you downloaded a non-vision build. Confirm the model card lists vision support and try a smaller quantization.

Related Reading

• How to Run Qwen 3 Locally — the text-model companion: Ollama and LM Studio setup for the Qwen language models.
• Local Vision Models Compared: LLaVA, Llama 3.2 Vision, MiniCPM-V — the full survey of other local vision models.
• Multimodal Local LLMs: Vision, Audio, and Text — broader overview of running multimodal models locally.
• Qwen vs Llama vs Mistral — how the Qwen text models compare to the alternatives.
• Local LLM Hardware Guide — VRAM and GPU requirements for local inference.
• Browse All Local LLM Guides

Frequently Asked Questions