LLaVA Review (2026): The Research Model That Started Local Vision AI

History: From a Research Paper to a Local AI Standard

What LLaVA Actually Does

• CLIP encoder plus LLM decoder. LLaVA uses a CLIP ViT-L/14 vision encoder to convert an image into a sequence of visual features, projects those features into the same embedding space as the language model's text tokens via a connector module, and feeds the combined sequence into a Vicuna (Llama 2-based) or, in later versions, Mistral or other base LLM to generate a response.
• Visual instruction tuning. Instead of training on raw image-caption pairs, LLaVA was fine-tuned on LLaVA-Instruct-150K, a dataset of multi-turn visual conversations generated by prompting GPT-4 with image captions and object-detection annotations — teaching the model to follow open-ended visual instructions rather than just generate captions.
• Three model sizes via Ollama. The Ollama library packages LLaVA in 7B (4.7 GB), 13B (8.0 GB), and 34B (20 GB) sizes — larger models generally produce more detailed and accurate descriptions at the cost of more VRAM and slower inference.
• LLaVA-1.6 (LLaVA-NeXT) resolution improvements. The current version supported through Ollama and the official repository handles higher input resolutions via dynamic patching, which noticeably improves text-reading and fine-detail tasks compared to the original 2023 release.
• English-centric training. LLaVA-Instruct-150K and the underlying benchmarks it was evaluated on are predominantly English-language, so out-of-the-box performance on non-English text within images is weaker than models specifically trained on multilingual document data.

Install and Run LLaVA: Step by Step

1. 1
   Install Ollama.
   Why it matters: Download and install [Ollama](https://ollama.com) for macOS, Linux, or Windows. This is the officially listed way to run LLaVA per its [Ollama library page](https://ollama.com/library/llava), and takes under two minutes.
2. 2
   Pull a LLaVA model size.
   Why it matters: Run `ollama pull llava` for the default 7B model (~4.7 GB), or `ollama pull llava:13b` (~8.0 GB) or `ollama pull llava:34b` (~20 GB) for higher quality at the cost of more VRAM and disk space.
3. 3
   Run it with an image from the CLI.
   Why it matters: Run `ollama run llava "describe this image: ./photo.jpg"`, referencing a local image file path directly in the prompt text — no separate flag is required.
4. 4
   (Alternative) Use the original repository for research-grade control.
   Why it matters: Clone [haotian-liu/LLaVA](https://github.com/haotian-liu/LLaVA), install its `requirements.txt`, and download a checkpoint from Hugging Face for full access to training scripts, evaluation harnesses, and the Gradio web demo — useful for research or fine-tuning, but a meaningfully higher setup bar than Ollama.
5. 5
   (Optional) Call it programmatically via the Ollama API.
   Why it matters: POST to `http://localhost:11434/api/generate` with the image as a base64-encoded string in an `images` array, or use the official `ollama` Python or JavaScript library — see the code example below.

Real Usage Examples

• Referencing a file path in the CLI prompt is enough — Ollama detects the .jpg/.png path and attaches the image automatically; no separate --image flag exists.
• Larger model sizes cost more VRAM and time. The 7B model is the fastest and lowest-VRAM option; use 13B or 34B when description accuracy matters more than speed.

# Install and pull the model
# (download Ollama from https://ollama.com first)
ollama pull llava

# CLI: describe an image by referencing its file path in the prompt
ollama run llava "describe this image: ./photo.jpg"

# --- HTTP API (documented in Ollama's own docs/api.md) ---
# /api/generate with a base64-encoded image
curl http://localhost:11434/api/generate -d '{
  "model": "llava",
  "prompt": "What is in this picture?",
  "stream": false,
  "images": ["<base64-encoded image data>"]
}'

# /api/chat with a base64-encoded image (multi-turn conversations)
curl http://localhost:11434/api/chat -d '{
  "model": "llava",
  "messages": [
    { "role": "user", "content": "What is in this image?", "images": ["<base64-encoded image data>"] }
  ]
}'

# --- Python example using the official ollama library ---
import ollama

response = ollama.chat(
    model="llava",
    messages=[{
        "role": "user",
        "content": "What is in this image?",
        "images": ["photo.jpg"],
    }],
)
print(response["message"]["content"])

License and Cost

What LLaVA Is Not Good For

• Text-only tasks. LLaVA is a vision-language model; for a pure text conversation with no image involved, use a dedicated text LLM (through Ollama or otherwise) — running a multimodal model for text-only chat wastes the VRAM its vision encoder occupies for no benefit.
• Needing the strongest available local OCR or document understanding. As of 2026, models released after LLaVA-NeXT — MiniCPM-V, Qwen2.5-VL, and Llama 3.2 Vision — outperform it on document OCR, tables, and structured extraction, per PromptQuorum's local vision models comparison. LLaVA's vision encoder was designed and trained before this generation of higher-resolution, document-focused training data existed.
• Reading non-English text in images. LLaVA-Instruct-150K and the model's core training data are predominantly English. For Chinese, Japanese, Korean, or other non-Latin-script document OCR, a model trained specifically on multilingual document corpora (Qwen2.5-VL) will meaningfully outperform it.
• Precise numeric extraction from charts and graphs. Like essentially every local vision-language model in 2026, LLaVA is unreliable at reading exact values off complex charts — verify any extracted numbers against the source data regardless of which model you use.
• Assuming ongoing active development. With no commits to the official repository since May 11, 2024, do not expect new features, bug fixes, or newer checkpoints from the original project — Ollama's own multimodal engine updates and newer model families have effectively superseded it as the actively developed option.
• A single, simple license story. Because the code (Apache-2.0) and the officially released checkpoints (Apache-2.0 plus inherited Llama 2 community license terms, plus a stricter research-only statement on the project's separate homepage) are governed by three overlapping, not fully aligned statements, LLaVA does not offer the one-line license clarity that a project like Bark (fully MIT, no extra conditions) does.

Alternatives to LLaVA

Frequently Asked Questions

Verdict

Sources

• LLaVA on GitHub — the official repository: README, LICENSE, install instructions, and commit history.
• "Visual Instruction Tuning" paper (NeurIPS 2023) — the project homepage with paper links, licensing statements, and version history.
• LLaVA on Ollama — the officially listed model page: sizes, description, and library metadata.
• Ollama API documentation — the documented /api/generate and /api/chat request/response shape for multimodal models.
• Local Vision Models 2026: LLaVA, Llama 3.2 Vision, Qwen3-VL & Ollama Multimodal Setup — PromptQuorum's broader comparison across current local vision models.

Related Reading

• Local Vision Models 2026: LLaVA, Llama 3.2 Vision, Qwen3-VL & Ollama Multimodal Setup — the multi-model comparison hub this review complements with LLaVA-specific depth.