Why Local Coding Models Caught Up
For the first years of the LLM era, cloud models were clearly ahead of local models on coding tasks. That gap narrowed in 2025–2026 as open-weight models reached the 24–30B range with coding-focused training.
Qwen3.6 27B, released in April 2026, reports 77.2 on SWE-bench Verified, a benchmark of real GitHub issues in open-source Python projects. In the same table on its model card, Qwen lists 75.0 for Qwen3.5-27B and 80.9 for Claude 4.5 Opus, so the open model sits close to a leading cloud model on this vendor-reported test. These are Qwen's own results from an internal agent scaffold, not an independent verification.
Commonly cited reasons for the convergence are (1) coding-focused training data at scale, (2) post-training on real software-engineering tasks rather than generic instruction following, and (3) better GGUF quantization that preserves coding ability at Q4 precision.
📍 In One Sentence
Qwen3.6 27B reports 77.2% on SWE-bench Verified and runs locally from an 18–19 GB download.
💬 In Plain Terms
SWE-bench Verified tests whether an AI can resolve real GitHub issues in open-source projects such as Django, Flask and scikit-learn. A score of 77.2% means the model resolved about 77 of every 100 test issues, measured with the vendor's own agent setup.
Benchmark Table
All figures are vendor-reported from model cards and Ollama listings. Test setups differ between vendors, so treat SWE-bench numbers as indicative, not as a controlled head-to-head. PromptQuorum has not re-run these benchmarks.
Benchmark | Qwen3.6 27B | DeepSeek V4 Flash | Devstral Small 2 | Codestral 22B |
|---|---|---|---|---|
| SWE-bench Verified (vendor-reported) | 77.2% | 78.6% (High) / 79.0% (Max) | 65.8% | Not published |
| Parameters (total / active) | 27B dense | 284B / 13B active | 24B dense | 22B dense |
| Context window | 256K | 1M | 384K | 32K |
📌Note: Qwen's 77.2 uses Qwen's internal agent scaffold. DeepSeek reports 73.7 (non-thinking), 78.6 (High) and 79.0 (Max) modes. Devstral Small 2's 65.8 comes from its Ollama listing. Different scaffolds mean the numbers are not directly comparable.
💡Tip: DeepSeek's API model names change. Check api-docs.deepseek.com/quick_start/pricing before deployment (current names: deepseek-flash and deepseek-v4-pro).
Per-Token Cost Math
The economics of coding LLMs depend on usage volume, task sensitivity and hardware. The projection below is for one developer and rests on the stated assumptions; it is arithmetic, not a measurement. Electricity is priced at €0.35/kWh, a typical German rate; check your own tariff.
At 5M tokens per day (a heavy session of autocomplete, test generation and code review) with a 1:2 input-to-output split, the DeepSeek API (deepseek-flash, $0.15 input and $0.60 output per 1M tokens off-peak) costs about $2.25 per day, or roughly $560 over 250 working days. At peak-hour rates ($0.30 input, $1.20 output) the same usage costs about $4.50 per day, or roughly $1,125 per year.
A local setup trades that for hardware and power. An RTX 4090 at an assumed $2,300, drawing 350 W for 8 hours on 250 days, uses 700 kWh, about €245 a year. Ignoring exchange rates, the hardware pays back against off-peak API rates in roughly 7 years and against peak rates in roughly 2.6 years. For sensitive code the comparison is not only about cost: a cloud API sends your code to a third party.
# Cost calculator: per-token math for coding LLMs (every input is an assumption)
# DeepSeek API, deepseek-flash, off-peak rates in USD per 1M tokens (api-docs.deepseek.com)
input_rate, output_rate = 0.15, 0.60
daily_tokens = 5_000_000 # per developer
blended = (input_rate * 1 + output_rate * 2) / 3 # 1:2 input:output split = $0.45/1M
daily_cost = daily_tokens / 1_000_000 * blended # $2.25/day
annual_api = daily_cost * 250 # ~$562/year (peak rates: ~$1,125/year)
# Local Qwen3.6 27B on an RTX 4090 (price is an assumption)
hardware_cost = 2300 # USD
power_kwh = 0.350 * 8 * 250 # 350 W, 8 h/day, 250 days = 700 kWh
annual_power = power_kwh * 0.35 # EUR 0.35/kWh = ~245 per year
# Break-even vs off-peak API: hardware_cost / (annual_api - annual_power) = ~7.3 years
# Break-even vs peak API: hardware_cost / (1125 - annual_power) = ~2.6 yearsLatency Reality
Latency matters for interactive coding: autocomplete feels broken above 500 ms, code review is acceptable up to 3 s, batch jobs are latency-insensitive. PromptQuorum has not measured these models. The table shows one third-party Ollama test on an RTX 4090 (24 GB, Ollama 0.34.4, September 2026), the only published run we found for these models; your speeds will differ with quantization, context length and drivers.
Setup | Sustained (tok/sec) | Notes |
|---|---|---|
| Qwen3.6 27B, RTX 4090 (draft decoding on) | 115.4 | Mean of five prompts, 256-token output |
| Qwen3.6 27B, RTX 4090 (draft decoding off) | 48.9 | Single-prompt rerun, same setup |
| Qwen3-Coder 30B, RTX 4090 | 224.7 | MoE with 3.3B active parameters |
| Devstral Small 2 | Not benchmarked | No published run found |
| DeepSeek API (deepseek-flash) | Not benchmarked | Depends on network and peak hours |
Source: Ollama Models Cheat Sheet (third-party test, not by PromptQuorum). According to the article, speculative (draft-token) decoding made Qwen3.6 27B about 2.4× faster in that test without changing output quality. Apple does not publish token speeds for local LLMs. Run ollama run --verbose on your own hardware for a definitive number. For autocomplete, local inference avoids the network round trip.
⚠️Warning: Ollama's default context window (scaled to available VRAM) raises apparent throughput because fewer tokens are processed, but it truncates context. Set num_ctx 32768 for meaningful coding latency measurements.
Hardware Requirements
- Qwen3.6 27B Q4_K_M (Ollama `qwen3.6:27b`, 18–19 GB download): a 24 GB GPU (RTX 3090, RTX 4090) or Apple Silicon with 36–48 GB or more of unified memory. A 16 GB card cannot hold the weights.
- Devstral Small 2 24B (Ollama `devstral-small-2`, 15 GB download): RTX 4070 Ti Super (16 GB, tight once context is added), RTX 3090 or RTX 4090 (24 GB), or Apple Silicon with 32 GB or more of unified memory.
- Codestral 22B Q4_K_M (13 GB download): RTX 4070 Ti (12 GB marginal, 16 GB recommended).
- Running two models at once: Qwen3.6 27B (18–19 GB) plus Devstral Small 2 (15 GB) is about 34 GB before context, so it needs two 24 GB GPUs or Apple Silicon with 64 GB or more of unified memory, for example a Mac Studio with M5 Max (up to 128 GB).
- Apple Silicon: Apple refreshed the Mac mini and Mac Studio lineup on August 25, 2026 (available from September 22, 2026), replacing the M4 Pro/M4 Max configurations. Per Apple, the Mac mini with M5 Pro starts at $1,699 (up to 64 GB unified memory, 307 GB/s), the Mac Studio with M5 Max at $2,499 (up to 128 GB) and the Mac Studio with M5 Ultra at $5,499 (up to 512 GB; the 512 GB configuration ships in late October). A 36–48 GB configuration holds Qwen3.6 27B; 64 GB or more holds Qwen3.6 27B and Devstral Small 2 together. Apple publishes no tokens-per-second figures for local LLMs and PromptQuorum has not benchmarked these chips, so measure with ollama run --verbose. Prices are US dollars; local prices may differ.
- Budget Apple Silicon option: the entry-level Mac mini with M6 starts at $899 (up to 32 GB unified memory, up to 170 GB/s per Apple) and fits Devstral Small 2 (15 GB) and Codestral 22B (13 GB) comfortably. Qwen3.6 27B (18–19 GB) fits only in the 32 GB configuration, with little room for context. Its memory bandwidth is lower than the M5 Pro's (170 versus 307 GB/s), which means slower generation; PromptQuorum has not benchmarked the M6.
# Ollama config for Qwen3.6 27B with num_ctx and GPU layers
cat > Modelfile-qwen36 <<'EOF'
FROM qwen3.6:27b
PARAMETER num_ctx 32768
PARAMETER num_gpu 1
PARAMETER num_thread 8
PARAMETER temperature 0.2
SYSTEM "You are an expert software engineer. Respond with clean, well-structured code."
EOF
ollama create qwen36-local -f Modelfile-qwen36
ollama run qwen36-localMulti-Model Dispatch Strategy
No single coding model wins every task. Qwen3.6 27B has the strongest local SWE-bench figure here. Devstral Small 2 is a lighter agentic model. A DeepSeek API is the cheapest per token at scale for non-sensitive code. A dispatch layer that routes tasks by type captures the benefits of all three.
A suggested dispatch matrix for a development team:
Task Type | Recommended Model | Why |
|---|---|---|
| Private code (client data) | Qwen3.6 27B (local) | Code stays on your hardware |
| Autocomplete (interactive) | Devstral Small 2 (local) | Lighter model, no network round trip |
| Code review (non-sensitive) | DeepSeek API | $0.15/1M input off-peak, high throughput |
| Complex refactoring (multi-file) | Qwen3.6 27B (local) + PromptQuorum consensus | Strongest local SWE-bench figure, code stays local |
| Batch test generation | DeepSeek API | Cost-optimised for non-sensitive volume |
PromptQuorum Integration
PromptQuorum routes code tasks across local Qwen, local Devstral, and cloud APIs based on task classification rules you define. This eliminates manual model switching and implements the dispatch matrix above automatically.
📍 In One Sentence
PromptQuorum routes coding tasks to local Qwen3.6 for sensitive code and to a DeepSeek API for non-sensitive bulk generation.
# PromptQuorum routing config for coding workloads
# Set in your PromptQuorum settings or .env file
# Local models (via Ollama)
LOCAL_OLLAMA_URL=http://localhost:11434/v1
LOCAL_CODING_MODEL=qwen36-local # Qwen3.6 27B with num_ctx 32768
LOCAL_AUTOCOMPLETE_MODEL=devstral-small-2 # Devstral Small 2 24B
# Cloud fallback
DEEPSEEK_API_KEY=your_key_here
DEEPSEEK_MODEL=deepseek-flash
# Routing rules (PromptQuorum dispatch)
# route: task_contains("private") OR task_contains("customer") → qwen36-local (local)
# route: task_type == "autocomplete" → devstral-small-2 (local)
# route: token_count > 50000 → deepseek-flash (cloud, non-sensitive only)
# default → qwen36-local (local)Frequently Asked Questions
Is Qwen3.6 27B better than DeepSeek for local coding?
Qwen3.6 27B reports 77.2% on SWE-bench Verified and runs fully on your own hardware from an 18–19 GB download, so plan for a 24 GB GPU. DeepSeek V4 Flash reports 78.6% (High mode) but is a 284B-parameter model, so most people use it through the cloud API at $0.15 per 1M input tokens off-peak. The better choice depends on data sensitivity and budget, not on a single winner. Both scores are vendor-reported with different test setups.
What is Devstral Small 2 and why is it mentioned here?
Devstral Small 2 is a 24B model from Mistral AI. The Devstral line targets agentic software-engineering work such as multi-file edits and tool use. Its Ollama listing shows a 15 GB download, a 384K context window, Apache 2.0 licensing and 65.8% on SWE-bench Verified. It is the lighter local option here; Qwen3.6 27B reports the higher SWE-bench figure.
Can I run Qwen3.6 27B and Devstral Small 2 at the same time?
Not on a single 24 GB GPU. Qwen3.6 27B downloads at 18–19 GB and Devstral Small 2 at 15 GB, about 34 GB combined before context. You would need two 24 GB GPUs or Apple Silicon with 64 GB or more of unified memory. On one GPU, run one model at a time and switch with Ollama.
Is a DeepSeek API safe for EU company code?
The DeepSeek API processes data on servers operated by DeepSeek, a company incorporated in China, and the EU Commission has not issued an adequacy decision for China. Sending EU personal data, or source code containing personal information, to it requires a legal assessment under GDPR Chapter V (Articles 44 ff.). For proprietary code without personal data, consult your legal team. Running a local model such as Qwen3.6 27B keeps code on your own hardware, which avoids a third-party transfer; it does not by itself make a project GDPR-compliant. This is general information, not legal advice.
What is SWE-bench Verified and why focus on it?
SWE-bench Verified tests whether a model can resolve real GitHub issues in open-source Python projects such as Django, Flask and scikit-learn, using a human-validated subset of tasks. It measures practical software engineering more than isolated function-level tests such as HumanEval. Scores depend heavily on the agent scaffold, so compare vendor numbers with care.