Key Takeaways
- Gemma 4 31B is the prose pick for a 24 GB GPU. About 19–20 GB of weights, 256K context, Apache 2.0, and the only model in this set with its own published EQ-Bench Creative Writing v3 score (rubric 16.01, Elo 1407.6).
- Qwen3.8-27B is the long-context and instruction-following pick. About 18 GB on Ollama, 256K context, Apache 2.0. No creative-writing score is published for it yet, so treat prose quality as unverified and test it on your own scenes.
- Match the model to your VRAM, not to a leaderboard. 12 GB: Gemma 4 12B. 16 GB: Mistral Small 3 24B. 24 GB: Gemma 4 31B or Qwen3.8-27B. 48 GB or more: Llama 3.3 70B.
- Hermes 4.3 36B is the low-refusal pick. Apache 2.0, built on ByteDance Seed-OSS-36B, with Nous Research describing sharply reduced refusals. Dolphin 3.0 is the older alternative.
- Sampling matters more than the model switch. Temperature 0.9–1.1 and top-p 0.92–0.95 for prose; turn thinking mode off for fiction. Coding-style settings (0.2–0.4) produce flat prose.
- Closed cloud models still lead the creative-writing benchmarks. The top EQ-Bench Creative Writing v3 entry is Claude Opus 4.7 at 2,206 Elo; the best open-weight entry listed is Kimi K2.6 at 1,782, far beyond consumer hardware. Local wins on privacy, cost, and control.
Quick Facts
- 24 GB GPU or 32 GB Mac: Gemma 4 31B (
gemma4:31b, 19–20 GB) or Qwen3.8-27B (qwen3.8:27b, 18 GB). Both Apache 2.0, both 256K context. - 12 GB GPU: Gemma 4 12B (
gemma4:12b, 7.7–8.0 GB), 256K context. - 16 GB GPU: Mistral Small 3 24B (
mistral-small:24b, 14 GB), 32K context, Apache 2.0. Released 2025. - 48 GB or more: Llama 3.3 70B (
llama3.3:70b, about 42 GB at Q4_K_M). Released December 2024; Meta community license. - Low-refusal fine-tunes: Hermes 4.3 36B (Apache 2.0) and Dolphin 3.0 (Mistral 24B variant, February 2025).
- Prose sampling baseline: temperature 1.0, top-p 0.95, min-p 0.05; add a repeat or DRY penalty if scenes loop.
- Memory rule: Ollama sizes are weights only. Add 2–6 GB for the KV cache once you raise the context window for long scenes.
How These Picks Were Made
This ranking is an editorial synthesis of published sources, not a new PromptQuorum test run. We compared official model cards, the Ollama library (sizes, context, tags), and public creative-writing leaderboards, then matched each model to a hardware tier.
- Sizes and context: read from the Ollama library pages for Gemma 4, Qwen3.8, Mistral Small, and the Hugging Face model cards. Weights only; context memory is extra.
- Creative-writing scores: EQ-Bench Creative Writing v3 and the Open Writing Leaderboard, as mirrored by third-party trackers. Gemma 4 31B has its own score; Qwen3.8-27B does not yet. Where a score belongs to a sibling model, we say so.
- Release dates: Qwen3.8-27B (August 2026), Gemma 4 (July 2026), Mistral Small 3 (2025), Llama 3.3 70B (December 2024), Hermes 4.3 36B (December 2025), Dolphin 3.0 (February 2025).
- What we did not do: we have not run a fresh scored prompt set on these models. The per-model verdicts below follow the cited sources and the model cards; "best for" labels are recommendations, not measured results.
- For prompting techniques that improve creative output on any model, see temperature and top-p control and persona prompting.
📌Note: Creative-writing benchmarks are subjective and models disagree between boards. Treat every pick here as a starting hypothesis: run the same three scenes through two candidates and keep the one whose voice you prefer.
Best Local Model by VRAM Tier
Pick by memory first: the model has to fit with room left for context. Sizes are Ollama download sizes at the default 4-bit quantization; add 2–6 GB for the KV cache on long scenes.
Your memory | Model | Weights | Context |
|---|---|---|---|
| 8 GB GPU | Qwen3.5 9B (qwen3.5:9b) | 6.6–7.6 GB | 256K |
| 12 GB GPU | Gemma 4 12B (gemma4:12b) | 7.7–8.0 GB | 256K |
| 16 GB GPU | Mistral Small 3 24B (mistral-small:24b) | 14 GB | 32K |
| 24 GB GPU / 32 GB Mac | Gemma 4 31B (gemma4:31b) | 19–20 GB | 256K |
| 24 GB GPU / 32 GB Mac | Qwen3.8-27B (qwen3.8:27b) | 18 GB | 256K |
| 24 GB GPU, faster output | Gemma 4 26B-A4B (gemma4:26b) | 16–19 GB | 256K |
| 48 GB or more | Llama 3.3 70B (llama3.3:70b) | ~42 GB | 128K |
💡Tip: On a 32 GB Mac, unified memory is shared with macOS. Plan for roughly 24 GB usable for the model plus context, which keeps Gemma 4 31B and Qwen3.8-27B comfortable.
Head-to-Head: Local Models for Creative Writing
No single model wins every task; pick by hardware tier and by what is load-bearing in your work. The table lists the models in this guide with size, weights, and release date.
📍 In One Sentence
Gemma 4 31B and Qwen3.8-27B are the best local picks for creative writing on a 24 GB GPU; Gemma 4 12B, Mistral Small 3 24B, and Llama 3.3 70B cover the 12 GB, 16 GB, and 48 GB tiers; Hermes 4.3 36B is the low-refusal option.
💬 In Plain Terms
Choose the biggest model that fits your memory with room for context. On a 24 GB card that is Gemma 4 31B for prose or Qwen3.8-27B for long projects. Add a low-refusal fine-tune only if the instruct model refuses scenes you need to write.
Model | Size | Weights (Ollama) | Released | Best for |
|---|---|---|---|---|
| Gemma 4 31B | 31B dense | 19–20 GB | Jul 2026 | Prose on 24 GB; has own EQ-Bench score |
| Qwen3.8-27B | 27B dense | 18 GB | Aug 2026 | Long projects, instruction-following |
| Gemma 4 26B-A4B | 26B MoE, 4B active | 16–19 GB | Jul 2026 | Faster generation on 24 GB |
| Gemma 4 12B | 12B | 7.7–8.0 GB | Jul 2026 | 12 GB GPUs, long context |
| Mistral Small 3 24B | 24B dense | 14 GB | 2025 | 16 GB GPUs; older but proven |
| Llama 3.3 70B | 70B dense | ~42 GB | Dec 2024 | 48 GB or more; older reference |
| Hermes 4.3 36B | 36B dense | ~21–23 GB (est.) | Dec 2025 | Fewer refusals, Apache 2.0 |
| Dolphin 3.0 | 24B (Mistral variant) | ~14 GB | Feb 2025 | Older low-refusal option |
💡Tip: Two-model setup is the common pattern: one instruct model as the daily driver, plus Hermes 4.3 36B for scenes the instruct model refuses. Both can sit in ollama list at the same time.
Per-Model Verdicts
- Gemma 4 31B — prose pick for 24 GB. It is the only model in this set with its own EQ-Bench Creative Writing v3 result (rubric 16.01, Elo 1407.6, as reported by a third-party tracker). 256K context and Apache 2.0. Google lists temperature 1.0, top-p 0.95, top-k 64 as its default.
- Qwen3.8-27B — long-project pick. 262,144 tokens of native context (extensible to 1M with YaRN), Apache 2.0, about 18 GB on Ollama. Alibaba positions it for coding, agents, and research; the model card says nothing about creative writing and no creative score is published, so verify voice on your own scenes. Thinking mode is on by default; turn it off for prose.
- Gemma 4 26B-A4B and 12B — same family, different trade-offs. The 26B mixture-of-experts model activates about 4B parameters per token, so it generates faster than the 31B at similar memory. The 12B fits 12 GB cards and still offers 256K context.
- Mistral Small 3 24B — the 16 GB option. Apache 2.0, about 14 GB, 32K context. It is a 2025 model and the oldest recommendation here other than Llama; it stays on the list because few newer models fit 16 GB. Its successor, Mistral Small 4 (March 2026, Apache 2.0), is a 119B mixture-of-experts model with about 6B active parameters, far beyond 16 GB.
- Llama 3.3 70B — the 48 GB reference. A December 2024 model under the Meta community license. For machines with 48 GB or more it remains a strong, widely supported choice; below that tier it is not practical.
- Hermes 4.3 36B — low-refusal pick. Built on ByteDance Seed-OSS-36B under Apache 2.0, with GGUF quantizations on Hugging Face. Nous Research describes sharply reduced refusals; Hermes 4.3 answered 72–75% of prompts on its own RefusalBench. Its card suggests temperature 0.6, top-p 0.95, top-k 20; raise temperature for prose.
- Dolphin 3.0 — older low-refusal option. The newest Dolphin generation we could confirm is Dolphin 3.0 (February 2025, including a Mistral 24B variant). Use it when you want a Mistral-family uncensored model; Hermes 4.3 is the more recent choice.
📌Note: Voice consistency is the metric most writers underweight. A model that writes one brilliant paragraph but loses the character's speech pattern by paragraph five is worse than a steady one. Test long-session continuity, not just one-shot quality.
Best Ollama Models for Creative Writing
Five ollama pull commands cover every hardware tier in this guide. Run ollama list afterwards to confirm the model is installed.
- 24 GB, prose:
ollama pull gemma4:31b— thenollama run gemma4:31b. - 24 GB, long projects:
ollama pull qwen3.8:27b— run withollama run qwen3.8:27b --think=falseto switch thinking off for prose. - 12 GB:
ollama pull gemma4:12b. - 16 GB:
ollama pull mistral-small:24b. - 48 GB or more:
ollama pull llama3.3:70b. - Low-refusal fine-tune:
ollama run hf.co/NousResearch/Hermes-4.3-36B-GGUFpulls the Hermes 4.3 GGUF from Hugging Face; choose a 4-bit quant on a 24 GB card. - Raise the context window for long scenes. Ollama's default context is small. In an interactive session type
/set parameter num_ctx 16384, or setnum_ctxin a Modelfile. Larger values use more VRAM.
Local vs Cloud Models for Writing
Closed cloud models score higher on creative-writing benchmarks; local models win on privacy, cost, and control. If a reader searches for "best LLM for writing" without "local", the leaderboards answer differently from this guide.
- Closed models lead. On EQ-Bench Creative Writing v3, as mirrored by evals.report, Claude Opus 4.7 sits first at 2,206 Elo, ahead of GPT-5.5 at 2,035.
- The best open-weight entries need a datacenter. Kimi K2.6 (1,782 Elo) is the highest open-weight entry on that list. Featherless lists GLM-5.3 at 753B parameters and DeepSeek-V4.1-Flash at 763B, which no consumer machine runs.
- What runs on one GPU. Gemma 4 31B scored 1,407.6 Elo and a 16.01 rubric on the same benchmark. Scores from different trackers do not share a scale, so compare within one list only.
- When local is the right call. Unpublished manuscripts, client work under NDA, content a cloud provider would refuse, offline work, and projects where a monthly API bill is the constraint.
- Hybrid workflow. Draft and brainstorm locally; use a cloud model only for non-sensitive final polish if you want the extra quality.
Temperature and Top-P for Creative Work
Creative writing wants higher sampling temperatures than coding does. Defaults in most chat UIs are tuned for question answering; a temperature of 0.7 produces flat, predictable prose on creative prompts.
- Vendor defaults are a floor, not a target. Google lists temperature 1.0, top-p 0.95, top-k 64 for Gemma 4; Alibaba lists 1.0 / 0.95 / top-k 20 for Qwen3.8 in thinking mode and 0.7 / 0.8 in non-thinking mode; Nous lists 0.6 / 0.95 / top-k 20 for Hermes 4.3. Start there, then tune for prose.
- Baseline for prose: temperature 0.95–1.0, top-p 0.92–0.95, repeat penalty 1.05–1.1. Adjust per task from here.
- Tight dialogue: temperature 0.7–0.85, top-p 0.9. Lower temperatures keep character voices consistent across exchanges.
- Lyrical prose and poetry: temperature 1.0–1.2, top-p 0.95. Higher temperatures unlock unexpected word choices.
- Surreal or genre fiction: temperature 1.1–1.3, top-p 0.95–0.98.
- Plot-driven scenes (action, mystery, twists): temperature 0.85–0.95, top-p 0.9. These want direction-following more than novelty.
- Turn thinking mode off for fiction. Reasoning tokens add latency and tend to produce planned, tidy prose. Qwen3.8 accepts
enable_thinking: false; in Ollama use--think=false. - min_p (0.05–0.1): an alternative to top-p that scales the cutoff to the top token's probability. It is more permissive on creative prompts without the incoherence of a very high top-p. Use it where your backend exposes the setting.
- DRY repetition penalty (multiplier 0.8, base 1.75, allowed length 2): catches phrase-level repetition that a standard repeat penalty misses, which helps long sessions. It is available in llama.cpp-based backends such as KoboldCpp; check whether your frontend exposes it.
- For a fuller treatment of why these parameters matter and how they interact, see temperature and top-p control.
💡Tip: Test sampling on one short scene per model: three runs at each setting, then pick the temperature where the model sounds most alive without losing the prompt. Settings do not transfer perfectly between models.
Uncensored Models: What They Are and When They Matter
Uncensored does not mean unethical. It means the model's refusal behaviour has been reduced by fine-tuning, so it follows the prompt instead of declining. The writer is still the author.
- What it means technically. Models such as Hermes 4.3 and Dolphin 3.0 are fine-tunes whose post-training reduces refusals on mature, violent, or morally complex prompts. Nous Research describes sharply reduced refusals and reports that Hermes 4.3 answered 72–75% of prompts on its own RefusalBench. Community "abliterated" or "heretic" variants of newer models also exist on Ollama; their quality is unaudited.
- When they matter for creative work. Mature fiction, crime fiction with graphic violence, horror, historically accurate writing about war or atrocity, and morally complex antagonists the instruct model will not voice convincingly.
- Where they fall short. They follow the prompt, including badly written prompts. Instruct models often soften a vague prompt into something publishable; low-refusal models give you exactly what you asked for.
- Ethical and legal boundaries. A model that will write something is not a licence to write content that targets real people, depicts non-consensual scenarios involving identifiable individuals, or is illegal where you live. Local hosting does not change the law; it changes who can see the draft. Laws differ by country, and this is not legal advice.
- For a longer treatment of ethics, legality, and best practice, see Uncensored Local LLMs for Creative Writing: Ethics, Legality & Best Practices.
📌Note: Uncensored is a workflow choice, not an identity. Many writers draft with the instruct model and switch to a low-refusal fine-tune only for specific scenes. Two models installed side by side is the usual pattern.
Frontends for Creative Work
The chat UI you write in matters as much as the model. Four options cover most workflows; pick by workflow shape.
- Open WebUI — the general-purpose pick. ChatGPT-style interface, one-click model switching, system-prompt character notes, document upload for context. Best for prose-heavy drafting.
- SillyTavern — the roleplay and dialogue pick. Character cards, persona management, lorebooks for worldbuilding, and group chat for multi-character scenes. Best for dialogue-driven work and long-running world projects.
- Agnai and RisuAI — lighter SillyTavern alternatives with easier first run and less customisation.
- Plain Ollama CLI plus a text editor — the minimal pick.
ollama run gemma4:31band paste scenes into your draft. It loses persistent character context but wins on focus. - For the head-to-head comparison of the roleplay-focused frontends, see SillyTavern vs Agnai vs RisuAI: Best Local Roleplay Frontend.
💡Tip: Drafting and editing want different tools. Generate in the chat frontend, then export to a plain text editor for revision. Editing inside the chat window teaches you to ask the model to fix prose instead of revising it yourself.
Decision: Which Model for Your Work
Match your memory first, then your task.
📍 In One Sentence
Pick Gemma 4 31B or Qwen3.8-27B on a 24 GB GPU, Gemma 4 12B on 12 GB, Mistral Small 3 24B on 16 GB, Llama 3.3 70B on 48 GB or more, and Hermes 4.3 36B for scenes the instruct models refuse.
💬 In Plain Terms
Start with the largest model that fits your memory with room for context. Switch to a specialist only when one task, such as long-form continuity or mature scenes, becomes the bottleneck. Two installs cost nothing but disk space.
Your situation | Pick |
|---|---|
| 24 GB GPU or 32 GB Mac, prose-first | Gemma 4 31B |
| 24 GB GPU, novel-length project | Qwen3.8-27B (256K context) |
| 24 GB GPU, want faster output | Gemma 4 26B-A4B |
| 12 GB GPU | Gemma 4 12B |
| 16 GB GPU | Mistral Small 3 24B |
| 48 GB or more, one model for everything | Llama 3.3 70B + Hermes 4.3 36B |
| Instruct model refuses scenes I need | Hermes 4.3 36B (keep an instruct model too) |
| I want one model to start and will iterate | Gemma 4 31B or Qwen3.8-27B, whichever fits |
💡Tip: Most writers overthink the model and underthink the prompt. A system prompt with character notes, voice samples, and explicit POV does more than switching models. See persona prompting.
Common Mistakes Picking and Using Local Models for Creative Writing
- Mistake 1: picking by general benchmark rank. Coding and reasoning leaderboards correlate poorly with prose quality. Look at creative-writing boards such as EQ-Bench, then test on your own scenes.
- Mistake 2: using coding-style sampling. Temperature 0.2–0.4 produces flat prose. Creative work wants roughly 0.9–1.1.
- Mistake 3: leaving thinking mode on. Reasoning-first models plan before they write, which can make fiction feel assembled. Turn thinking off for prose.
- Mistake 4: ignoring context memory. A 31B model at 19–20 GB leaves little room on a 24 GB card once you raise the context window. Budget 2–6 GB for the KV cache.
- Mistake 5: thin system prompts. "You are a helpful assistant" is the worst prompt for creative work. Pair character notes, POV, and tense with negative prompting to rule out clichés.
- Mistake 6: editing inside the chat window. Generate in chat; revise in an editor. Writers who let the model revise lose their voice quickly.
⚠️Warning: The biggest skill risk with creative-writing AI is outsourcing the revision pass. Generation is mechanical work that benefits from a model; revision is what makes the prose yours.
Sources
- Ollama library: Gemma 4 — tags, download sizes, and context windows.
- Ollama library: Qwen3.8 — tag, size, and capabilities.
- Hugging Face: Qwen3.8-27B model card — license, context length, recommended sampling.
- Hugging Face: Gemma 4 31B model card — license, context length, sampling defaults.
- Hugging Face: Hermes 4.3 36B — base model, license, refusal benchmark, GGUF quantizations.
- Ollama library: Mistral Small — Mistral Small 3 24B size, context, and license.
- EQ-Bench Creative Writing and evals.report EQ-Bench scores — creative-writing leaderboard figures cited above.
- Featherless: best open models for creative writing — open-weight sizes and Gemma 4 31B EQ-Bench figures.
- SillyTavern documentation — character cards, personas, group chat.
Frequently Asked Questions
What is the best local LLM for creative writing?
Gemma 4 31B and Qwen3.8-27B are the best local starting points in 2026, and both fit a 24 GB GPU (about 18–20 GB of weights). Gemma 4 31B is the prose pick with its own published EQ-Bench score; Qwen3.8-27B is the long-context pick. On smaller cards use Gemma 4 12B (12 GB) or Mistral Small 3 24B (16 GB).
What is the best Ollama model for creative writing?
On a 24 GB GPU, ollama pull gemma4:31b for prose or ollama pull qwen3.8:27b for long projects. On 12 GB, gemma4:12b; on 16 GB, mistral-small:24b; on 48 GB or more, llama3.3:70b. Raise num_ctx for long scenes and turn thinking off for fiction.
What is the best local LLM for writing fiction and stories?
Pick the largest model that fits your memory with room for context. Gemma 4 31B is the prose pick on 24 GB, Qwen3.8-27B suits novel-length projects with its 256K context, and Llama 3.3 70B is the reference on 48 GB or more. Use temperature 0.9–1.1 and a system prompt with POV, tense, and voice samples.
Which local model is best for worldbuilding?
Long context matters most: Qwen3.8-27B and Gemma 4 both offer 256K tokens, enough to keep a lore bible and chapter summaries in the prompt. Pair either with SillyTavern lorebooks or a pinned world document, and keep factions, rules, and names in one reference block.
Which LLM is best for writing a novel?
For local use, Qwen3.8-27B (256K context) or Gemma 4 31B (256K context) on a 24 GB GPU. The bottleneck is structure, not model size: an outline, a character bible, and chapter summaries keep continuity. Closed cloud models score higher on creative benchmarks, but they send your manuscript to a third party.
What is the best local model with 32 GB of VRAM or unified memory?
Gemma 4 31B (19–20 GB) or Qwen3.8-27B (18 GB) with a larger context window; the extra memory over 24 GB pays for the KV cache. Gemma 4 26B-A4B is the faster alternative. A 70B model does not fit comfortably below 48 GB.
What is an uncensored local LLM and when should I use one?
It is a fine-tune whose post-training reduces refusals on mature or morally complex prompts, so the model follows the prompt instead of declining. Hermes 4.3 36B is the most recent option we could confirm; Dolphin 3.0 is older. Use one for mature fiction or historically accurate writing, and keep an instruct model for everything else.
What temperature should I use for creative writing?
Temperature 0.9–1.1 with top-p 0.92–0.95 for prose. Tight dialogue wants 0.7–0.85; poetry 1.0–1.2; surreal or genre work 1.1–1.3. Vendor defaults such as Gemma 4's 1.0 / 0.95 / top-k 64 are a sound starting point. Turn thinking mode off for fiction.
Are local creative-writing models as good as ChatGPT or Claude?
Not on benchmarks. On EQ-Bench Creative Writing v3, closed models such as Claude Opus 4.7 (2,206 Elo) lead, and the best open-weight entries need datacenter hardware. A 24 GB local model can still be good enough for drafting, and it keeps manuscripts private. Calibrated sampling and a strong system prompt close much of the practical gap.
Can a local model write a full novel?
It can help draft one. A 256K-context model such as Qwen3.8-27B or Gemma 4 can hold a long outline and several chapters in memory, but the KV cache grows with context, so check VRAM. Draft section by section with a character bible and a lorebook; without them, any model drifts.
Do uncensored models produce illegal content?
They produce the text the prompt asks for, and legal responsibility sits with the writer and the publication, not the model. Laws such as the EU AI Act, German StGB §184/§184c, and national obscenity rules apply regardless of how text was generated. Local hosting changes who sees the draft, not what is legal. This is not legal advice.
How do I avoid the "shiver down their spine" cliché?
List 8–12 banned phrases in the system prompt ("do not write 'shiver,' 'tapestry,' 'delve,' or 'she felt'"). Keep temperature at 0.85–1.0 rather than 1.2+, and add a DRY penalty where your backend supports it; a plain repeat penalty does not catch phrase-level clichés. A manual revision pass is the final filter. See negative prompting.