How This Ranking Is Made (and What It Is Not)
This is an editorial ranking compiled from model cards, the Ollama library and public leaderboards. PromptQuorum did not run a new scored writing test for it. Creative writing has no single right answer, so benchmarks measure it imperfectly and leaderboards disagree.
- Sizes and context: read from the Ollama library pages for Gemma 4, Qwen3.8, Qwen3.5, Mistral Small and Llama 3.3. Sizes are the download sizes of the default 4-bit builds; context memory is extra.
- Licences and sampling defaults: read from the Hugging Face model cards listed under Sources.
- Creative-writing scores: EQ-Bench Creative Writing v3, an LLM-judged benchmark. The Gemma 4 figures come from a Featherless article dated 2026-09-17; Qwen3.8-27B has no published score. Scores from different trackers are not directly comparable, so compare within one list only.
- "Best for" labels are editorial recommendations built on these sources, not measured results.
- Test any pick yourself with four prompts: continue a scene for 500 words (tone and voice), write a paragraph in a named author's style (does the style shift?), write a 1,000-word story with a twist ending (setup and payoff), and write a two-character dialogue (are the voices distinct?).
- For frontends, worldbuilding and low-refusal models in more depth, see the extended creative writing guide.
📌Note: Treat every pick as a starting hypothesis: run the same three scenes through two candidates and keep the one whose voice you prefer.
Best Local AI Model for Creative Writing by Memory Tier
Pick by memory first: the model has to fit with room left for context. Sizes are Ollama download sizes at the default 4-bit quantization; context memory (the KV cache) is extra, roughly 2–6 GB for long scenes (estimate).
Your memory | Model (Ollama tag) | Weights | Context |
|---|---|---|---|
| 8 GB GPU | Qwen3.5 9B (qwen3.5:9b) | 6.6–7.6 GB | 256K |
| 12 GB GPU | Gemma 4 12B (gemma4:12b) | 7.7–8.0 GB | 256K |
| 16 GB GPU | Mistral Small 3 24B (mistral-small:24b) | 14 GB | 32K |
| 24 GB GPU / 32 GB Mac | Gemma 4 31B (gemma4:31b) | 19–20 GB | 256K |
| 24 GB GPU / 32 GB Mac | Qwen3.8-27B (qwen3.8:27b) | 18 GB | 256K |
| 24 GB GPU, faster output | Gemma 4 26B A4B (gemma4:26b) | 16–19 GB | 256K |
| 48 GB GPU / 64 GB Mac | Llama 3.3 70B (llama3.3:70b) | 43 GB | 128K |
💡Tip: On a Mac, unified memory is shared with macOS. On a 32 GB Mac plan for roughly 24 GB usable for model plus context, which fits Gemma 4 31B or Qwen3.8-27B with only a few GB left for context.
#1 Gemma 4 31B: Prose Pick for a 24 GB GPU
Gemma 4 31B is our first pick for prose on a 24 GB GPU or a 32 GB Mac. It is a dense 30.7B model (19–20 GB on Ollama) with 256K context under Apache 2.0, and the only model in this guide with a published EQ-Bench Creative Writing v3 result: rubric 16.01, Elo 1407.6, as reported by Featherless.
Google's model card lists text generation such as poems and scripts as an intended use and recommends temperature 1.0, top-p 0.95, top-k 64. At 19–20 GB the model leaves little room for context on a 24 GB card, so keep the context window modest or choose Gemma 4 26B A4B (16–19 GB), a mixture-of-experts model that activates about 4B parameters per token and scored 1286.9 Elo on the same Featherless list.
Spec | Value |
|---|---|
| Best for | Prose on a 24 GB GPU |
| Weights (Ollama) | 19–20 GB |
| Context | 256K tokens |
| Licence | Apache 2.0 |
| Ollama command | ollama run gemma4:31b |
#2 Qwen3.8-27B: Long-Project Pick on the Same Hardware
Qwen3.8-27B is the pick for long projects on a 24 GB GPU. It is a dense 27B model at 18 GB on Ollama with 262,144 tokens of native context (extensible to 1,000,000 tokens per the model card) under Apache 2.0.
Alibaba positions it for coding, agents and research; the model card does not mention creative writing and no creative-writing score is published, so treat its prose as unverified. Thinking mode is on by default; switch it off for fiction.
Spec | Value |
|---|---|
| Best for | Novel-length projects, instructions |
| Weights (Ollama) | 18 GB |
| Context | 256K tokens |
| Licence | Apache 2.0 |
| Ollama command | ollama run qwen3.8:27b |
#3 Picks for 8, 12 and 16 GB
Below 24 GB the choice is smaller and less proven, so test before you commit. Smaller models generally keep long scenes consistent less reliably than larger ones.
- 8 GB: Qwen3.5 9B (`qwen3.5:9b`, 6.6–7.6 GB, 256K context). A recent model in the 8 GB class on Ollama. No creative-writing score is published for it, so run your own scenes.
- 12 GB: Gemma 4 12B (`gemma4:12b`, 7.7–8.0 GB, 256K context, Apache 2.0). Featherless lists a rubric of 15.71 and 1270.4 Elo for it on EQ-Bench Creative Writing v3.
- 16 GB: Mistral Small 3 24B (`mistral-small:24b`, 14 GB, 32K context, Apache 2.0). An older model that stays on the list because few newer models fit 16 GB. It leaves only about 2 GB for context, so keep prompts short.
#4 Llama 3.3 70B: Reference Model for 48 GB or More
Llama 3.3 70B (`llama3.3:70b`, 43 GB, 128K context) is the largest model in this guide and needs a 48 GB GPU setup or a Mac with 64 GB of unified memory. It was released by Meta in December 2024 under the Meta Llama 3.3 Community License, a custom licence rather than an OSI open-source licence.
Earlier versions of this page ranked it first for prose. We no longer do: no creative-writing score for it appears on the lists we read, while newer models that fit 24 GB do have one, and newer Llama releases score low on EQ-Bench (Llama 4 Maverick 927, Llama 4 Scout 883 on the evals.report list). Keep it if you have the memory and like its voice.
#5 Community and Low-Refusal Models
Community fiction models are now a second layer, not the default. Check each model's licence before commercial use.
- Hermes 4.3 36B (Nous Research, Apache 2.0): built on ByteDance Seed-OSS-36B-Base, with 4- to 8-bit GGUF files on Hugging Face. Nous reports that it answered 74.60% (non-reasoning) and 72.29% (reasoning) of prompts on its own RefusalBench (developer-reported; higher means fewer refusals). Use it when an instruct model declines a legitimate fiction scene, and raise temperature above the card's 0.6 for prose.
- Fimbulvetr-11B-v2 (Sao10K): a Solar-based model under CC BY-NC 4.0, so non-commercial use only. The card recommends SillyTavern presets. We found no creative-writing benchmark score for it.
- Midnight-Rose-70B (sophosympatheia): a merge built on Llama 2 70B, under the Llama 2 licence plus inherited restrictions; the card advises about 6144 tokens of context. It is not a Llama 3.3 model.
- If a GGUF conversion exists for an older model, you can load it in LM Studio or through `ollama run hf.co/<user>/<repo>`; check the licence of that specific repository first.
- For refusal behaviour, ethics and limits, see Uncensored Local LLMs for Creative Writing: Ethics, Legality & Best Practices. Local hosting does not change the law, and the writer stays responsible for what they publish.
Local vs Cloud Models for Writing
Closed cloud models score higher on creative-writing leaderboards; local models win on privacy, cost and control.
- On the EQ-Bench Creative Writing v3 list read via evals.report (data snapshot 2026-11-15), Claude Opus 4.7 leads at 2206 Elo, followed by GPT-5.5 at 2035.
- The best open-weight entry on that list is Kimi K2.6 at 1782 Elo. Featherless lists the open models it recommends at that level at 753B (GLM-5.3), 763B (DeepSeek-V4.1-Flash) and 2.8T (Kimi K3) parameters, far beyond the 8–48 GB tiers in this guide.
- When local is the right call: unpublished manuscripts, client work under NDA, offline work, and projects where per-token cost matters.
- Hybrid workflow: draft and brainstorm locally; use a cloud model only for non-sensitive polish, after checking the provider's data terms.
Settings and Prompts That Improve Local LLM Creative Writing
Five habits change output more than switching models.
- 1Set sampling for prose, not for code. Vendor defaults: Gemma 4 temperature 1.0, top-p 0.95, top-k 64; Qwen3.8 1.0 / 0.95 / top-k 20 in thinking mode and 0.7 / 0.8 / top-k 20 in non-thinking mode; Hermes 4.3 0.6 / 0.95 / top-k 20. As an editorial starting range, try temperature 0.9–1.1 for prose and 0.7–0.85 for tight dialogue.
- 2Turn thinking mode off for fiction. Reasoning models plan before they write, which can make scenes feel assembled. Qwen3.8 thinks by default; in the Ollama API set `think` to `false`, or use your frontend's thinking toggle.
- 3Name the style concretely. "Write in second person, present tense, short sentences, no adverbs" works better than "write literary fiction". Naming an author gives a style hint; do not publish text that reproduces a living author's wording.
- 4Use a persistent system prompt with point of view, tense, genre, a character sheet and voice samples, and re-paste it at the start of each session.
- 5Work in sections and raise the context window. Generate about 500 words at a time and re-state a short story summary in each new session, because a model only sees what fits in its context. Ollama's default context is limited; in an interactive session type `/set parameter num_ctx 16384`. Larger values use more memory.
Best Open-Weight Models for Poetry
No leaderboard scores poetry separately, so the poetry pick follows the memory-tier ranking above. For strict forms (sonnets, haiku, villanelles) lower the temperature to 0.7–0.85 (editorial range), because high temperatures tend to break meter and rhyme; for free verse the prose range applies.
- 24 GB: Gemma 4 31B or Qwen3.8-27B. Run the same short poem prompt on both and keep the voice you prefer.
- Smaller tiers: Gemma 4 12B (12 GB) or Qwen3.5 9B (8 GB).
- Strict forms: models work on tokens, not syllables, so count syllables and check rhyme by hand.
- State the form explicitly: "Write a haiku (5-7-5 syllables) about autumn rain" or "Write a Shakespearean sonnet (ABAB CDCD EFEF GG, iambic pentameter) about loss". Vague prompts such as "write a poem about love" produce generic free verse.
Bad Prompt vs Good Prompt
- ❌ "Write a fantasy story" → ✅ "Write a 500-word fantasy scene where a smuggler negotiates with a dragon over ancient artifacts. Use sensory details and make the dialogue tense."
- ❌ "Write something interesting" → ✅ "Write a 300-word opening scene of a heist gone wrong. The protagonist discovers their partner betrayed them mid-mission. Use short, punchy sentences to match the pace."
- ❌ "Write a mystery" → ✅ "Continue this detective scene: [previous text]. The detective realizes the suspect is lying based on one detail. Show, do not tell, how she catches the inconsistency."
- ❌ "Make it more interesting" → ✅ "Rewrite the previous paragraph as noir fiction: sparse dialogue, cynical internal monologue, specific sensory details (sounds, smells, textures)."
Frequently Asked Questions
What is the best local LLM for creative writing?
Gemma 4 31B and Qwen3.8-27B are the best local starting points in this ranking, and both fit one 24 GB GPU (19–20 GB and 18 GB of weights). Gemma 4 31B is the prose pick because it has a published EQ-Bench Creative Writing v3 score; Qwen3.8-27B is the long-project pick. On smaller cards use Gemma 4 12B (12 GB) or Mistral Small 3 24B (16 GB). This is an editorial ranking, so test two candidates on your own scenes.
What is the best local LLM for story writing and novels?
Pick the largest model that fits your memory with room for context. Gemma 4 31B suits prose on 24 GB, Qwen3.8-27B suits novel-length projects with its 256K context, and Llama 3.3 70B is the reference on 48 GB or more. The bottleneck is usually structure: an outline, a character sheet and short chapter summaries keep a long project consistent.
Which Ollama model is best for creative writing?
On 24 GB, `ollama run gemma4:31b` for prose or `ollama run qwen3.8:27b` for long projects. On 12 GB, `gemma4:12b`; on 16 GB, `mistral-small:24b`; on 8 GB, `qwen3.5:9b`; on 48 GB or more, `llama3.3:70b`. Raise `num_ctx` for long scenes and turn thinking off for fiction.
Which local LLMs work best for writing with only 8 GB of VRAM?
Qwen3.5 9B (6.6–7.6 GB, 256K context) is a recent model in the 8 GB class on Ollama. No creative-writing score is published for it, so run your own test scenes, and expect smaller models to hold long scenes together less reliably than the 24 GB models.
What are the best open-source models for poetry?
Most models people call open source are open-weight releases, and no leaderboard scores poetry separately, so use the memory-tier ranking: Gemma 4 31B or Qwen3.8-27B on 24 GB, Gemma 4 12B on 12 GB, Qwen3.5 9B on 8 GB. For strict forms such as sonnets or haiku, lower the temperature to 0.7–0.85 and check syllables and rhyme by hand.
Which model should I load in LM Studio for creative writing?
The same ranking applies. LM Studio and Ollama both run GGUF quantized files, so there is no separate LM Studio model: search for the model name in LM Studio's model browser and load the build that fits your memory.
Does the model remember earlier parts of my story?
Only within its context window. The models in this guide offer 32K to 256K tokens, but longer context uses more memory, and a model forgets anything outside the window. For long projects, start each session with a short story summary and the character sheet.
Can a local LLM replace Claude or ChatGPT for fiction?
Not on creative-writing leaderboards: on the EQ-Bench Creative Writing v3 list read via evals.report, closed models lead (Claude Opus 4.7 at 2206 Elo, GPT-5.5 at 2035). A 24 GB local model can still be good enough for drafting, and it keeps manuscripts on your machine. Sampling settings and a strong system prompt close part of the practical gap.
Is quantization noticeable in creative writing?
It can be, because lower-bit builds lose some precision. We cite no creative-writing study on this. Ollama's default builds are 4-bit; if you have memory headroom, compare a higher-bit build with the 4-bit build on the same scene and keep the one you prefer.
What temperature should I use for creative writing?
As an editorial starting range, 0.9–1.1 for prose and 0.7–0.85 for tight dialogue or strict poetic forms. Vendor defaults are a sound anchor: Google lists 1.0 (top-p 0.95, top-k 64) for Gemma 4, and Alibaba lists 1.0 in thinking mode and 0.7 in non-thinking mode for Qwen3.8. Turn thinking mode off for fiction.
Can I use local LLM output in commercial work?
It depends on the licence. Gemma 4, Qwen3.8, Mistral Small 3 and Hermes 4.3 are listed under Apache 2.0, Llama 3.3 uses the custom Meta Llama 3.3 Community License, and Fimbulvetr-11B is CC BY-NC 4.0 (non-commercial). Read the licence on the model card for the exact build you use. This is not legal advice.
Are Fimbulvetr and Midnight-Rose still worth using?
Only if you prefer their voice. Fimbulvetr-11B-v2 is a Solar-based model under a non-commercial licence, and Midnight-Rose-70B is a Llama 2 70B merge; both predate the 2026 models in this guide, and we found no creative-writing benchmark score for either.
Sources
- Ollama library: Gemma 4 -- tags, download sizes and context windows
- Ollama library: Qwen3.8 and Qwen3.5 -- tags, sizes and context windows
- Ollama library: Mistral Small and Llama 3.3 -- tags, sizes, context and licence notes
- Hugging Face: Gemma 4 31B model card and Gemma 4 12B model card -- licence, context length, sampling defaults
- Hugging Face: Qwen3.8-27B model card -- licence, native context, recommended sampling
- Hugging Face: Hermes 4.3 36B -- base model, licence, RefusalBench figures, GGUF files
- Hugging Face: Fimbulvetr-11B-v2 and Midnight-Rose-70B -- base models and licences
- EQ-Bench Creative Writing v3 and evals.report EQ-Bench scores -- leaderboard figures (data snapshot 2026-11-15)
- Featherless: best open models for creative writing -- Gemma 4 EQ-Bench figures and open-weight model sizes (dated 2026-09-17)
- Ollama documentation: thinking -- the `think` parameter
Common Mistakes When Choosing and Using Local Models for Writing
- Picking by a general benchmark rank. Coding and reasoning leaderboards say little about prose. Look at creative-writing lists such as EQ-Bench, then test on your own scenes.
- Using coding-style sampling. Temperature 0.2–0.4 tends to produce flat prose; creative work wants roughly 0.9–1.1 (editorial range).
- Leaving thinking mode on. Planning-first output can read as assembled. Turn it off for fiction.
- Ignoring context memory. A 19–20 GB model leaves little room on a 24 GB card once you raise the context window.
- Thin prompts. "Write a story" produces generic output; add point of view, tense, length, a character sheet and an explicit constraint.
- Editing inside the chat window. Generate in the chat, revise in a text editor; revising the prose yourself is what keeps the voice yours.
