Wichtigste Erkenntnisse
- Use Git for prompts; one YAML file per prompt with metadata; PR review = quality gate
- Shared testing: PromptFoo (free, local) or PromptQuorum (no-code); all team members run tests before merge
- Shared prompt library: Git repo + README listing all prompts and who owns them
- Weekly sync: 30 min; share wins (best prompt this week), blockers (prompts failing), and ideas
- No database yet: Start with Git; only move to Braintrust/LangSmith if team grows >10 people
Why Small Team Prompt Work is Different
Small teams lack resources for databases, microservices, and complex governance.
- Budget: Can't afford Braintrust ($500/month) or LangSmith ($300/month); need free or cheap
- Headcount: No dedicated prompt engineer or DevOps; everyone wears multiple hats
- Complexity: 50 prompts, not 5,000; over-engineering slows the team
- Communication: Everyone in same Slack; lightweight tools work
Setup 1: Prompts in Git
Single repo; one YAML file per prompt; standard directory structure.
- Directory: `prompts/{domain}/{use-case}/v{N}.yaml` (e.g., `prompts/support/email-draft/v1.yaml`)
- File format: System prompt + example input/output + metadata (owner, created, tests, model)
- README: List all prompts with one-line description; link to owner
- Naming: Consistent, kebab-case, version-numbered; predictable
- Access: Clone repo locally; all team members can read; PRs to edit
Setup 2: PR Review Workflow
No prompt merges without review; branch protection enforces it.
- Branch protection: Require 1 approval on any prompt change
- Reviewer checklist: (1) Tests pass, (2) Metadata filled, (3) Naming correct, (4) No breaking changes to existing usage
- Comment template: Reviewers use: "Approved for {domain}. Testing shows {accuracy improvement}."
- Merge: Squash merge to keep history clean; commit message includes prompt intent
Setup 3: Lightweight Testing
Use free or cheap tools; no complex infrastructure.
- Option A — PromptFoo (free, open-source): YAML test files + local CLI; run before PR
- Option B — PromptQuorum (free, cloud-hosted): No code needed; team shares test workspace
- Option C — Simple Python script: If team all Python devs; custom tests for custom needs
- Test requirement: Before merging any prompt change, run test suite; must pass on GPT-4o + one cheap model (Ollama or Gemini)
Setup 5: Slack Notifications
Keep team in sync with lightweight Slack bots; no complex Zapier flows.
- On PR created: Slack message "New prompt PR by {author}: {prompt name}. Review in {link}."
- On PR merged: Slack message "New prompt version deployed: {name} v{N} by {author}"
- Tool: GitHub Actions + Slack webhook (built-in); no additional SaaS needed
- Channel: Use #prompts channel for all notifications; easy to mute if desired
Setup 6: Weekly Team Sync
30-minute sync; share wins, blockers, and ideas.
- Agenda: (1) Wins this week (best prompt, successful test), (2) Blockers (what failed?), (3) Ideas (what should we try?)
- Output: Document in Slack; reference previous weeks for pattern recognition
- Retrospective: Monthly review—are chosen tools/processes helping? What's not working?
- Low friction: Video call, not formal meeting; chat-like energy
When to Upgrade (Team >10 people)
If team grows beyond 10, consider tools designed for scale.
- Braintrust: Database of prompts + evals; team collaboration; recommended >5 people
- LangSmith: LLM ops platform; tracing, evals, feedback; good if building AI products
- Trigger to upgrade: Team spending >2 hours/week managing prompts in Git (overhead sign)
- Transition: Move Git prompts to database gradually; no big bang migration
Common Mistakes
- Over-engineering: Adopting Braintrust when 3 prompts exist; unnecessary cost
- No process: Prompts scattered in Slack, personal files, Google Docs; no version control
- No reviews: Anyone can edit main branch; no quality gate
- No documentation: Team doesn't know which prompts exist; duplication happens
- Ignoring feedback: Not tracking what prompts work; can't improve systematically
Sources
- GitHub Actions documentation
- PromptFoo open-source setup guide
- Small team DevOps practices adapted for prompts