SOC 2 & ISO 27001 Readiness for Self-Hosted LLM Deployments (2026)

No — this guide is not legal or compliance advice. It explains, at a technical level, the categories of controls a SOC 2 or ISO 27001 auditor typically checks and how they map onto a self-hosted LLM deployment. Whether a specific control satisfies your audit depends on your auditor's judgment, your organization's risk assessment, and the exact scope statement you file. Consult a qualified auditor, QSA, or your compliance counsel before you schedule an audit or make claims about your compliance status.

SOC 2 evaluates an organization against five Trust Services Criteria (TSC), and every one of them applies to a self-hosted LLM system the moment it touches production data. An auditor does not test the model — they test whether your organization can demonstrate the control existed and worked over the review period.

Use security if your report only needs to cover the baseline criterion (mandatory in every SOC 2 report). Add availability if the LLM serves a production workflow with an uptime commitment. Add confidentiality if prompts or outputs contain contractually protected data. Add processing integrity if the model's output feeds a downstream decision. Add privacy if prompts contain personal data about identifiable individuals.

ISO 27001 certifies an organization's Information Security Management System (ISMS), and Annex A is the reference list of controls the ISMS draws from — not a checklist applied directly to any one system. A self-hosted LLM deployment sits inside the ISMS scope like any other asset: it needs a risk assessment, a Statement of Applicability entry, and evidence the relevant controls operate.

Use the organizational controls (A.5) first — they cover vendor risk for the model publisher, not just the software vendor. Add the technological controls (A.8) next — they cover encryption, logging, and endpoint hardening directly.

An auditor expects to see who called the model, with what credential, and when — a requirement none of the common self-hosted engines satisfy without a gateway in front of them. Ollama binds to `127.0.0.1:11434` by default and has no user accounts; vLLM and Hugging Face TGI expose an OpenAI-compatible HTTP API with no built-in authentication.

Use an API gateway (Kong, Envoy, or a cloud provider's API management layer) in front of the inference engine to add per-caller API keys or OAuth2, then log every request with caller identity, timestamp, model version, and token counts to a system your SIEM can ingest. Avoid logging full prompt/response bodies in the access log itself — keep that in the separately governed prompt-log store covered under data retention below.

Model weights, prompt logs, and response logs need the same at-rest and in-transit encryption an auditor already expects for any other production data store. Weights themselves are not typically secret, but the inference server's disk usually also holds cached prompts, fine-tuned adapters, and logs that are.

Use full-disk encryption (LUKS on Linux, BitLocker on Windows, FileVault on macOS) on the inference host as the baseline. Add TLS termination at the gateway for every inbound API call — never expose the raw inference port over plaintext HTTP, even inside a VPC. Use a managed KMS (AWS KMS, Azure Key Vault, HashiCorp Vault) for the keys protecting the prompt-log store rather than a static key baked into a config file.

Every model version swap, quantization change, or system-prompt edit is a production change and needs the same approval trail as a code deployment. Auditors specifically look for evidence that changes were reviewed and approved before going live, not just that a changelog exists after the fact.

Use a version-pinned deployment (tag the exact model checksum, not a mutable "latest" pointer) so a rollback is possible and auditable. Use a documented approval step — even a lightweight one, like a pull-request review — before any production model swap, and log the approver, the date, and the reason for the change.

"Open-weight" does not mean "no vendor" — the model publisher is a supply-chain party the same way a SaaS vendor is, and an auditor expects a documented risk assessment for it. This is one of the most commonly missed controls: teams treat a downloaded GGUF or safetensors file as internal, not third-party, even though it originated outside the organization.

Use the publisher's identity and track record as the first check — a model from Meta, Mistral AI, Alibaba/Qwen, or a similarly identifiable organization is a different risk profile than an anonymous upload. Verify the file checksum against the publisher's published hash before deployment. Check the licence terms for commercial-use restrictions. Track known CVEs not in the model itself but in the serving stack (llama.cpp, vLLM, TGI) that loads it.

A model-serving system has incident categories a standard web app runbook does not cover — model exfiltration, prompt injection that leaks data through the model's own output, and inference-endpoint compromise — and each needs a named response path. An auditor checks whether the plan exists and has been tested, not whether an incident has actually occurred.

Use your existing incident-response plan as the base and add model-specific triggers: unexpected model weight file changes, an unusual spike in requests from one credential, or output logs showing a prompt-injection pattern. Define who is notified, what evidence gets preserved (logs, not just a summary), and how quickly the endpoint can be isolated.

Prompt logs are the highest-risk data your LLM stack produces, because they often contain the same sensitive content a user would type into any other business system. A written retention policy — how long logs are kept, who can access them, and how they are deleted — is a control an auditor will specifically ask to see, not just infer from your general data-retention policy.

Use a retention window that matches your actual audit or debugging need, not "keep everything forever by default." Use field-level minimization where possible — log metadata (caller, timestamp, token count, model version) without the full prompt text for routine traffic, and reserve full-content logging for a shorter, more tightly access-controlled window used for debugging or abuse investigation.

The inference server should sit in its own network zone, reachable only through the authenticated gateway — not flat on the same subnet as general application servers. This limits the blast radius if any other service on the network is compromised and gives an auditor a clean network diagram to review.

Use a dedicated VLAN or subnet for the inference host, with firewall rules that allow inbound traffic only from the API gateway and outbound traffic restricted to what the engine actually needs (model registry pulls, if any — many air-gapped deployments block outbound entirely after the model is downloaded).

None of the common inference engines ship a finished audit trail — the difference between them is how much you have to build versus how much a platform already provides. This is a readiness comparison, not a compliance claim about any of these tools.

Use Ollama or vLLM if you are comfortable building the gateway, logging, and access-control layer yourself. Use an enterprise inference platform if you would rather pay for built-in RBAC and audit logging and reduce the custom-instrumentation work — you still need the surrounding ISMS documentation either way.

Compliance automation platforms — Vanta, Drata, and Secureframe are the three most commonly used — pull evidence automatically from cloud infrastructure, HR systems, and identity providers, but a self-hosted, on-prem inference server usually sits outside their default integration list. These platforms can still automate evidence collection for the parts of your infrastructure they do integrate with (cloud accounts, identity provider, ticketing system), while the inference host itself typically needs a manual evidence upload or a custom API integration.

Use a compliance automation platform if you are running a broader SOC 2 or ISO 27001 program across the company and want continuous monitoring for everything except the self-hosted LLM layer. Skip one if the LLM deployment is your only in-scope system — a smaller, manually maintained evidence folder may be cheaper for a narrow scope.

Most self-hosted LLM audit findings come from treating the inference server as outside the normal IT control environment, not from a fundamentally missing control. These are the five most frequent gaps.

Work through this checklist before your auditor's fieldwork begins — each item maps to a control category covered above.