Skip to main content
PromptQuorumBuilt for humans. Structured for AI.
Home/Local LLMs
Enterprise

On-Prem Air-Gapped Local LLMs: Isolated Deployment for Classified Environments

·12 min readBy · Founder of PromptQuorum · Discovery engine for open-weight & open-source AI

An air-gapped LLM runs on infrastructure with no internet connection, so prompts and documents cannot reach external APIs. Governments, defense and regulated industries use this pattern for classified or highly sensitive data. Models, updates and logs move in only through controlled, offline transfer steps.

Key Takeaways

  • Air-gapped = completely isolated from internet. Zero network connectivity to external systems.
  • Use case: Government (classified), military (war scenarios), finance (high-security), healthcare (HIPAA ultra-sensitive).
  • Challenge: Cannot auto-update models, embeddings, or dependencies. Requires manual update procedures.
  • Architecture: Dedicated hardware, encrypted storage, restricted user access, physical security.
  • Network isolation: Firewalls, air-gap appliances, and network segmentation prevent any external communication.
  • Model management: Transfer models via USB, secure media, or internal networks only; no cloud access.
  • Updates: Manual process -- test updates offline first, then deploy via secure channels.
  • Air-gapped deployment is a common pattern for government agencies and defense contractors handling classified data.

📍 In One Sentence

Air-gapped deployment means zero network connectivity between the LLM infrastructure and the internet or any external system -- models and updates move only via physical media or isolated internal networks.

💬 In Plain Terms

An air-gapped AI server is completely disconnected from the internet, like a computer with no network cable and no WiFi. Air-gapping removes the remote-attack path, which makes it one of the strongest isolation measures for sensitive data. It does not protect against insiders, compromised removable media or physical access. The tradeoff is that updating the AI model takes manual work instead of an automatic download.

An air-gapped LLM runs on infrastructure with no internet connection, so prompts and documents cannot reach external APIs. Governments, defense and regulated industries use this pattern for classified or highly sensitive data. Models, updates and logs move in only through controlled, offline transfer steps.

On-Prem Air-Gapped Local LLMs: Isolated Deployment for Classified Environments

What Does Air-Gapped Mean?

Air-gapped infrastructure has no network connection to the internet or any external system. All data and computation stays on isolated hardware.

Absolute isolation: No WiFi, no Ethernet to external networks, no USB connections to internet-connected devices.

Data stays local by design: Inference runs locally and results stay local; data can still leave through removable media, insiders or physical access.

Updates are manual: Cannot auto-download model updates. Updates require physical media (USB drive, SD card) or internal network transfer.

What Does Typical Air-Gapped Architecture Look Like?

A single machine or small cluster, completely isolated, with restricted physical access.

  1. 1
    Dedicated hardware: Servers used only for LLM inference, nothing else.
  2. 2
    Isolated network: No connection to corporate network or internet. Separate VLAN at most.
  3. 3
    Encrypted storage: All model files, data, logs encrypted at rest.
  4. 4
    Controlled access: Only authorized personnel can access. Multi-factor authentication required.
  5. 5
    Physical security: Locked server room, surveillance, access logs.
  6. 6
    No removable media: USB ports disabled, CD/DVD drives removed.
  7. 7
    Local monitoring: Logs stay on-system, not sent to external monitoring services.

How Do You Isolate Networks From External Connections?

Isolation is enforced at multiple levels:

  • Physical: Separate network hardware (switch, router) for air-gapped infrastructure.
  • Firewall: Inbound and outbound traffic blocked. No exceptions for cloud services, NTP, DNS to external servers.
  • Software: Host-level firewalls (iptables, Windows Firewall) as secondary control.
  • Monitoring: Network traffic audited. Any attempt to connect externally is logged and flagged.

How Do You Manage Models in Air-Gapped Environments?

Model updates are manual, requiring physical media transfer or internal secure processes.

  1. 1
    Download model on internet-connected machine (separate, non-classified environment).
  2. 2
    Verify model integrity (checksum, digital signature).
  3. 3
    Transfer to air-gapped system via encrypted USB drive or internal file server.
  4. 4
    Validation: Run tests to ensure model integrity was not compromised during transfer.
  5. 5
    Deployment: Load model into inference engine.
  6. 6
    Documentation: Log which models are deployed, versions, dates.

How Do You Handle Updates and Patching?

Security updates and model updates are manual:

  • Model updates: Follow process above. Schedule updates quarterly or annually.
  • OS patches: Test on isolated staging environment first, then deploy to production.
  • Dependencies: Evaluate new versions carefully. Air-gapped systems run older versions longer.
  • No auto-updates: Disable automatic updates entirely. All updates are controlled, documented, audited.

How Do You Ensure Security in Air-Gapped Systems?

Air-gapped inherently more secure (no external attacks), but new risks emerge:

  • Insider threats: Employees with access could copy data via USB. Require two-person integrity.
  • Supply chain attacks: Compromised models or dependencies during transfer. Verify checksums, digital signatures.
  • Physical theft: Models and data worth millions. Require locked room, surveillance, intrusion detection.
  • Social engineering: "Trust me, I need to update the model." Require formal change control processes.
  • Logging gaps: If logs are not audited, nobody detects insider activity. Require external audit of logs regularly.

Common Air-Gapped Deployment Mistakes

  • Incomplete isolation: Leaving one port open, one wireless card enabled, or allowing USB devices defeats air-gapping. Audit rigorously.
  • No change control: Model updates happen informally, no documentation. Leads to loss of knowledge about which version is deployed.
  • Poor backup strategy: Air-gapped systems need redundant storage and off-site backups. But backups themselves must be air-gapped.
  • Insufficient logging: Air-gapped systems require comprehensive auditing (who accessed what, when). Without logs, breaches go undetected.
  • Trusting physical security alone: Security requires multiple layers: isolation, encryption, access control, auditing, not just locked doors.

Common Questions About Air-Gapped Systems

How do we update models in an air-gapped system?

Manually via encrypted USB, or via an internal secure network isolated from the internet. All updates require change control approval and audit trails.

Can we use cloud backups for air-gapped systems?

No. Cloud backups require internet connection (defeating air-gapping). Use physical backups (encrypted hard drives) stored in separate secure facility.

Is air-gapped truly secure against all attacks?

Mostly, but insider threats remain. Air-gapped is secure against remote attacks, but physical access or insider access can compromise it.

How expensive is air-gapped deployment?

Cost depends heavily on scope. Hardware is typically comparable to a regular on-prem deployment; operating costs rise because updates, auditing and change control are manual. Get quotes for your own requirements.

Can we use standard tools (Ollama, vLLM) in air-gapped environments?

Yes. Both tools work without internet. Once models and dependencies are installed offline, and with update checks and telemetry disabled, no internet connectivity is needed.

What if we need GDPR compliance but don't need full air-gapping?

For workloads where air-gapping isn't required but GDPR is, EU-hosted cloud GPU options → (Hetzner, Scaleway, OVHcloud) offer EU data residency without isolation complexity. Overall GDPR compliance still depends on your lawful basis, contracts and safeguards. This is not legal advice.

A Note on Third-Party Facts

This article references third-party AI models, benchmarks, prices, and licenses. The AI landscape changes rapidly. Benchmark scores, license terms, model names, and API prices can shift between the time of writing and the time you read this. Before making deployment or compliance decisions based on this article, verify current figures on each provider’s official source: Hugging Face model cards for licenses and benchmarks, provider websites for API pricing, and EUR-Lex for current GDPR and EU AI Act text.

Run PromptQuorum with a local LLM, your own API keys, or both — you pick the backend.

Download the PromptQuorum Beta →

← Back to Local LLMs

MCP