The Operating Constraint
The engagement began where most enterprise AI projects end: inside a SCIF, on a classified network, with no egress and no exceptions. No cloud APIs, no managed services, no vendor support tunnels. Every byte of model traffic — weights, context, outputs — had to live on-premises behind a physical air gap.
That constraint ruled out the entire managed-inference market. It also ruled out naive open-weight serving: the environment demanded prompt-level data separation, sub-second retrieval over four decades of technical orders, and a reasoning model that could actually coordinate supply decisions across multiple legacy databases.
Stack Selection
We evaluated sovereign-capable serving stacks against three hard criteria: FP8 weight support on available accelerators, quality retention at quantization, and operational simplicity for an air-gapped fleet. The final topology:
Retrieval That Doesn't Leak
The retrieval pipeline had to index classified maintenance documentation without ever exposing it to a third party. We deployed an on-prem FAISS cluster behind the same air gap, chunked documents with section-level granularity, and embedded them with a model whose weights were already sovereign-approved.
Query routing happens at the gateway: the reasoning model decomposes a requisition into part numbers, technical-order references, and hazard constraints, then issues structured retrieval calls. Every retrieval result includes a provenance chain back to the source technical order so maintainers can verify before acting.
Results
What Would Break This
Three failure modes deserve attention. First, quantize without an eval harness and you silently degrade classification-heavy tasks — we shipped a quantization-aware eval suite to gate every model update. Second, assume the air gap protects you from bad retrievers, and you will surface stale technical orders. Third, treat model latency as a fixed cost while the depot grows, and the four-minute budget will slip. All three were addressed with instrumentation that ships with the deployment, not after it.