Comparison
On-prem LLM vs an API
An 8-point comparison of running an LLM inside your perimeter versus calling a model on someone else's servers, and when each one fits.
By James Drayson
In short
An API is access to a model running on someone else's servers, billed per token; an on-prem LLM runs inside your own perimeter on hardware you control. With an API your data leaves your environment and you never own the model; on-prem, the data stays in and the model is yours.
When each one fits
- An API fits: early experiments, low-sensitivity workloads, and spiky usage where you don't yet want to provision hardware.
- On-prem is required: when data legally cannot leave your perimeter, when you need air-gapped operation, or when you need to own and audit the model.
- The cost crossover: per-token billing scales with usage forever; at sustained enterprise volume, owning the model becomes substantially cheaper and you keep the asset.
- The one-way door: every prompt sent to an external API is data you can't recall. On-prem avoids that irreversible exposure entirely.
On-prem LLM vs API LLM
| On-prem LLM | API LLM | |
|---|---|---|
| Data stays inside perimeter | Yes | No, sent to the provider |
| Own the weights | Yes | No |
| Air-gapped operation | Possible | Not possible |
| Domain-trained on your data | Yes, via post-training | No, general-purpose |
| Cost model | Fixed, owned asset | Per-token, recurring |
| Behaviour changes under you | Only when you choose | Can change at any time |
| Latency | Local, predictable | Network-dependent |
| ISO 27001-aligned deployment | Available | Depends on provider |
What this looks like with Locai
Sovereignty stops being a contractual promise when the machine is standing in your own building. Here is what that looks like in practice.
Locai Labs builds Locai One, an on-prem AI appliance. It is one machine that arrives with everything already in it: the hardware, our open-weight Locai Juno models, and Locai OS, the operating system that serves the models and handles users, access and monitoring. You plug it into a mains socket and your network, and your team is working in about 15 minutes. No cloud account, no per-token bill, and nothing leaving the building.
The reason a data-centre-class model fits in a box on your floor is SPACE, our compression algorithm. Instead of asking how much of a model can be cut while keeping it broadly similar, SPACE asks what the model needs to be good at, preserves the subnetworks behind those capabilities and strips back the rest. The result is a smaller specialist rather than a shrunken generalist, tuned to the exact hardware it ships on.
Locai One starts at £29,950 for a team, and Locai One Pro at £49,950 for an organisation, bought once and owned outright. Both run air-cooled on standard mains power and work fully air-gapped. Any compatible open-weight model runs alongside Juno, and if you need a model trained on your own proprietary data we can post-train one and deploy it on the same machine.
Frequently asked questions
Is on-prem AI hard to operate?
It used to be. Locai One packages a sovereign model, application layer, and serving stack into a fixed-cost on-prem appliance, so you get on-prem control without building the MLOps yourself.
What hardware do I need for an on-prem LLM?
It depends on the model size and concurrency. Locai One ships with one NVIDIA Blackwell GPU and 96 GB of VRAM; Locai One Pro has two GPUs and 192 GB of VRAM. Larger models or heavier concurrency need multi-GPU nodes.
Can on-prem models still improve over time?
Yes. With continual learning the on-prem model is retrained on your evolving data on a schedule, so it compounds in value rather than freezing at deployment.
Book a sovereign AI briefing
A 30-minute session on owning your model: deployment options, the data path, and a clear cost range for your use case.