White paper · Coming soon
Dense infra
AI is changing what a workload looks like. Instead of a few long-running services, teams now run thousands of short-lived agents, sandboxes and small models, most of them idle most of the time. Infrastructure built for the old shape keeps each one warm and pays for it. Dense infra is our work on the opposite: fit thousands of AI workloads on one machine, start each in milliseconds, and charge nothing for the time it waits.
01The problem
A coding agent runs a test, waits a minute for the model, then runs another. A support bot handles one message every few minutes. An evaluation spins up five hundred sandboxes for thirty seconds each. These workloads are many, small and bursty.
Today's stacks answer them in one of two ways, and both are expensive:
- Keep them warm. Every agent or model holds its memory, a process and often a GPU slice while it does nothing. Utilisation sits in single digits.
- Start them cold. Containers and microVMs take hundreds of milliseconds to seconds to start, and loading a model takes longer. Users wait, so teams go back to keeping things warm.
The result is that the cost of running AI is set by the number of workloads that exist, not the number doing work.
02The thesis
If starting a workload is cheaper than keeping it, the economics flip. When a start takes a few milliseconds and an idle workload holds almost nothing, a single machine can host thousands of them, and the bill follows the work actually done.
That needs four things to be true at once: starts fast enough to hide inside a request, idle cost near zero, everything identical shared rather than copied, and isolation strong enough that density does not cost safety.
03Where we start: Kiri
Kiri, our container runtime, replaces the dockerd, containerd, shim and runc chain with one Rust daemon and one minimal VM. It was built for developers' laptops, but it already shows the start and packing numbers dense infra depends on.
| Measurement | Kiri | Docker Desktop | OrbStack | Colima |
|---|---|---|---|---|
docker run --rm | 22 ms | 178 ms | 279 ms | 152 ms |
| Container lifecycle, Docker API | 3.1 ms | 163 ms | 238 ms | 88 ms |
| Parallel starts per second | 88.9 | 11.4 | 8.2 | 12.9 |
| Density, containers per second | 19.7 | 3.2 | 2.9 | 4.1 |
Medians from one full kiri-bench run on one Apple Silicon Mac, 2 November 2026, every engine limited to the same 2 CPUs and 8 GB. Kiri still fails two tests in that run (an image build and an iperf3 network test), and OrbStack is faster on raw networking. Full results.
Kiri's numbers are for one developer machine. Dense infra is about taking the same design to servers, at thousands of workloads per host, and adding what AI workloads need on top.
04The design
05What we are working on
Start in milliseconds Kiri today
No warm pools. A workload starts when its request arrives. Kiri starts a container in 22 ms on a laptop; the goal is the same on servers, under load, with thousands of neighbours.
Sleep at zero Research
An idle agent should hold no CPU and close to no memory. Its state lives in a snapshot until the next request wakes it.
Freeze and resume Research
Snapshot a running agent or a loaded model and restore it in milliseconds, so a model does not reload its weights and an agent picks up mid-task. This is how thousands of workloads take turns on hardware sized for a few hundred.
Share what is identical Research
A thousand agents built on the same image and the same model should not hold a thousand copies. Shared pages, mapped read-only, with copy-on-write for what changes.
Split the GPU Research
Many small models and agents on one card, each with a memory limit and a fair share of time, so a small workload does not hold a whole GPU.
Isolate cheaply Research
Density only matters if it is safe. Each workload gets its own boundary, without the cost of a full virtual machine per workload.
06What we will measure
- Workloads per host: how many agents or models one machine holds before p95 latency moves.
- Time to first response from asleep: from request to first byte, for a workload that was frozen.
- Idle cost: memory and CPU held by a workload doing nothing.
- Cost per useful second: what the machine costs, divided by the seconds of real work done on it.
We will publish these the way we publish Kiri's benchmarks: on equal hardware, against the alternatives, including where we lose.
07What this is not
- Not a model provider. Dense infra runs your agents and models; it does not train or sell models.
- Not a database host. State that must last lives in your own storage.
- Not only for agents. Agent sandboxes are one use; small models, evaluations and per-user tools are others.
08Status
Dense infra is a research project at Canestack Labs and is not available yet. The runtime it builds on, Kiri, is in beta today. This paper will be updated as the work lands. To hear when it is ready, write to [email protected].