kcell: Docker apps, each in a tiny VM
What you get
- Unmodified Docker: the official static binaries (Docker 29.8, Compose 5.5). Compose files work as they are.
- 18 app templates (Jellyfin, Nextcloud, Immich and others), or your own Compose file. keelOS makes the kcell for the app by itself, with network, CPU, memory and disk at sensible defaults.
- From your own computer:
docker context createwith an ssh address, thendocker compose up -das on any Linux machine. - Volumes are ZFS datasets on the host, so their files are visible there. Ports are published on the kcell's own address, and by default on the host's too.
- Stopping a kcell saves a checkpoint, so it comes back where it was.
How it works
Docker needs a Linux kernel, which FreeBSD jails and the Linux compatibility layer do not provide. So a kcell is a bhyve VM with a Linux 7.2 kernel configured by us, a tiny init and a read-only system (86 MB compressed) that all kcells on a host share.
Volumes and the system are host datasets shared over virtio-fs. Only Docker's image layers live on a sparse zvol (40 GB by default). From start to a ready Docker takes 3.9 seconds, and a resume from a checkpoint takes 1 second.
What was measured
- Volume on the host (virtio-fs): 742–777 MB/s sequential write, 22–23k IOPS of 4K random writes; databases and npm run as fast as on a local disk.
- Integrated GPU: a SR-IOV virtual function (12th generation Intel and newer) lets a kcell transcode 1080p to 720p HEVC at 73 fps.
Limits
- Memory is set for the whole kcell, and changing it needs a cold restart of a few seconds.
- Many small files (unpacking, compiling) are 5–6 times slower on a host volume; use a Docker volume for those.
- Only x86. No Swarm encrypted overlay networks, no SELinux or AppArmor inside. A kcell cannot run nested virtualization.
- GPU sharing needs a 12th generation or newer Intel integrated GPU. AMD and NVIDIA are not done.
- A keelNAS share as a media library is in the form but has not been run from scratch on real hardware.