How elsieOS works
One graphics card and several AI programs that each want all of its memory. elsieOS is built around a referee that decides who gets the card, and around making the hand-over fast.
One referee for the graphics card
On an ordinary machine, a chat server and ComfyUI do not know about each other. Each one loads what it needs, and whoever comes second runs out of memory, crawls along half on the processor, or crashes. Home setups patch this with scripts that poll and unload things every few seconds.
On elsieOS every request for the card goes through one referee: the model server. Chat apps reach it on Ollama's
port 11434. ComfyUI sits behind a gate on its usual port 8188, so its web page and its
API work as before, but every job is announced to the referee first.
What that looks like
- A new image or video job arrives: idle chat models are parked and the job starts, 0.27 s after it arrived.
- A chat request arrives during a job: it waits instead of crashing the job, and is answered 0.6-0.9 s after the job ends.
- A chat model that fits next to the job (judged by that workflow's last measured peak) runs beside it.
- The model you keep loaded comes back by itself 4.8 s after a job, not half a minute later.
- Two chat models that do not fit together take turns; a busy one finishes its answer first, and waiting requests take their turn instead of failing.
Models park, they do not unload
Most model servers make room by throwing a model away and reading it back from disk later. elsieOS parks it instead: its weights are copied into pinned (page-locked) RAM, which the card can read at the full speed of the PCIe bus, about 50 GB/s on a PCIe 5.0 x16 card. Waking it is the same copy the other way.
The prompt cache survives a park, so a chat app's long instructions are not read again after every switch.
Under the hood is llama.cpp with a small elsieOS patch that adds park and wake. Parking only uses RAM up to a limit you set; a model that does not fit in it is unloaded the ordinary way.
Bringing back a 24B chat model
Mistral Small 3.2 24B, 13.3 GiB of weights, RTX 5090.
Updates that undo themselves
The system lives in two slots. Only one is in use; an update goes into the other one, so the version you are running is never touched while the new one is written.
elsie updatewrites the new version into the spare slot. The system files are read-only and checked against a hash tree, so a damaged file is noticed instead of used.- The next start is a trial. A health check has to pass: the model server answers and no service has failed. Then the new version is kept.
- Three failed starts and the machine starts the previous version by itself. On the test machine that took about two and a half minutes, with nobody there.
A hardware watchdog covers the rest: if the system freezes, the board restarts it about 30 seconds later. Measured on real hardware.
Secure by default
A machine that only runs AI does not need much, so it gets very little.
- Secure Boot with a signed kernel and signed NVIDIA modules
- Kernel lockdown on
- Read-only system files, checked by a hash tree
- SSH with keys only, no passwords
- Tunnel-only users for apps that just need a port or two
- Model server open to this machine, your home network and your Tailscale network only, unless you change it
- Remote access through Tailscale, built in
- No desktop, no browser, no extra services
Works with the apps you have
The model server speaks Ollama's API on Ollama's port (/api/chat, /api/generate,
/api/tags and the rest) and the OpenAI-style API (/v1/chat/completions,
/v1/models, ...), so chat front ends, scripts and coding tools keep working.
elsie models import brings models over from an existing Ollama install and keeps Ollama's prompt
formats, so answers stay what they were (checked word for word against Ollama with Mistral Small 3.2). ComfyUI moves the same way with
elsie apps import comfyui: models, custom nodes, settings and workflows.
Tested as a replacement for a working Ollama machine for 15.5 hours: 84 chats from real apps, none of them changed, 83 answered. The one failure was a GPU-sharing bug, fixed in the next version. The test day.
$ curl http://elsieos:11434/api/chat -d '{ "model": "mistral-small3.2", "messages": [{"role": "user", "content": "hi"}] }' {"model":"mistral-small3.2","message":{"role":"assistant", "content":"Hi! How can I help?"},"done":true, ...} $ curl http://elsieos:11434/v1/models {"object":"list","data":[{"id":"mistral-small3.2", ...
What elsieOS is not
Saying no to these is what keeps it small, fast and safe to update.
- Not a desktop: you use it from other computers
- Not a general-purpose Linux distribution
- Not a cloud service: it runs on your machine
- No telemetry, no tracking, no account
- NVIDIA only for now (AMD later)
- One GPU per machine for now