Why we built elsieOS
It started with one computer and one graphics card doing too many jobs.
One card, too many jobs
The machine was a home AI server: an NVIDIA RTX 5090 with 32 GB of memory, 64 GB of RAM, and a list of things to do every day. A couple of chat models answered questions for websites and apps, and ComfyUI made images and videos. Any one of those jobs could use the whole card. Together they could not fit.
Switching was slow
A 24B and a 27B chat model do not fit on a 32 GB card together, so every time a question needed the other model, the model server threw one away and read the other back from its file. That took 1.6 to 3 seconds when the file was still in RAM, and 5 to 11 seconds when it had to come from the SSD. A chat app's long instructions then had to be read again from the start.
Memory was a fight nobody was refereeing
The chat server and ComfyUI did not know about each other. Whoever came second ran out of graphics memory, crawled along half on the processor, or crashed. The usual fix, which we had too, was a helper script that checked every ten seconds and unloaded or reloaded things. It worked, mostly:
- An image job could start while the chat model still filled half the card.
- After a job, the chat model came back half a minute later at the earliest.
- A chat question that arrived in between made the model load next to ComfyUI, partly on the processor, and slow.
RAM was too small to rescue it
The models alone were bigger than the 64 GB of RAM, which everything else shared too, so models kept falling out of RAM, and a "switch" often meant reading the SSD.
It was fragile
One request that asked for a different context size made the server reload the model. And every system or driver update carried the risk of a machine that did not start properly until someone sat down and fixed it.
None of this was one person's bad setup. People running AI at home write the same scripts for the same fights, because the tools were each built for a machine of their own.
- The card
- NVIDIA RTX 5090, 32 GB
- Memory
- 64 GB RAM
- The jobs
- a 24B and a 27B chat model, ComfyUI for images and video
- A model switch
- 1.6-3 s from RAM, 5-11 s from the SSD
- Keeping the peace
- a helper script checking every ten seconds
An operating system for one job
In November 2026 we wrote down what we actually wanted, and decided to build an operating system around it instead of another script:
- One referee for the graphics card. Chat models and ComfyUI share one NVIDIA card. Every request goes through one GPU manager that decides who gets the card, so nothing runs out of memory and nothing gets pushed onto the processor.
- Switch models in about half a second, instead of reloading them from disk.
- Updates that undo themselves. A new version that does not start cleanly goes back to the old one by itself.
- Works with the apps you already have: the same API as Ollama, so chat front ends and scripts keep working.
- One job, done well: no desktop, no browser, no tracking. Installable on anyone's machine with an NVIDIA card, not just ours.
What we built
- Park and wake. Instead of throwing a model away, elsieOS parks it: its weights are copied into pinned RAM, which the card reads at the full speed of the PCIe bus, about 50 GB/s. Waking it is the same copy back, and the prompt cache survives. How parking works
- A GPU manager that every chat request and every ComfyUI job goes through. A new job makes idle chat models park first; a chat that does not fit next to a running job waits for it instead of crashing it. The referee
- A/B system slots. An update goes into the slot that is not running; three failed starts and the machine goes back to the previous version on its own. A hardware watchdog restarts a frozen system. Updates
- An Ollama-compatible model server, and importers that bring over an existing Ollama and ComfyUI install.
What changed
Measured on one test machine (RTX 5090 32 GB, 64 GB RAM), with Ollama on the same machine for comparison. Your numbers will differ.
| What happens | Before | Now |
|---|---|---|
| Switching between a 24B and a 27B chat model, per question, answer included | 1.6-3 s from RAM 5-11 s from SSD | 0.87-1.06 s |
| The same with a ~3,200-token system prompt | 2.3-3.7 s 5.7-11.7 s | 0.89-1.03 s |
| A model coming back to the card | 7.5 s from SSD | ~0.3 s |
| An image job arriving while a chat model is loaded | starts beside it | 0.27 s, card cleared |
| A chat question during a job | half on the processor | 0.6-0.9 s after the job |
| The everyday chat model after an image job | 30 s or more | back in 4.8 s |
Under load
- 3.5 hours of chat, images and video together: 783 chats across four models, 240 images, 21 videos. 0 failures, no out-of-memory anywhere; the card peaked at 31.8 of 32.6 GB.
- 15.5 hours as a drop-in replacement for a working Ollama machine: 84 chats from real apps, none of them changed, 83 answered. The one failure was a GPU-sharing bug, fixed in the next version.
- An update broken on purpose went back to the previous version by itself after three failed starts, in about two and a half minutes, with nobody touching the machine.
From one machine to yours
elsieOS is in development. Next is an installer, so it can go on machines other than ours, and then a beta with testers on other cards, RTX 20 series to RTX 50.
Know these fights?
Join the beta, and follow @getElsieOS on X and r/elsieOS on Reddit for the beta release.