SideMiniServer & LLM box
Running Ollama on a Mac mini around the clock
Apple silicon is unusually good at running language models: the GPU shares system memory, so what you can load is decided by how much memory you bought rather than by a graphics card. The models are the easy part. Keeping the machine awake, reachable and able to recover is where people get stuck.
How much memory you need
Memory is the constraint. A 16 GB Mac mini runs 7 to 8 billion parameter models comfortably at common quantisations; 24 to 32 GB opens up mid-sized models; 64 GB and above starts to make the large ones practical.
The base M-series Mac mini is the best value per watt of anything you can buy for this, and it is silent enough to sit in a living room.
Install and serve
- Install Ollama and pull a model, for example
ollama run llama3.1. The first run downloads several gigabytes. - By default the server listens on localhost only. Set
OLLAMA_HOST=0.0.0.0so other devices on your network can reach it, or enable the equivalent switch in LM Studio. - Test from a second machine using the Mac’s address before you go any further.
- Optionally put a chat interface such as Open WebUI in front of it so the rest of the household gets a web page instead of a terminal.
Keep it answering at three in the morning
A Mac that goes to sleep stops answering, and macOS sleeps by default. There are two layers to fix and they are not the same thing.
A power assertion, which is what caffeinate and SideMini’s keep-awake switch both create, holds the Mac awake while a process runs. It dies with that process and it does not survive a logout or a power cut.
The energy settings themselves survive everything. Never sleep, no disk spin-down, restart after a power failure, wake for network access. Set them once, from Energy settings or from SideMini’s Server mode.
Reaching a machine with no screen
SSH is enough to manage Ollama and you should enable Remote Login regardless. When you want the desktop, Screen Sharing needs the Mac to have a display, or the picture is blank; a virtual display solves that without occupying a port.
From outside the house, use Tailscale or a similar private network. Do not forward the model’s port to the internet: it has no authentication.
Surviving a reboot
Ollama installs a login item, so it starts when a user session starts. That means an unattended Mac needs automatic login, which in turn means FileVault off. Decide that deliberately; the trade-off is set out in our FileVault post.
Test the whole loop before you trust it: pull the power for a minute, let the Mac come back on its own, and check that the model answers from another device without you touching anything.