There are several ways to run local models. We use Ollama because it handles downloading, quantisation choice and serving in one tool, and because the same commands work identically on all three operating systems. Chapter nine covers the graphical alternative if you would rather not use a terminal at all.
macOS
Download the app from ollama.com/download and drag it to Applications. It needs macOS 14 Sonoma or later.
Or with Homebrew:
brew install ollama
Linux
curl -fsSL https://ollama.com/install.sh | sh
This installs the binary and sets up a systemd service, so Ollama starts with the machine.
A note on piping a script into a shell: you are executing code from the internet with your own privileges. That is standard practice for this tool and it is still worth understanding. If you would rather read it first:
curl -fsSL https://ollama.com/install.sh -o install.sh
less install.sh
sh install.sh
Windows
Download the installer from ollama.com/download and run it. It installs into your user account and does not require Administrator rights.
Or with winget:
winget install Ollama.Ollama
Docker, on any platform
docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
The named volume matters. Without -v ollama:/root/.ollama,
every model you download disappears when the container is removed — and you
will download several gigabytes again to find that out.
Verify it worked
ollama --version
A version number means you are done. "Command not found" on macOS or Linux usually means the shell has not picked up the new PATH — close the terminal and open a new one.
Then check the service is running:
ollama list
An empty list is the correct answer at this point. It means Ollama is running and you simply have no models yet, which the next chapter fixes.
Where models get stored
- macOS:
~/.ollama/models - Linux:
/usr/share/ollama/.ollama/models - Windows:
C:\Users\<you>\.ollama\models
Worth knowing for two reasons: that directory grows quickly, and it is what you back up if you want to keep a copy of a model independent of whether it stays published.
To put models on a different drive, set OLLAMA_MODELS to the
path you want before starting Ollama.