Chatting in a terminal is a demonstration. The reason to run models locally is to use them from your own software, and that is what this chapter sets up.
Ollama runs an HTTP server on port 11434 whenever it is running. No key, no account, no network.
Check it is there
curl http://localhost:11434/api/tags
That returns JSON listing your installed models. If it does, everything below will work.
A single request
curl http://localhost:11434/api/generate -d '{
"model": "llama3.1:8b",
"prompt": "Summarise the water cycle in two sentences.",
"stream": false
}'
"stream": false returns one complete JSON response. Leave it
out and you get tokens as they are produced, which is what you want behind a
user interface.
The part that saves you a rewrite
Ollama exposes an OpenAI-compatible endpoint. Any library that talks to OpenAI can usually be pointed at your local model by changing the base URL and passing a placeholder key.
Python:
from openai import OpenAI
client = OpenAI(
base_url="http://localhost:11434/v1",
api_key="ollama", # required by the client, ignored by the server
)
response = client.chat.completions.create(
model="llama3.1:8b",
messages=[{"role": "user", "content": "Write a one-line commit message for a bug fix"}],
)
print(response.choices[0].message.content)
PHP, with no SDK at all:
<?php
$payload = [
'model' => 'llama3.1:8b',
'messages' => [['role' => 'user', 'content' => 'Summarise this in one line: ...']],
'stream' => false,
];
$ch = curl_init('http://localhost:11434/v1/chat/completions');
curl_setopt_array($ch, [
CURLOPT_RETURNTRANSFER => true,
CURLOPT_HTTPHEADER => ['Content-Type: application/json'],
CURLOPT_POSTFIELDS => json_encode($payload),
]);
$result = json_decode(curl_exec($ch), true);
echo $result['choices'][0]['message']['content'] ?? 'no response';
The practical consequence is worth stating plainly: you can prototype against a hosted API and switch to a local model by changing two lines, or the reverse. You are not locked in either direction.
Keeping the model warm
By default Ollama unloads a model from memory after a few idle minutes, so the next request pays the loading cost again. For a service that needs to respond quickly:
curl http://localhost:11434/api/generate -d '{
"model": "llama3.1:8b",
"keep_alive": "30m"
}'
Or set OLLAMA_KEEP_ALIVE in the environment. The trade-off is
memory held continuously, which matters on a laptop and rarely on a
server.
Running it on a server for a team
By default Ollama listens only on localhost, which is the safe default. To serve a team, bind it wider:
OLLAMA_HOST=0.0.0.0:11434 ollama serve
Do not do this on anything internet-facing without protection in front of it. There is no authentication whatsoever: anyone who can reach the port can use your hardware and read anything your prompts contain. The next chapter covers this properly.