Saturday, 21 November 2026 Press Releases Newsletter RSS
Breaking
WhatsApp Business Account Hacked in Pakistan: Full Recovery Guide Special Technology Zones Authority (STZA) & IT Park Expansion in Pakistan: Islamabad, Karachi, and Lahore Ground Report Pakistan Software Export Board & Industry Consortium Launch 'TechBridge Pakistan 2026' to Connect 500 Startups With Global Markets Global Tech Safety Summit 2026: International Standards for Frontier AI and Data Security Announced WhatsApp Business Account Hacked in Pakistan: Full Recovery Guide Special Technology Zones Authority (STZA) & IT Park Expansion in Pakistan: Islamabad, Karachi, and Lahore Ground Report Pakistan Software Export Board & Industry Consortium Launch 'TechBridge Pakistan 2026' to Connect 500 Startups With Global Markets Global Tech Safety Summit 2026: International Standards for Frontier AI and Data Security Announced
WritePures

Chatting in a terminal is a demonstration. The reason to run models locally is to use them from your own software, and that is what this chapter sets up.

Ollama runs an HTTP server on port 11434 whenever it is running. No key, no account, no network.

Check it is there

curl http://localhost:11434/api/tags

That returns JSON listing your installed models. If it does, everything below will work.

A single request

curl http://localhost:11434/api/generate -d '{
  "model": "llama3.1:8b",
  "prompt": "Summarise the water cycle in two sentences.",
  "stream": false
}'

"stream": false returns one complete JSON response. Leave it out and you get tokens as they are produced, which is what you want behind a user interface.

The part that saves you a rewrite

Ollama exposes an OpenAI-compatible endpoint. Any library that talks to OpenAI can usually be pointed at your local model by changing the base URL and passing a placeholder key.

Python:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:11434/v1",
    api_key="ollama",            # required by the client, ignored by the server
)

response = client.chat.completions.create(
    model="llama3.1:8b",
    messages=[{"role": "user", "content": "Write a one-line commit message for a bug fix"}],
)

print(response.choices[0].message.content)

PHP, with no SDK at all:

<?php

$payload = [
    'model' => 'llama3.1:8b',
    'messages' => [['role' => 'user', 'content' => 'Summarise this in one line: ...']],
    'stream' => false,
];

$ch = curl_init('http://localhost:11434/v1/chat/completions');
curl_setopt_array($ch, [
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_HTTPHEADER => ['Content-Type: application/json'],
    CURLOPT_POSTFIELDS => json_encode($payload),
]);

$result = json_decode(curl_exec($ch), true);
echo $result['choices'][0]['message']['content'] ?? 'no response';

The practical consequence is worth stating plainly: you can prototype against a hosted API and switch to a local model by changing two lines, or the reverse. You are not locked in either direction.

Keeping the model warm

By default Ollama unloads a model from memory after a few idle minutes, so the next request pays the loading cost again. For a service that needs to respond quickly:

curl http://localhost:11434/api/generate -d '{
  "model": "llama3.1:8b",
  "keep_alive": "30m"
}'

Or set OLLAMA_KEEP_ALIVE in the environment. The trade-off is memory held continuously, which matters on a laptop and rarely on a server.

Running it on a server for a team

By default Ollama listens only on localhost, which is the safe default. To serve a team, bind it wider:

OLLAMA_HOST=0.0.0.0:11434 ollama serve

Do not do this on anything internet-facing without protection in front of it. There is no authentication whatsoever: anyone who can reach the port can use your hardware and read anything your prompts contain. The next chapter covers this properly.