Saturday, 21 November 2026 Press Releases Newsletter RSS
Breaking
WhatsApp Business Account Hacked in Pakistan: Full Recovery Guide Special Technology Zones Authority (STZA) & IT Park Expansion in Pakistan: Islamabad, Karachi, and Lahore Ground Report Pakistan Software Export Board & Industry Consortium Launch 'TechBridge Pakistan 2026' to Connect 500 Startups With Global Markets Global Tech Safety Summit 2026: International Standards for Frontier AI and Data Security Announced WhatsApp Business Account Hacked in Pakistan: Full Recovery Guide Special Technology Zones Authority (STZA) & IT Park Expansion in Pakistan: Islamabad, Karachi, and Lahore Ground Report Pakistan Software Export Board & Industry Consortium Launch 'TechBridge Pakistan 2026' to Connect 500 Startups With Global Markets Global Tech Safety Summit 2026: International Standards for Frontier AI and Data Security Announced
WritePures
Beginner 10 chapters about 25 minutes By Bakhtawar 21 Nov 2026
Key figures for running open-weight AI models on local hardware

Open-weight models such as Llama, Mistral, Qwen and DeepSeek have already been downloaded onto private servers and laptops in numbers nobody can count. Whatever happens to the companies that released them, those copies keep working. No vendor can switch them off, no government can recall them, and no price change can reach them.

That is the practical argument for learning this. A model running on your own hardware costs nothing per token, works without a connection, and never sends a word of your data to anyone else. For legal work, medical notes, client records, exam scripts or anything you simply would rather not upload, that last point is the whole game.

This tutorial takes you from an empty machine to a working local model you can use from a chat window and from your own code. It assumes no prior experience with AI tooling. It does assume you can open a terminal and paste a command.

What you will end up with

  • A local runtime installed and verified.
  • Llama, Mistral, Qwen and DeepSeek downloaded and running.
  • An understanding of which model size your hardware can actually take.
  • A local HTTP API your own applications can call.
  • A clear view of the licensing and security limits, which are real.

Before you start

You need about 10 GB of free disk space for a first model, more if you collect several. You need a terminal. You do not need a GPU — chapter two explains what changes if you have one.

One honest warning before you begin: model tags and version numbers change faster than any written guide. Every command below is current at the time of writing, and the right habit is to check ollama.com/library for the exact tag before pulling a model, rather than trusting a tag copied from a blog post months later.

Chapters

Start with chapter 1 →

Frequently asked questions

No. A 7B model at Q4_K_M needs roughly 4–6 GB and runs on an ordinary laptop CPU, slowly but usably — think a few words per second rather than instant. A GPU makes it dramatically faster rather than making it possible, and Apple Silicon machines are unusually good at this because the CPU and GPU share one pool of memory.

Running them is legal and the downloads are free, but "open weight" is not the same as "open source". Each model carries its own licence, and several impose conditions — restrictions on commercial use above a user threshold, or on using outputs to train competing models. Chapter ten covers this. Read the licence of the specific model before you build a business on it.

Not the copies already downloaded. A publisher can remove a model from its hub, change the licence on future releases, or stop publishing altogether, and none of that reaches the weights already sitting on your disk. That is the genuinely novel property of open weights, and it is why keeping your own copy of a model you depend on is a reasonable precaution.

Not at the sizes most people can run. A 7B or 8B model on a laptop is capable at summarising, drafting, extraction, classification and simple coding, and noticeably weaker at long reasoning chains. The honest framing is that local models are good enough for a large share of ordinary work, free, and private — not that they are equivalent to a frontier model.

Sources & references

  1. Ollama — model library
  2. Ollama — download and install
  3. Hugging Face — open model hub