Run AI on your own machine.

To run an LLM locally, install a free app such as Ollama or LM Studio, download an open-weight model that fits in your computer’s memory, and start chatting. An LLM, or large language model, is the kind of AI behind chatbots like ChatGPT and Claude. Once the model is on your machine, it works offline and nothing you type leaves your computer.

Memory decides what you can run. As a rough guide, 16 GB is enough for capable small models, and 32 GB opens up the 27 to 31 billion parameter class, a real step up in quality.

01Why run an LLM locally?

Privacy is the most common reason. If you work with client files, health records or unreleased code, a local model means there is no third party to trust. For the wider picture, see our AI privacy guide.

02The four tools worth knowing

All four are free. They differ mainly in how much they do for you.

Start with LM Studio if you prefer clicking, or Ollama if you are comfortable in a terminal. Move to llama.cpp or MLX when you want fine control over every setting.

03Which open models should you try in 2026?

As of September 2026, three families stand out for computers you might own. All three use Apache 2.0, a permissive license that allows commercial use.

The open frontier is far bigger. DeepSeek V4-Pro has 1.6 trillion parameters. It is free to download under the MIT license, but it needs server hardware with hundreds of gigabytes of memory.

04How much memory do you need?

A model’s size is counted in parameters, the learned numbers inside it. More parameters usually mean a smarter model, and always mean a bigger file.

At that setting, plan on roughly 0.6 to 0.7 GB of memory per billion parameters. Then add a few gigabytes for the conversation, your operating system and your other apps. Real download sizes show the rule:

On a Mac, Apple silicon shares one pool of memory between the processor and the graphics chip, so total memory is what counts. On a Windows or Linux PC, speed depends on the graphics card’s own memory, called VRAM. A model that fits in VRAM runs fast. One that spills into regular memory still runs, but much more slowly.

Long conversations need memory too, because the model keeps working notes on everything in its context window. If a big document makes a model crawl, lower the context length setting or pick a smaller model.

05Run your first local model in five minutes

1. 01

   Install Ollama

   Download the app from ollama.com. On a Mac or Linux, you can instead run the install script from the project’s README:

   curl -fsSL https://ollama.com/install.sh | sh

   On Windows, the README gives a PowerShell command: irm https://ollama.com/install.ps1 | iex.

2. 02

   Download a model and start chatting

   Run ollama run gemma4. The first time, it downloads Gemma 4 E4B, about 9.6 GB, then opens a chat. After that, it loads from your disk and works offline.

3. 03

   Ask it something

   Type a question and press Enter. To leave the chat, type /bye or press Ctrl+D.

4. 04

   Manage your models

   Each model takes gigabytes of disk space, so remove the ones you stop using. The commands below cover the everyday jobs.

# Ask one question without opening a chat
ollama run gemma4 "Explain quantization in two sentences."

# List downloaded models, and the ones loaded in memory right now
ollama list
ollama ps

# Unload a model from memory, or delete it from disk
ollama stop gemma4
ollama rm gemma4

Prefer LM Studio? Search for a model in the app, download it and open a chat. Its lms command line does the same: lms get openai/gpt-oss-20b downloads a model in a version suited to your hardware, and lms chat opens a chat.

06Use a local model from your own apps

Local tools also run a small server on your computer, so scripts and apps can use a model the way they would use a cloud API. Ollama listens at http://localhost:11434:

curl http://localhost:11434/api/chat -d '{
  "model": "gemma4",
  "messages": [{ "role": "user", "content": "Write a haiku about local AI." }],
  "stream": false
}'

LM Studio (lms server start) and llama.cpp both offer servers compatible with OpenAI’s API format. Many apps built for cloud models can point at your own machine instead: change the address, and your data stays home.

If you want to go a level deeper, these are the quick-start commands from the llama.cpp and mlx-lm READMEs as of September 2026:

# llama.cpp: download a model from Hugging Face and chat with it
llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF

# llama.cpp: serve the same model through an OpenAI-compatible API
llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF

# MLX on Apple silicon: install, then chat with the default small model
pip install mlx-lm
mlx_lm.chat

Older guides use separate llama-cli and llama-server programs. The current README uses llama cli and llama serve.

07When local is not worth it

Local is usually the wrong choice when:

• You need the strongest answers on hard problems. Laptop-sized models are far smaller than the leading cloud models.

• You work with very long documents. A long context eats memory quickly.

• Your computer is modest. An 8 GB machine only runs tiny models, and answers arrive slowly.

• You are on battery. Long generations drain a laptop and make it hot.

• You want zero upkeep. You choose, update and remove models yourself.

A hybrid works well for many people: a local model for private or routine work, and a cloud model when a task gets hard. Our guide on how to choose an AI model covers the cloud side.

08FAQ

Next, weigh open vs closed AI models, or read what happens to what you type in cloud AI tools.