Lesson 8 of 8 · 3 min read

Run an LLM in your browser

You don't need a server or an API key to run a language model. A modern browser can download a small model and run it on your own GPU, entirely inside a web page. Here's how that works, what you need, and what a 1-billion-parameter model can and can't do.

How it works

  1. WebGPU is a browser API that lets web pages run programs on your graphics card, much like games and AI apps do natively.
  2. WebLLM (from the MLC project) compiles open models into WebGPU programs, and gives them an OpenAI-style chat API inside the page.
  3. The model's weights (its learned numbers, compressed to about 4 bits each) download once and are stored in the browser's cache. Later visits load from disk.
  4. The model runs in a Web Worker, a background thread, so the page stays responsive while it thinks.

After the download, nothing leaves your computer. Your prompts and the model's answers stay local.

What you need

Models that fit

The playground ships five models, all 2 billion parameters or less:

Model                    Params   Download   Good for
Llama 3.2 1B Instruct    1B       ~0.9 GB    lightest download
Qwen3 0.6B               0.6B     ~1.4 GB    fastest first try
Qwen2.5 1.5B Instruct    1.5B     ~1.6 GB    following tool formats
SmolLM2 1.7B Instruct    1.7B     ~1.7 GB    compact English chat
Qwen3 1.7B               1.7B     ~2.0 GB    best quality of the five

All run with a 4,096-token context window.

What to expect from a small model

Good at: casual chat, rewriting and summarising short text, simple questions, following a persona from the system prompt, and simple tool calls.

Weak at: facts (they confidently make things up), arithmetic, long multi-step reasoning, strict output formats and long conversations.

That's why tools matter so much for small models. A calculator fixes the arithmetic, a search tool fixes the facts. The model only has to decide when to use them.

Speed depends heavily on your GPU. Expect anything from a few to several dozen tokens per second for decoding. See prefill vs decode for why.

Common problems

If your machine can't run models locally, the playground also connects to any OpenAI-compatible API, including a local Ollama server, which can run bigger models outside the browser.