Run an LLM in your browser
You don't need a server or an API key to run a language model. A modern browser can download a small model and run it on your own GPU, entirely inside a web page. Here's how that works, what you need, and what a 1-billion-parameter model can and can't do.
How it works
- WebGPU is a browser API that lets web pages run programs on your graphics card, much like games and AI apps do natively.
- WebLLM (from the MLC project) compiles open models into WebGPU programs, and gives them an OpenAI-style chat API inside the page.
- The model's weights (its learned numbers, compressed to about 4 bits each) download once and are stored in the browser's cache. Later visits load from disk.
- The model runs in a Web Worker, a background thread, so the page stays responsive while it thinks.
After the download, nothing leaves your computer. Your prompts and the model's answers stay local.
What you need
- A browser with WebGPU. Recent Chrome and Edge on Windows, macOS and ChromeOS are the most reliable. Other browsers and Linux have been adding support, but may be incomplete or behind a setting.
- GPU memory roughly equal to the model's download size: about 1–2 GB for the models below. Laptops with integrated graphics share system RAM, so close heavy tabs and apps.
- One download of 0.9–2 GB per model. Use Wi-Fi.
Models that fit
The playground ships five models, all 2 billion parameters or less:
Model Params Download Good for
Llama 3.2 1B Instruct 1B ~0.9 GB lightest download
Qwen3 0.6B 0.6B ~1.4 GB fastest first try
Qwen2.5 1.5B Instruct 1.5B ~1.6 GB following tool formats
SmolLM2 1.7B Instruct 1.7B ~1.7 GB compact English chat
Qwen3 1.7B 1.7B ~2.0 GB best quality of the five
All run with a 4,096-token context window.
What to expect from a small model
Good at: casual chat, rewriting and summarising short text, simple questions, following a persona from the system prompt, and simple tool calls.
Weak at: facts (they confidently make things up), arithmetic, long multi-step reasoning, strict output formats and long conversations.
That's why tools matter so much for small models. A calculator fixes the arithmetic, a search tool fixes the facts. The model only has to decide when to use them.
Speed depends heavily on your GPU. Expect anything from a few to several dozen tokens per second for decoding. See prefill vs decode for why.
Common problems
- "WebGPU is not available": use an up-to-date Chrome or Edge. Some managed or older machines block it.
- "Device lost" or a crash while generating: the GPU ran out of memory or its driver gave up. Try a smaller model, close other GPU-heavy tabs, and update your graphics driver. On some Linux and integrated-GPU setups, WebGPU is still experimental and can fail even with small models.
- First load is slow: that's the one-time download. The next load comes from the cache in seconds.
- The model disappeared: clearing site data or browser storage removes the cached weights.
If your machine can't run models locally, the playground also connects to any OpenAI-compatible API, including a local Ollama server, which can run bigger models outside the browser.