Run a GGUF in your browser
A quantized language model runs on your own CPU through WebAssembly. Nothing is sent anywhere, there is no server behind this page, and the model is cached after the first load. Pick a quantization, load it, and watch how the size changes the speed and the answers.
Q4_K_M is the usual balance between quality and size.
Nothing loaded yet. The first load downloads the file once.
load -
first token -
speed -
tokens -
The model runs through wllama, which is llama.cpp compiled to WebAssembly. Tokens stream as they are produced. Your browser caches the file, so switching back to a quantization you already loaded skips the download. Weights are read only, the only thing written is the model cache in this browser.