Why is Ollama so slow?

“Slow” can mean a long wait before the first token, slow token generation, or a model that is repeatedly loading and unloading. Identify which one you have before changing settings.

1. Check processor placement

Start a model, then run this in a second terminal:

ollama ps

If the output includes a processor column, check whether the model is on CPU, GPU or split between them. The answer helps decide whether to investigate driver support or memory pressure first.

2. Compare with a smaller installed model

ollama list

Try a smaller model you already have. If it responds much faster, the larger model may simply need more compute or memory. Model size alone is not a perfect speed predictor.

3. Notice context length

Long conversations and large context windows can increase memory use and affect performance. For a fair test, start a fresh conversation with a short prompt, and compare the same prompt across models.

4. Check other workloads

Close games, GPU-heavy applications or other model servers temporarily. For NVIDIA, nvidia-smi can show processes and GPU memory use. Other vendors have their own monitoring tools.

5. Use a repeatable test

1. Restart Ollama and let the machine settle.
2. Run the same short prompt with the same model.
3. Note the delay before the first response and the generation speed.
4. Repeat with a smaller model and compare.

A single speed number without the model, quantization, context and hardware is not very meaningful.

Official reference: Ollama documentation.