Ollama
Ollama is a tool for downloading, running, and serving open-weight large language models locally or through its own hosted cloud, exposing them through a command-line interface and a local REST API.
Baseline topologies
| Topology | Minimum nodes | Roles |
|---|---|---|
| single-node-cpu | 1 | ollama |
| single-node-gpu | 1 | ollama-gpu |
What drives the cost
- Peak concurrent chats
- Simultaneous users or requests generating tokens at the same time.
- Largest model size (B parameters)
- Largest model you plan to serve, in billions of parameters. Main driver for RAM or VRAM needs.
Where it runs
bare_metal, dedicated, cloud_vm
Size Ollama and get a cost estimate