Ollama

Ollama is a free, MIT-licensed runtime for downloading and running open LLMs locally through a single command, with an OpenAI-compatible API and an optional paid cloud tier for larger models.

local LLMopen modelsOllama runtimeOpenAI-compatible APIlocal inferenceprivacyGGUFllama.cpp alternativedeveloper tools

Visit Website

Overview

What is Ollama?

Ollama is an MIT-licensed runtime for running open models locally. It handles model download, loading, lifecycle management, and serving through both a dedicated REST API and an OpenAI-compatible endpoint, without requiring an account or API key for local use. The project is positioned around three principles: ownership of models, affordability through local hardware, and privacy because data never has to leave the machine.

Ollama does not train models or bind users to a single model provider. It is a layer that makes open weights usable as ordinary software: pull a model, run it, and call it from an application or agent framework. An optional cloud tier extends the same workflow to larger models hosted on Ollama’s infrastructure for users whose local hardware cannot fit them.

Workflow

How does Ollama work?

Users install the runtime for macOS, Linux, or Windows. A single command such as `ollama run llama3.2` pulls the model from Ollama’s registry, loads it, and opens a terminal chat session. For application integration, Ollama exposes a local API on port 11434, including an OpenAI-compatible `/v1/chat/completions` route, so existing SDKs and tools can target a local model with the same interface they use for hosted providers.

Model behavior can be adjusted through environment variables for context length, parallelism, keep-alive time, and host binding. By default, the server listens only on 127.0.0.1, and going beyond that requires adding a reverse proxy for authentication and TLS. For models that exceed local VRAM, users can switch to Ollama’s cloud models through the same CLI or API, with usage billed by compute time or tokens depending on the plan.

Use cases

Use Cases

Running open models such as Llama, Qwen, Gemma, DeepSeek, and Mistral entirely offline for privacy-sensitive development
Prototyping applications against a local OpenAI-compatible endpoint before switching to a hosted provider
Giving coding agents or IDE plugins a local model backend without per-token API costs
Experimenting with quantization levels and model sizes on consumer hardware
Offloading larger models to Ollama’s cloud when local VRAM is insufficient

Pricing

Pricing model: Freemium

Plan	Price	Billing	Included usage	Notes
Free	$0	local use	Run models locally with no usage limits, plus starter cloud credits and access to starter cloud models. No service fees for local execution.	—
Pro	$20	per month	$60 of monthly cloud usage credits, access to larger Pro models, and concurrent model execution. Available at $16.67/mo when billed annually ($200/yr).	—
Max	$100	per month	$300 of monthly cloud usage credits, early access to the newest models, and up to 10 concurrent requests.	—
Team	$500	per month	Unlimited users, $1,000 of shared monthly usage credits, centralized billing and administration, and priority support. Early access tier.	—
Enterprise	Custom	—	Volume usage pricing, model access controls, cost budgets per user or API key, and a private Slack channel with dedicated support.	—

Free tier: The desktop runtime is free to download and use locally without token-based billing. Cloud inference is available via pay-as-you-go usage credits, with a Free tier that includes starter credits and access to a limited set of models.

Limitations

Common Limitations

Hardware / VRAM · Local inference depends on available memory. At Q4_K_M quantization, a 7B–8B model needs roughly 8 GB of VRAM or RAM; a 70B model needs about 48 GB. Running entirely on CPU is possible but produces slower responses than GPU-assisted inference.
Context window · Ollama sets a default context length based on available VRAM: under 24 GiB defaults to 4K context; 24–48 GiB defaults to 32K; 48 GiB or more defaults to 256K. Longer contexts increase memory use and may not fit on lower-VRAM hardware.
No built-in authentication · The Ollama server does not provide native authentication. Binding beyond 127.0.0.1 requires a reverse proxy with TLS and an access token to secure the endpoint.
Small-model agentic performance · Ollama’s own performance notice states that models below 30 billion parameters have shown significantly lower performance on agentic tasks, which may affect multi-step tool use and autonomous workflows.
Cloud vs. local billing · Local execution has no per-token cost. Cloud model usage is billed via usage credits at per-million-token rates, with off-peak pricing available outside weekday peak hours and on weekends.

Features

Key Features

One-command model pulling and execution
OpenAI-compatible local API on port 11434
Support for GGUF models and a curated registry
CLI chat and background daemon operation
GPU acceleration on NVIDIA, AMD, Apple Silicon, and Vulkan devices

Comparison

Ollama vs. LM Studio and llama.cpp

This comparison focuses on documented differences in interface, licensing, model management, and deployment style among the three most common local LLM runtimes. It does not rank output quality or speed, which depend heavily on model, quantization, and hardware.

Tool	Positioning	Key differences	Best fit	Pricing note
Ollama Official overview Pricing Context length documentation	A daemon and model manager for running open models locally with a simple CLI and OpenAI-compatible API.	Combines one-command model pulling, lifecycle management, a background service, and an OpenAI-compatible endpoint. MIT-licensed and free for local use, with an optional paid cloud tier for larger models.	Developers who want a headless, scriptable local runtime that can be dropped behind existing agent frameworks and applications.	Local use is free under the MIT license. Cloud plans range from a Free tier with starter credits to Pro at $20/mo, Max at $100/mo, and Team at $500/mo.
LM Studio LM Studio is free for use at work Open WebUI comparison note (proprietary, free for personal and commercial use)	A desktop application for discovering, downloading, and running local LLMs through a graphical interface.	GUI-first experience with a built-in model browser, side-by-side prompt comparison, and one-click GPU offload. Proprietary software that is free for personal and commercial use. Uses llama.cpp as a backend and adds MLX on Apple Silicon.	Users who prefer a visual, workstation-oriented tool for browsing models and comparing prompts without writing CLI commands.	Free for personal and commercial use. The application is proprietary, not open-source.
llama.cpp Official GitHub repository (MIT license) Project description (LLM inference in C/C++)	A C/C++ inference engine for GGUF models, focused on minimal setup and performance across a wide range of hardware.	Provides the lowest-level control over quantization, sampling parameters, compilation flags, and GPU offloading. Does not include a model registry or background daemon by default; users compile the engine and manage model files themselves.	Developers who need fine-grained control over inference parameters or want to integrate a minimal engine into a custom stack.	MIT-licensed and free to use, modify, and distribute.

Comparison checked against official product information on 2026-11-04. Features and pricing may change.

FAQ

Frequently Asked Questions

Is Ollama free to use?

Yes, the Ollama runtime is MIT-licensed and free to use locally, with no per-token billing for local inference. Ollama also offers paid cloud plans and pay-as-you-go cloud model usage for larger models; those cloud services are separate from local use.

Does Ollama require an API key or account for local use?

No. Local execution does not require an account or API key. The official overview states that you can run models locally without accounts, API keys, or tokens billed by the thousand.

Can Ollama use a GPU?

Yes. Ollama supports GPU acceleration, including NVIDIA CUDA, AMD ROCm, Apple Silicon, and Vulkan devices. Without a compatible GPU, Ollama can run on CPU and system RAM, though responses will be slower.

What context length does Ollama use by default?

Ollama sets default context length based on available VRAM: under 24 GiB defaults to 4K, 24–48 GiB defaults to 32K, and 48 GiB or more defaults to 256K. The context window can be increased, but longer contexts consume more memory.

Is the Ollama API compatible with OpenAI’s API?

Ollama exposes an OpenAI-compatible route under `/v1` on port 11434, so applications and SDKs that target OpenAI’s chat completions format can often point to Ollama with minimal changes. Streaming, tool calls, and structured output are supported on that route.