Self-Hosting Ollama: From Install to Secure API

By /

Your Model, Your Server, Your Rules

Running ollama run on a VPS is a fun demo. Self-hosting Ollama means turning it into infrastructure: reachable from your devices, secured properly, serving an API your scripts and apps can rely on. This guide takes you from the basic install to a production-worthy setup.

New to this entirely? Start with running Llama on a $12 VPS, then come back — this guide assumes Ollama is already installed and a model is pulled.

Step 1: Understand What Ollama Runs

The installer creates a systemd service (ollama.service) running as the ollama user. By default it binds to 127.0.0.1:11434 — localhost only. Check it:

systemctl status ollama
curl http://localhost:11434/api/tags

Configuration happens through environment variables. The useful ones:

  • OLLAMA_HOST — bind address. 0.0.0.0:11434 listens on all interfaces.
  • OLLAMA_MODELS — where model blobs live (default /usr/share/ollama/.ollama/models).
  • OLLAMA_KEEP_ALIVE — how long models stay loaded in RAM (default 5m).
  • OLLAMA_NUM_PARALLEL — concurrent requests (default 1; raise it if you have the RAM).

Set them with a systemd override — never edit the shipped unit file directly:

mkdir -p /etc/systemd/system/ollama.service.d
cat > /etc/systemd/system/ollama.service.d/override.conf <<'EOF'
[Service]
Environment="OLLAMA_HOST=127.0.0.1:11434"
Environment="OLLAMA_KEEP_ALIVE=24h"
EOF
systemctl daemon-reload && systemctl restart ollama

Note we kept it on localhost. That’s deliberate — read on.

Step 2: Do NOT Expose Port 11434 to the Internet

This is the single most important section. Ollama’s API has no authentication. Anyone who can reach port 11434 can list your models, run prompts, pull new models onto your disk, and burn your CPU. Shodan is full of exposed Ollama instances — don’t join them.

You have three sane options for remote access:

Option A: Tailscale / private network (easiest, safest). If your devices share a tailnet or VPC, bind Ollama to the private interface and skip public exposure entirely. No TLS certs, no auth proxy, no attack surface.

Option B: Reverse proxy with TLS + basic auth. Keep Ollama on localhost, put nginx or Caddy in front:

# nginx snippet
server {
    listen 443 ssl;
    server_name ollama.example.com;

    ssl_certificate     /etc/letsencrypt/live/ollama.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/ollama.example.com/privkey.pem;

    auth_basic "Ollama";
    auth_basic_user_file /etc/nginx/.ollama_htpasswd;

    location / {
        proxy_pass http://127.0.0.1:11434;
        proxy_set_header Host $host;
    }
}

Option C: API key via proxy. If basic auth feels 1997, Caddy with a basic_auth or header-check snippet works the same way. The principle is identical: something authenticated must sit between the internet and Ollama.

Step 3: Use the OpenAI-Compatible API

Ollama speaks the OpenAI chat-completions dialect at /v1/chat/completions, which means your existing tooling just works — point it at your server instead of api.openai.com:

curl https://ollama.example.com/v1/chat/completions \
  -u "user:yourpassword" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama3.1:8b",
    "messages": [{"role": "user", "content": "Explain systemd socket activation briefly."}]
  }'

The native /api/generate endpoint offers more Ollama-specific knobs (raw templates, system overrides); the /v1 endpoint offers compatibility. Use whichever fits the client.

Step 4: Manage Models Like Artifacts

ollama list              # what's installed
ollama pull qwen2.5:14b  # add one (needs ~9GB RAM)
ollama rm llama3.2:1b    # remove one
ollama ps                # what's currently loaded in RAM

For repeatable setups, write a Modelfile — it’s a Dockerfile for models:

FROM llama3.1:8b
PARAMETER temperature 0.7
PARAMETER num_ctx 8192
SYSTEM "You are a terse systems assistant. Answer with commands first, explanations second."
ollama create sysadmin -f Modelfile
ollama run sysadmin

Back up $OLLAMA_MODELS if re-pulling multi-GB blobs would hurt — otherwise treat models as disposable and keep the Modelfiles in git.

Step 5: Keep It Updated and Watched

  • Updates: re-run the install script; it upgrades in place. Then ollama pull your models again to get fresh quantizations.
  • Logs: journalctl -u ollama -f — inference errors and OOM kills show up here.
  • Monitoring: ollama ps in a cron or your monitoring agent tells you what’s loaded; alert on the service being down, not on much else.
  • Firewall: 11434 should be reachable only from localhost (or your tailnet). Verify with ss -tlnp | grep 11434 — if it says 0.0.0.0:11434 without a proxy in front, fix that today.

What This Gets You

A private inference endpoint: no per-token billing, no data leaving your infrastructure, no rate limits, no deprecation emails. Point your scripts, your IDE plugin, or a web UI like Open WebUI at it and it’s just… yours.

Total build time from a fresh VPS is under an hour, most of it waiting on model downloads. If you haven’t picked the VPS yet, start with the VPS guide — RAM first, everything else second.

Scroll to Top