VRAM Is the Whole Conversation
Shopping for a GPU VPS is nothing like shopping for a regular VPS. Clock speed barely matters. Core count barely matters. There is one spec that decides everything: VRAM — how much memory the GPU has, because your model either fits in it or it doesn’t.
This guide won’t give you a ranked list of providers with affiliate links (we’re still evaluating providers ourselves). Instead, here’s the framework for comparing GPU VPS offerings so you can spot the good deals and the traps — because this market is full of traps.
How Much VRAM Do You Actually Need?
The math for inference: parameters × bytes-per-parameter, plus context overhead. In practice:
| Model size | Q4 quantized | Full precision (FP16) | Typical use |
|---|---|---|---|
| 8B | ~5 GB | ~16 GB | Chat, RAG, agents |
| 32B | ~20 GB | ~64 GB | Strong reasoning, coding |
| 70B | ~40 GB | ~140 GB | Frontier-class open models |
Fine-tuning needs roughly 3–4× the inference VRAM (optimizer states, gradients). If a provider’s marketing says “fine-tune 70B models!” on a 24 GB card, they’re selling you QLoRA and hoping you don’t read the fine print.
The GPU Tiers, Honestly Ranked
Consumer cards (RTX 3090 24GB, RTX 4090 24GB, RTX A6000 48GB). The price/performance kings for inference. A 4090 rents far cheaper than a datacenter card and is plenty for 8B–32B models. Downsides: no ECC memory (irrelevant for inference, mildly relevant for long training runs), and some providers oversubscribe them.
Datacenter cards (A100 40/80GB, H100 80GB). What you want for serious fine-tuning or serving 70B models at speed. NVLink, ECC, MIG partitioning — real enterprise features. Also real enterprise prices: an H100 can cost more per hour than your entire monthly VPS budget.
The “vGPU” trap. Some providers sell fractional GPUs (“1/4 of an A100!”). For inference this can be fine and cheap. For anything where you need the full memory bandwidth, it’s a disappointment. Always check whether you’re getting a whole card or a slice.
Hourly vs Monthly: Do the Math Both Ways
GPU VPS pricing comes in two flavors, and picking wrong is expensive:
- Hourly/billed-per-second: great for experiments, fine-tuning runs, and batch jobs. Spin up, run, destroy. A 3-hour fine-tuning session on an A100 might cost less than lunch.
- Monthly: only makes sense if the GPU runs 24/7 — a public inference endpoint, for example. If your utilization is under ~40%, hourly almost always wins.
Watch for spot/preemptible pricing: 50–70% cheaper, but the provider can reclaim the instance anytime. Perfect for checkpointed training; terrible for a production API.
The Comparison Checklist
When you’re lining up providers, compare these — in this order:
- VRAM per dollar. The master metric. Divide monthly (or hourly × 730) cost by GB of VRAM.
- Whole card or slice? Fractional GPUs should cost fractionally.
- Egress fees. Downloading a 140 GB model or serving lots of tokens? Some providers charge brutal bandwidth fees that dwarf the GPU cost.
- Setup fees and minimums. “From $0.50/hour*” with a $200/month minimum isn’t $0.50/hour.
- Availability. H100s are perpetually “coming soon” at budget providers. If you can’t actually provision it, the price is fiction.
- Software stack. Do you get root? Can you install your own CUDA/drivers, or are you stuck with their image? For Ollama and vLLM, you want root and a recent driver.
- CPU/RAM pairing. A great GPU bolted to 2 vCPU and 8 GB system RAM will bottleneck on data loading. Look for balanced configs.
Start on CPU, Rent GPU by the Hour
The smart money path: prototype on a cheap CPU VPS first (see running Llama on a $12 VPS). When you hit a wall — too slow, model too big — rent a GPU instance by the hour for the specific job, then kill it. Most people’s “need” for a GPU VPS is really a need for about six hours of A100 time per month.
If you haven’t bought any VPS yet, start with how to choose a VPS — the RAM-first advice applies double for AI work.