VPS Tutorials
GPU VPS Recommendations 2026: How to Choose GPUs and Configurations for Ollama, ComfyUI and AI Agents
A GPU VPS buying and configuration guide: compare GPU cloud, GPU VPS and dedicated servers, choose VRAM, CPU, RAM, disk and billing for Ollama, ComfyUI, LoRA fine-tuning and AI Agent scenarios, with price estimates, a security checklist and troubleshooting.
GPU VPS Recommendations 2026: How to Choose GPUs and Configurations for Ollama, ComfyUI and AI Agents
If you want to run Ollama, ComfyUI, Stable Diffusion or an AI Agent on a VPS, the real difficulty is not typing a Docker command — it is buying the right GPU first. Many people only look at the "GPU model" and the "price per hour", while ignoring VRAM, disk read/write speed, drivers, whether the instance will be reclaimed, and whether data survives after shutdown.
This article is written around the buying decision: first determine which type of GPU you need, then give configuration starting points for Ollama, ComfyUI, training and AI Agents, and finally compare GPU cloud instances, GPU VPS and dedicated GPU servers. Prices change with inventory and region; the numbers in this article are only for building a budget and should not be treated as long-term quotes.
If your goal is "occasionally run local models, do image generation, test workflows", prioritize hourly-billed GPU cloud instances rather than signing up for a one-year GPU VPS right away. This lets you stop when done and first verify the model, drivers and actual VRAM requirements.
- Ollama running 7B/8B quantized models: 24GB VRAM is a comfortable starting point; RTX 3090, RTX 4090, L4 or equivalent GPUs are all worth considering.
- Ollama running 14B/32B quantized models: prioritize 48GB VRAM; if the model needs a larger context or higher concurrency, consider 80GB+ or multi-GPU setups.
- ComfyUI image generation: SDXL usually starts from 16GB–24GB VRAM; for complex workflows, upscaling or batch generation, 24GB is more stable.
- LoRA fine-tuning and small-scale training: 24GB works for entry-level experiments; 32GB–48GB is better for reducing the hassle of batch size and VRAM optimization.
- Long-running online AI Agents, APIs or team services: you cannot look at GPU alone; at minimum also consider 8 vCPU, 32GB RAM, NVMe, a fixed public IP, persistent disks and a backup strategy.
A practical principle: buy by VRAM first, then optimize by GPU compute. No matter how strong the GPU compute is, if the model does not fit in VRAM, it ultimately will not run.
"GPU VPS" may mean completely different products at different providers. Before ordering, confirm the virtualization method and billing method.
| Type | Billing | Suitable scenarios | Main risks |
|---|---|---|---|
| GPU cloud instance | Per second, per hour or per month | Model testing, short tasks, elastic inference | Whether disks and public IP continue to be billed after shutdown |
| GPU VPS | Monthly or pay-as-you-go, usually with a fixed system environment | Long-running Ollama, ComfyUI, personal APIs | Whether the GPU is exclusive, whether it is oversold, whether drivers can be changed |
| Dedicated GPU server | Monthly, lease or colocation | Stable high concurrency, training, team sharing | Long minimum lease term, high machine and bandwidth costs |
| On-demand GPU marketplace | Provider bidding, per second or per hour | Low-cost batch processing, fault-tolerant tasks | Preemption, node quality, large differences in network and data persistence |
Taking the on-demand GPU marketplace as an example, Vast.ai officially states that its prices are determined by supply and demand, and distinguishes On-Demand, Interruptible and Reserved: Interruptible may be reclaimed, while Reserved suits stable tasks. RunPod also bills Pods, Serverless and Clusters separately, corresponding to long-running instances, inference workloads and multi-node scenarios. Do not directly compare the per-inference price of Serverless with the monthly fee of a long-running VPS.
The configurations below are buying starting points, not a vendor plan list. The same model on different cloud platforms may have completely different VRAM, PCIe generation, CPU quota, disk and network.
| Goal | Recommended VRAM | CPU / RAM starting point | Disk recommendation | Suitable instances |
|---|---|---|---|---|
| Ollama 7B/8B quantized | 16GB–24GB | 4 vCPU / 16GB | 80GB NVMe | RTX 4000, L4, 3090, 4090 |
| Ollama 14B/32B quantized | 24GB–48GB | 8 vCPU / 32GB | 120GB NVMe | 4090, A6000, A40, L40S |
| ComfyUI / SDXL | 16GB–24GB | 8 vCPU / 16GB | 100GB NVMe | 3090, 4090, L4, A5000 |
| LoRA fine-tuning | 24GB–48GB | 8–16 vCPU / 32–64GB | 200GB NVMe | 4090, A6000, A40, L40S |
| Multi-user inference API | 48GB–80GB | 16 vCPU / 64GB | 200GB+ NVMe | L40S, A100, H100 |
| Multi-GPU training | 80GB+ | 32 vCPU / 128GB+ | 500GB+ NVMe | A100, H100, B200, etc. |
8GB VRAM can run small models, low-resolution images and some quantized inference, but the headroom is very small. After increasing context length, loading ControlNet, enabling image upscaling or concurrent requests, the common result is CUDA out of memory. Unless you have confirmed that the workflow is fixed and the tasks are very light, 16GB is a more reasonable minimum, and 24GB is the sweet spot for most individual users.
Ollama's model size is not the only metric; you also need to consider quantization format, context length, concurrency and KV cache. An 8B Q4 model may fit in 8GB VRAM, but that does not mean it can stably run long contexts, tool calls and multiple sessions on the same machine.
For a single user with low concurrency, you can start from 24GB VRAM and reserve 80GB–120GB for the system disk. Model files, Docker images and logs will continuously take up space, and a disk that is too small often becomes a problem earlier than VRAM. When running the API, have Ollama listen only on the internal network or localhost, and let Caddy, Nginx or an application gateway provide HTTPS, authentication and rate limiting.
If you want to stably run 14B or 32B quantized models, 48GB VRAM is more worry-free. 24GB can also run them via CPU offload or more aggressive quantization, but speed and concurrency will be noticeably affected. Before buying, confirm whether the platform provides complete NVIDIA drivers, whether the CUDA version is compatible with containers, and whether custom Docker images are allowed.
AI Agents usually do not only occupy the GPU: vector databases, browsers, code sandboxes, queues, logs and file uploads all consume CPU, RAM and disk. For a single GPU VPS, at least 8 vCPU and 32GB RAM are recommended; if you also run PostgreSQL, Redis, SearXNG or multiple Workers, 64GB RAM is recommended, or split the database and object storage out.
ComfyUI's VRAM consumption depends on the base model, resolution, sampler, ControlNet, LoRA, VAE and upscaling nodes. Do not judge whether the configuration is sufficient only by "it can start".
- Only SD 1.5, small-size images: 12GB–16GB can be tried;
- SDXL, regular 1024×1024: 16GB–24GB is more suitable;
- Multiple ControlNets, high-res fix, upscaling and batch queues: 24GB–48GB;
- Video generation, large Flux models or complex node graphs: evaluate based on 48GB VRAM first, and use 80GB-class GPUs when necessary.
ComfyUI's model directory should be placed on a persistent disk or a separate Volume. After a temporary instance is reclaimed, the container may still exist, but the models, output images and custom nodes may not. When buying, confirm: whether the system disk and data disk are separated, whether the Volume is retained across instances, whether snapshots are billed, and whether the public IP is released after deleting the instance.
Do not just multiply "price per hour × 30 days". The full cost includes at least the GPU, CPU/RAM, system disk, persistent Volume, public IPv4, outbound traffic, snapshots and idle instance fees.
For example, based on the example prices on RunPod's current public page, RTX 3090 is about $0.50/hour, RTX 4090 about $0.69/hour, A40 about $0.44/hour, and L40S about $0.99/hour; if running all day, the monthly GPU compute costs are roughly $360, $497, $317 and $713 respectively, and actual prices vary with region, inventory and instance type. Hourly instances suit intermittent use; before running 24×7 long-term, you should compare against monthly dedicated servers or Reserved prices.
You can estimate with this formula:
月成本 = GPU小时价 × 实际运行小时 + 磁盘/月 + Volume/月 + IP/月 + 出站流量 + 快照
If you only use it in the evening, running 4 hours a day, about 120 hours a month, hourly GPUs are often more cost-effective than renting all day. Conversely, if the inference API needs to be online all day, the price and stability of a monthly exclusive GPU are usually easier to control.
- Is the GPU exclusive, partitioned or shared?
- Are the VRAM capacity and GPU model clearly stated on the order page?
- Will the instance be preempted or reclaimed? Is there notification before reclamation?
- After shutdown, are the GPU, disk, IP and Volume still billed?
- Can official NVIDIA drivers and Docker be used, and what is the CUDA version?
- Are custom kernels, containers and ports allowed?
- Is the public IPv4 fixed, and does it change after reinstalling the system?
- Is the data disk independent of the compute instance, and can it be retained after deleting the instance?
- How are snapshots, backups and cross-region replication billed respectively?
- Is there a limit on outbound traffic, and how is it billed or throttled after exceeding it?
- Is there a refund, migration, GPU replacement and failure compensation policy?
- Does the provider offer a console, VNC or rescue mode, and can you recover yourself if the GPU driver breaks?
If customer service cannot clearly answer "is the GPU exclusive" and "is data retained after the instance is deleted", do not order just because it is cheap. A few dollars difference in GPU price is far less than the cost of one data loss and environment reinstall.
GPU servers usually expose Jupyter, ComfyUI, Open WebUI or API ports, and directly exposing management interfaces to the public internet is dangerous.
- Allow only key-based SSH login, disable password login and restrict management IPs;
- Use Caddy or Nginx for HTTPS, and do not expose databases, Redis and the Docker Socket to the public internet;
- Add authentication to ComfyUI, Jupyter, Open WebUI and the Ollama API, or put them behind a VPN;
- Back up model files and user-uploaded files separately; do not back up only the container;
- Set alerts for GPU, disk, VRAM, temperature, container restarts and API errors;
- Set budget, balance and abnormal usage alerts for pay-as-you-go accounts;
- Pin image versions, and create snapshots before upgrading CUDA, drivers and PyTorch;
- Training and batch jobs should save checkpoints regularly to prevent starting over after a preempted instance is reclaimed.
First confirm that the instance is actually allocated a GPU, then check the host driver, container runtime and NVIDIA Container Toolkit. Do not reinstall PyTorch right away; if the host driver is missing, installing more CUDA packages inside the container will not help.
In order, reduce batch size, resolution and context length, disable unnecessary ControlNet or concurrency, and then confirm whether other processes are occupying VRAM. For long-term use, upgrading VRAM is usually more time-saving than repeatedly changing parameters.
Check whether the GPU is shared, whether power is limited, whether the PCIe link is degraded, and whether the CPU has become a bottleneck for data preprocessing. For AI Agents, also check model API latency, vector retrieval and browser tools, not just GPU utilization.
This is usually because models were written into a temporary container layer or temporary disk. Map the model directory to a persistent Volume, save the Compose file, environment variables, plugin list and startup scripts, and do off-site backups regularly.
If you are running local AI for the first time, it is recommended to start with an hourly GPU instance of 24GB VRAM, 8 vCPU, 32GB RAM, 120GB NVMe, and first verify the real workflow with Ollama or ComfyUI. If you only use it a few hours a day, start and stop on demand; if you need an all-day API service, then compare monthly GPU VPS, Reserved instances and dedicated servers.
When the budget is limited, prioritize VRAM and persistent disks, and do not sacrifice data safety for a higher model. For large models, multi-user and training tasks, directly choose 48GB, 80GB or even multi-GPU solutions, and include databases, object storage and backups in the total cost.
As a next step, list your models, target concurrency, daily running hours and whether you need a public API, then use these four items to filter GPU VPS. This is less likely to lead to a wrong purchase than looking at a "GPU price ranking" first.