Deploy a LiteLLM Unified LLM API Gateway on a VPS (2026): Docker, Virtual Keys, Budget Control and Multi-Model Routing Guide

If you're just using one OpenAI Key yourself, LiteLLM may be a bit redundant.

But as soon as any of the following situations appear, it becomes very valuable:

My own judgment is simple:

• Personal single-Key use: not necessary
• Multiple projects, multiple models, multiple members: very necessary
• Want to build an internal AI API platform: worth deploying

It is not a chat interface, nor an AI application platform. It is more like an LLM API gateway.

LiteLLM itself is not heavy; what really consumes resources is PostgreSQL, Prometheus, request concurrency and log volume.

If you're only doing proxying and not running local models, 2C4G is basically enough.

Don't lump it together with Ollama when calculating resources. LiteLLM is a forwarding gateway and is not responsible for local inference. What you connect to are remote APIs like OpenAI, DeepSeek, Claude, so its own CPU pressure is not high.

Don't rush to copy the Compose.

There are three things in LiteLLM you need to decide from the start:

1. LITELLM_MASTER_KEY
2. LITELLM_SALT_KEY
3. The model naming rules in config.yaml

MASTER_KEY is the admin key and must start with sk-.

SALT_KEY is more important; it is used to encrypt the model API Keys in the database. The official docs also warn: after adding models, don't change this value casually. If you change it, the old encrypted credentials may no longer be decryptable.

So don't write:

LITELLM_SALT_KEY=sk-1234

Instead, generate a longer random value:

openssl rand -hex 32

Then write it into .env.

This article uses Ubuntu / Debian as an example.

apt update && apt upgrade -y
apt install -y ca-certificates curl gnupg ufw nano

Install Docker:

install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | gpg --dearmor -o /etc/apt/keyrings/docker.gpg
chmod a+r /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo $VERSION_CODENAME) stable" > /etc/apt/sources.list.d/docker.list
apt update
apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

Confirm:

docker --version
docker compose version

mkdir -p /opt/litellm
cd /opt/litellm

Generate two Keys:

echo "sk-$(openssl rand -hex 32)"
echo "sk-$(openssl rand -hex 32)"

One for LITELLM_MASTER_KEY, one for LITELLM_SALT_KEY.

Create .env:

cat > .env << 'EOF'
LITELLM_MASTER_KEY="sk-replace-with-master-key"
LITELLM_SALT_KEY="sk-replace-with-long-salt-key"
OPENAI_API_KEY="sk-replace-with-openai-key"
DEEPSEEK_API_KEY="replace-with-deepseek-key"
POSTGRES_PASSWORD="replace-with-postgres-password"
EOF

If you're not using OpenAI or DeepSeek for now, you can leave them empty for the moment, but MASTER_KEY and SALT_KEY must be filled in carefully.

config.yaml is the core of LiteLLM.

It determines:

• What the externally visible model names are
• Which provider it actually forwards to
• Where the API Key is read from
• How the database and backend settings are configured

Create the file:

nano config.yaml

Write a minimal working version:

model_list:
  - model_name: gpt-4o-mini
    litellm_params:
      model: openai/gpt-4o-mini
      api_key: os.environ/OPENAI_API_KEY

  - model_name: deepseek-chat
    litellm_params:
      model: deepseek/deepseek-chat
      api_key: os.environ/DEEPSEEK_API_KEY

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  database_url: "postgresql://litellm:${POSTGRES_PASSWORD}@db:5432/litellm"
  store_model_in_db: true

litellm_settings:
  request_timeout: 600
  set_verbose: false
  json_logs: true

The easiest thing to confuse here is:

model_name: gpt-4o-mini

This is the name you expose externally.

model: openai/gpt-4o-mini

This is the provider and model that LiteLLM recognizes internally.

In other words, you can expose a unified name externally:

company-fast-model

But behind it, it actually connects to DeepSeek, OpenAI or another provider.

Create docker-compose.yml:

services:
  db:
    image: postgres:16
    container_name: litellm-db
    restart: unless-stopped
    environment:
      POSTGRES_DB: litellm
      POSTGRES_USER: litellm
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
    volumes:
      - ./postgres-data:/var/lib/postgresql/data
    ports:
      - "127.0.0.1:5432:5432"
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U litellm -d litellm"]
      interval: 10s
      timeout: 5s
      retries: 5

  litellm:
    image: ghcr.io/berriai/litellm-database:main-stable
    container_name: litellm
    restart: unless-stopped
    depends_on:
      db:
        condition: service_healthy
    env_file:
      - .env
    volumes:
      - ./config.yaml:/app/config.yaml:ro
    command: ["--config", "/app/config.yaml", "--port", "4000"]
    ports:
      - "127.0.0.1:4000:4000"

Note that I bound both 4000 and 5432 to 127.0.0.1.

Many examples, for convenience, expose services directly to the public internet. Don't do that in production. Put Caddy or Nginx in front of LiteLLM, and definitely don't expose the database directly.

Start:

docker compose up -d

Check status:

docker compose ps
docker logs litellm --tail=100

Local test:

curl http://127.0.0.1:4000/health

If the health check is normal, you can continue to configure HTTPS.

Assume the domain is:

llm.example.com

Add an A record in DNS pointing to your VPS IP.

Install Caddy:

apt install -y debian-keyring debian-archive-keyring apt-transport-https curl
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' > /etc/apt/sources.list.d/caddy-stable.list
apt update
apt install -y caddy

Edit the configuration:

nano /etc/caddy/Caddyfile

Write:

llm.example.com {
    reverse_proxy 127.0.0.1:4000
}

Check and reload:

caddy validate --config /etc/caddy/Caddyfile
systemctl reload caddy

Visit in the browser:

https://llm.example.com/ui

Log in to the admin backend with your LITELLM_MASTER_KEY.

Only open SSH, 80, 443:

ufw allow 22/tcp
ufw allow 80/tcp
ufw allow 443/tcp
ufw enable
ufw status

Don't open 4000 and 5432:

ufw deny 4000/tcp
ufw deny 5432/tcp

If you need to let internal systems call it, it's recommended to use Tailscale, WireGuard or Cloudflare Tunnel, and not leave the admin backend exposed to the public internet.

Request LiteLLM in OpenAI-compatible format:

curl https://llm.example.com/v1/chat/completions \
  -H "Authorization: Bearer sk-your-master-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [
      {"role": "user", "content": "用一句话解释什么是 VPS"}
    ]
  }'

If you want to test DeepSeek:

curl https://llm.example.com/v1/chat/completions \
  -H "Authorization: Bearer sk-your-master-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-chat",
    "messages": [
      {"role": "user", "content": "给我一个 Docker Compose 备份建议"}
    ]
  }'

As long as the application already supports the OpenAI API, in most cases you can just change base_url to your LiteLLM address and it will work.

For example:

OPENAI_BASE_URL=https://llm.example.com/v1
OPENAI_API_KEY=sk-your-virtual-key-or-master-key

What really makes LiteLLM valuable is not proxy forwarding, but virtual Keys.

You can issue different Keys to different projects:

• project-docs-bot
• project-customer-support
• developer-test
• marketing-agent

Set each Key individually:

• Available models
• Requests per minute
• Monthly budget
• Expiry time
• Usage records

This is much safer than handing out the real OpenAI Key to everyone.

In actual use, I would divide them like this:

When something goes wrong, it's also easy to investigate: who exactly used up the money, which project has abnormal request volume, just look at the Key dimension.

If it's just for personal use, you can skip Prometheus for now.

If a team shares it, it's recommended to add it, because sooner or later you'll want to know:

• How much request volume there is
• Which model is used most
• Whether latency is high
• Whether errors occur frequently

LiteLLM's official examples use prometheus.yml, and there's a pitfall here: this file must be a real file.

If you mount it directly in Compose:

- ./prometheus.yml:/etc/prometheus/prometheus.yml

But the file doesn't exist locally, Docker may create it as a directory, and Prometheus won't start.

Create it first:

cat > prometheus.yml << 'EOF'
global:
  scrape_interval: 15s

scrape_configs:
  - job_name: "litellm"
    static_configs:
      - targets: ["litellm:4000"]
EOF

Then extend the Compose. Don't expose the Prometheus dashboard directly to the public internet; at minimum add Basic Auth or allow only internal network access.

Back up at least three things:

/opt/litellm/.env
/opt/litellm/config.yaml
/opt/litellm/postgres-data

.env contains keys, so don't casually put it on GitHub.

config.yaml can go into a private Git repository, but don't hardcode real API Keys in it; try to use:

api_key: os.environ/OPENAI_API_KEY

PostgreSQL backup can be done like this:

cd /opt/litellm
docker exec litellm-db pg_dump -U litellm litellm | gzip > litellm-db-$(date +%F).sql.gz

Then sync the backup to object storage:

rclone copy litellm-db-$(date +%F).sql.gz r2:backup-bucket/litellm/

If you only back up the Compose and not the database, virtual Keys, budget records and usage statistics will all be lost.

Back up before upgrading:

cd /opt/litellm
docker exec litellm-db pg_dump -U litellm litellm | gzip > litellm-before-upgrade-$(date +%F).sql.gz
cp config.yaml config.yaml.bak.$(date +%F)
cp .env .env.bak.$(date +%F)

Then pull the new image:

docker compose pull
docker compose up -d

Check the logs:

docker logs litellm --tail=200

LiteLLM involves database migrations. Don't upgrade during peak hours in production; it's more stable to verify on a test machine first.

These tools often get compared together.

If you're a site operator who wants to issue Keys to users, manage channels and check balances, New-API is more like a ready-made panel.

If you're a development team that wants internal projects to go through a unified OpenAI-compatible entry point, LiteLLM is more convenient.

They are not completely mutually exclusive. You can also have LiteLLM connect to an upstream New-API, or have internal applications all go through LiteLLM, and then let LiteLLM do fallback and budget control.

If I were deploying LiteLLM for a small team myself, I would do it like this:

• VPS: start with 2 cores 4GB
• Deployment: Docker Compose + PostgreSQL
• Entry: Caddy HTTPS
• Port: 4000 listens only on 127.0.0.1
• Database: not exposed to the public internet
• Keys: one virtual Key per project
• Budget: small budget for development Keys, separate limits for production Keys
• Configuration: config.yaml in a private repository, real API Keys in .env
• Backup: back up PostgreSQL, .env and config.yaml daily

What LiteLLM suits best is not "I want to build a chat website", but "I have multiple applications that need to call multiple models, but I don't want each application to manage its own set of API Keys and model adaptations".

As model providers keep increasing, a unified entry point will save more and more effort. Spend half an hour setting up the gateway now, and later when you switch models, add budgets or check bills, you'll save a lot of trouble.