RunPod Serverless tutorial

Direct answer: RunPod Serverless runs containerized GPU workers on demand. You package a handler, push a Docker image, create an endpoint, and send jobs through the RunPod API. The main tradeoff is simple: you stop paying for idle GPU time, but you accept queueing, cold starts, and container build work.

Before you start

• A RunPod account with credits.
• A Docker image for your model worker, or comfort adapting an example worker.
• A container registry such as Docker Hub, GHCR, or a private registry.
• A model small enough to load within your acceptable cold-start window.

If you do not have a RunPod account yet, create one before the endpoint step. The free credit is useful for one or two small tests, but set a budget and stop endpoints you are not using.

Serverless vs pod decision table

Cost example

Suppose an endpoint handles 1,000 requests per day, each request uses about 4 seconds of GPU time, and the comparable GPU costs $0.69/hr. The rough compute cost is 1,000 x 4 seconds / 3,600 x $0.69, or about $0.77/day before platform-specific overheads and storage. Keeping a $0.69/hr pod on all day is $16.56/day.

That example is why serverless matters. The math flips if your endpoint is busy all day, your model takes too long to load, or you keep warm workers running for latency.

Step 1: write a handler

Your handler receives an event with input data and returns JSON-serializable output. Keep the response small. Store large images or files in object storage and return URLs if your app needs bigger payloads.

import runpod


def handler(event):
    job_input = event.get("input", {})
    prompt = job_input.get("prompt", "a clean product render")

    # Load your model outside the handler in real workers.
    # Return compact JSON from the handler.
    return {"prompt": prompt, "status": "ok"}


runpod.serverless.start({"handler": handler})

Step 2: package the worker

Use a CUDA-ready base image that matches your framework. Pin dependencies. Serverless failures are much easier to debug when the image is boring.

FROM runpod/pytorch:2.1.0-py3.10-cuda11.8.0-devel-ubuntu22.04

WORKDIR /workspace
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY handler.py .
CMD ["python", "-u", "handler.py"]

Step 3: push the image

docker build -t your-registry/runpod-worker:latest .
docker push your-registry/runpod-worker:latest

Step 4: create the endpoint

1. Open RunPod and go to Serverless.
2. Create a new endpoint.
3. Paste your container image URL.
4. Choose a GPU class that fits your model VRAM needs.
5. Set max workers based on expected concurrency.
6. Start with conservative worker counts and raise them after real traffic.

Step 5: call the endpoint

Use the RunPod endpoint URL and API key from the dashboard. Keep keys server-side. Do not expose them in a browser app.

curl -X POST "https://api.runpod.ai/v2/YOUR_ENDPOINT_ID/runsync" \
  -H "Authorization: Bearer YOUR_RUNPOD_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"input": {"prompt": "a blue robot reading logs"}}'

Production checklist

• Load models at module startup, not inside every request handler.
• Log input shape, model load time, execution time, and failure reason.
• Add request validation so a bad prompt or oversized payload does not burn GPU time.
• Use object storage for large outputs.
• Test cold starts with your actual image, not a hello-world worker.
• Set a spend limit and monitor endpoint usage during the first week.

When not to use RunPod Serverless

Do not use serverless for notebook work, live ComfyUI sessions, or anything where a user is watching a spinner and every cold-start second hurts. Use a pod for those. Also avoid serverless until your Docker image is reproducible; debugging a half-working model inside an endpoint is a bad afternoon.

Related guides

• RunPod review
• RunPod vs Lambda Labs
• RunPod vs Vast.ai
• Run ComfyUI on RunPod

FAQs

Is RunPod Serverless cheaper than a pod?

It is cheaper for intermittent inference. If the GPU would sit idle most of the day, serverless usually wins. If requests are constant, a pod may be simpler and cheaper.

What is the difference between RunPod pods and serverless endpoints?

Pods are running GPU machines you control directly. Serverless endpoints are on-demand workers that run your container when jobs arrive.

Can I deploy Stable Diffusion on RunPod Serverless?

Yes, but package it carefully. Model size, image build time, and cold-start behavior matter more than the basic handler code.

Does RunPod Serverless remove cold starts?

No. It reduces idle cost, not physics. You can tune worker settings and image size, but sleeping workers still need time to become ready.