GPU vs CPU

A CPU is like a team of 16 expert engineers. Each one can solve complex problems — branching logic, decision trees, operating system tasks. They work fast and handle anything you throw at them, but there are only 16 of them.

A GPU is like a factory floor with 16,000 workers. Each one can only do simple arithmetic, but they all work at the same time. If your job is "multiply these 16,000 pairs of numbers," the GPU finishes in one step while the CPU needs 1,000 steps.

This is the core trade-off: latency vs throughput. CPUs minimize the time for one task. GPUs maximize the number of tasks done simultaneously.

Neural network training boils down to one operation repeated trillions of times: matrix multiplication. Multiply a matrix of weights by a matrix of inputs, get outputs, compute error, adjust weights, repeat.

Matrix multiplication is embarrassingly parallel — every element in the output matrix can be computed independently. A 1024×1024 matrix multiply involves over 1 billion multiply-add operations, and a GPU can do them all at once across its thousands of cores.

This isn't just about raw speed — it's about economic viability. Many AI models simply cannot be trained on CPUs in any practical timeframe.

GPUs aren't universally better. CPUs excel at:

• Sequential logic — if/else chains, recursive algorithms, tree traversals
• Low-latency single tasks — serving a web request, running a database query
• Operating system tasks — file I/O, process management, networking
• Small workloads — the overhead of sending data to a GPU isn't worth it for small computations
• Branch-heavy code — GPUs stall when different threads need to take different code paths

In a GPU cloud instance, the CPU handles data loading, preprocessing, and orchestration while the GPU handles the heavy math. That's why GIS documents include both gpu and compute sections — you need both.

A GIS document captures both sides:

{
  "gpu": {
    "model": "nvidia-h100",
    "count": 1,
    "vram_gb": 80,
    "tflops_fp16": 989.5
  },
  "compute": {
    "vcpus": 26,
    "ram_gb": 200,
    "storage_gb": 512,
    "network_gbps": 25
  }
}

The gpu section describes the parallel compute engine. The compute section describes the CPU, system memory, storage, and network — the supporting infrastructure that feeds data to the GPU.

The ratio matters: 26 vCPUs for 1 GPU is typical. For 8-GPU instances, you'll see 192+ vCPUs. See CPU-to-GPU Ratio for why.