~VRAM-calculator
MODEL7B MODEINFERENCE PRECISION16-BIT FIT24 GB

AI Deployment Calculator

Estimate the GPU VRAM and hardware tier needed to deploy an AI model's workload.

Presets
Model

Deployment

Advanced assumptions

Estimated VRAM Required

19.0 GB

Recommended Example

24 GB hardware tier

e.g. RTX 4090

Usage Capacity

    Why this recommendation

    • Minimum GPU VRAM Capacity
    • Usable VRAM Target
    • Usable VRAM on Recommended Hardware Tier
    • Fit Headroom
    • Estimated Speed
    Values Used In Calculations
      Formula used

      Assumptions used

          Hardware tier

          single accelerator

          ≤ 24 Common consumer / small inference
          32–48 High-end consumer, workstation, larger local inference
          64–96 Large local systems and datacenter accelerators
          141–192 High-memory datacenter / frontier accelerators
          > 192 Does not fit on a single accelerator
          • 160 GB 2x 80 GB GPUs with tensor/model parallelism
          • 320 GB 4x 80 GB GPUs with tensor/model parallelism