F6 - Foundry development options, PTUs

model deployment can have different underlying infrastructures:

				MODEL
	  ┌──────────┴──────────┐
	  ▼                                                              ▼
SERVERLESS API                     MANAGED COMPUTE
     │                                                               │
 Microsoft-managed                              dedicated GPU
   model serving                                              capacity

serverless API

Microsoft hosts and manages the model-serving infrastructure and exposes an inference API

one does not need to manage: VMs, GPU clusters, serving containers, OS patches
one important Serverless API deployment type is standard/pay-per-token which is attractive for development, testing, low-volume apps and unpredictable workloads
serverless API removes much of infrastructure-management burden

provisioned throughput

deployment reserves model-processing capacity for the application using Provisioned Throughput Units (PTUs)

suited to predictable traffic, latency-sensitive workloads, and sustained production-scale usage
the throughput delivered by a PTU allocation depends on the model and workload characteristics; it isn't universal
PTUs are region-related, and their quotas can be region-specific

estimate workload
      ↓
estimate PTUs
      ↓
benchmark
      ↓
observe utilization/latency
      ↓
adjust

managed compute

managed compute hosts supported open-source, partner and custom models on dedicated GPU capacity, while Microsoft manages the serving runtime and underlying infrastructure

they support autoscaling and scale-to-zero capabilities for appropriate configurations

serving path	billing
serverless standard	tokens
serverless provisioned	PTUs
managed compute	accelerator/GPU time

current Foundry deployment options support private networking