A5 - ML compute
- a managed cloud development environment/workstation associated with Azure ML
- simply, the user's personal development machine in Azure
- useful for:
- notebooks
- interactive Python
- exploring data
- debugging
- writing/testing training code
- using development tools
- primarily associated with interactive development, rather than being the ideal scalable production training platform
- a pool of machines that Azure ML can use to execute jobs that is managed scalable compute for Azure ML workloads
-
the cluster can scale according to workload/configuration
-
eg: set
maximum nodes = 0 \ minimum nodes = 4- nothing is happening:
0 nodes - three jobs arrive:
3 nodes - jobs finish:
0 nodes
- nothing is happening:
-
useful for cost control
-
min nodes = 0: cheap while idle but cold-start delay when work arrives -
min nodes = 1: compute already available but ongoing idle cost -
hence there is a cost vs responsiveness trade-off
-
max_nodes = unlimitedis not done as compute costs money and subscriptions have quotas -
eg: 50 jobs arrive with
max_nodes = 4, then 4 execute concurrently remaining jobs wait/queue -
creating a 10-node cluster does not magically split one single training job across 10 machines
-
need a workload capable of distributed execution
-
clusters are also extremely useful for parallel independent jobs, eg: hyperparameter tuning
-
CPU is good for:
- pre-processing
- many classical ML algorithms
- lightweight inference
- data processing
- smaller models
-
GPU is useful for:
- deep learning
- large tensor operations
- some LLM workloads
- GPU-enabled frameworks
-
infrastructure decisions must consider performance, cost, and availability
- Azure provisions/manages compute for a job without you maintaining a dedicated cluster configuration in the same way
- convenient for intermittent or simpler workloads
| Compute Instance | Compute Cluster | Serverless | |
|---|---|---|---|
| Main purpose | Development | Scalable job compute | Managed job compute |
| Interactive | Yes | Not primary purpose | No |
| Named/persistent compute | Yes | Yes | Abstracted |
| Autoscaling | Not cluster-style | Yes | Managed |
| Good for notebooks | Yes | Usually not primary choice | No |
| Good for training jobs | Possible | Yes | Yes |
| Infra management | Moderate | More explicit | Less |
| Typical mental model | Workstation | Elastic worker pool | On-demand managed compute |
- attached compute is when compute resources outside the standard Azure ML-managed compute patters are used