X3 - training a model

• this stage involves:
  • loading the data: import and inspecr
  • pre-processing: normalise and clean for consistency
  • splitting: training and test sets
  • choosing the model: select and configure an algorithm
  • training the model: learn patterns from the training data
  • scoring the model: generate predictions
  • evaluating: calculate performance metric

image: AI-300, Microsoft Learn

• not just about picking an algorithm but about building trust in the model

image: AI-300, Microsoft Learn

image: AI-300, Microsoft Learn

pre-processing and configure featurisation

• before running an AutoML experiment, data needs to be prepared
• eg: classification requires tabular data, so a data asset in Azure ML needs to be created; create a MLTable data asset and store the data in a folder together with a MLTable file

from azure.ai.ml.constants import AssetTypes
from azure.ai.ml import Input

my_training_data_input = Input(type=AssetTypes.MLTABLE, path="azureml:input-data-automl:1")

• scaling and normalisation of numeric data is done automatically to prevent any large-scale features from dominating training

• multiple techniques will be applied

• configure optional featurisation:

  • missing value imputation to eliminate nulls in the training dataset
  • categorical encoding to convert categorical features to numeric indicators
  • dropping high-cardinality features, such as record IDs
  • feature engineering (eg: deriving individual date parts from DateTime features)

from azure.ai.ml import automl

classification_job = automl.classification(
	compute="aml-cluster",
	experiment_nmae="auto-ml-class-dev",
	training_data=my_training_data_input,
	target_column_name="Diabetic",
	primary_metric="accuracy",
	n_cross_validations=5,
	enable_model_explanability=True
)

classification_job.set_limits(
	timeout_minutes=60,
	trial_timeout_minutes=20,  # max time one trial can take
	max_trials=5,
	enable_early_termination=True  # end experiment if the score isn't improving
)

# Submitting an AutoML experiment
returned_job = ml_client_jobs.create_or_update(classification_job)

• the experiment consists of child jobs
• featurisation is performed in a child job
• each model is trained in a separate child job