X5 - monitoring the model

keeps ML reliable, relevant and trustworthy once in production

models slowly degrade as real-world data changes

track performance metrics like accuracy, precision, and F1 score

detect data drift in input or target distributions

identify concept drift in feature-label relationships

• relationship between input and predictions change

set retraining triggers if metrics fall below thresholds

incorporate fresh data to improve relevance

schedule periodic retraining to prevent unseen drift

task: app to help doctors diagnose diseases faster using the patient's medical data

• first feature: whether or not to further screen for diabetes
• a reliable, responsible and clinically-useful ML model needs to be trained that detects diabetes
• accuracy interpretability and trust is paramount
• need python-friendly CSV data files to train the model
• also want to extract the privacy-sensitive data from the patient database and store the data in an Azure data storage solution

solution: treat the task as a supervised binary classification problem, using historical patient records labelled with a confirm diabetes outcome - yes/no

• train only on the same inputs the app will collect to avoid data leakage, and ensure the model works in real life

• supervised binary classification was selected as the model is trained on labelled historical patient data and must predict one of two distinct outcomes

• for the ingestion, consider an automated ETL/ELT pipeline using Azure Data Factory/Synapses pipelines that performs scheduled extracts from the patient database and lands curated CSV files in Azure Data Lake Storage Gen 2

• this choice ensures a scalable, repeatable, and secure ingestion path

• for the ingestion solution, focus on understanding the requirements and constraints that drive architectural decisions: data volume, format, scalability, security, transformation needs and downstream consumption

• Data Lake Gen 2 stores large volumes of CSV data at a low-cost, supports a hierarchical name space, folder-ready organisation, granular access control, and better security governance

• this is great when multiple teams and tools need controlled access to shared data

• Azure Synapse Analytics connects to source systems through APIs, orchestrate data movement, and apply transformations as a part of the transformation process

• allows data to be efficiently landed in the data lake in an analytics ready format, and easy consumption by Azure ML or other downstream analytis tools