B5 - responsible AI

• responsible AI is concerned with where the model fails, why, and what factors influence its predictions

• overall metrics are insufficient, so cohort analysis is needed

• a cohort is a subset of the data selected according to relevant characteristics

• if a model as multiple incorrect predictions, rather than treating all errors as identical, investigate patterns, eg:

FALSE NEGATIVES

72% occur when:
temperature > 35°C
AND
detector generation = C

• this is error analysis which helps identify systematic failure patterns

• feature importance can help understand which inputs contribute strongly to model behaviour

• but a feature being predictive doesn't prove that changing it causes the outcome

• feature importance may be global, so also need local explanations

• Azure ML Responsible AI capabilities bring several analyses together around a model

RESPONSIBLE AI ASSESSMENT
│
├── model performance
├── cohort analysis
├── error analysis
├── explanations / feature importance
└── other responsible-AI analyses

• in human-impacting applications, cohort analysis may also be used to examine whether model behaviour differs across relevant groups

• the exact fairness requirements depend on the application's domain and context

• model lifecycle integration:

TRAIN
  ↓
MLflow tracking
  ↓
candidate
  ↓
standard metrics
  ↓
Responsible AI analysis
  ├── cohort performance
  ├── error analysis
  └── explanations
  ↓
quality gates
  ↓
registration/promotion