CD and infrastructure as code

docker

• docker aims to package the runtime environment alongside the application for reproducibility

• a dockerfile is a recipe for describing how to construct an environment

FROM python:3.12

WORKDIR /app  # create \app

COPY requirements.txt .
RUN pip install -r requirements.txt

COPY . .  # copy the appication

CMD ["python", "app.py"]  # when launched, run app.py

• the image is a packaged immutable(ish) template containing what is needed to run the application
• they can be versioned, or associate them with immutable identifiers/digests

docker build ...

# example image
detector-model:v3
├── Linux filesystem
├── Python 3.12
├── numpy
├── scikit-learn
├── application code
├── model-serving code
└── startup configuration

• running the image creates a container

• multiple containers can be instantiated from the same image

• containers provide portable and reproducible runtime environments

• simply providing a model.pkl is not enough, need to know how to execute it

• therefore, containers are important in production ML

# changing code in v17
change source code
  ↓
commit
  ↓
build NEW image
  ↓
detector-model:v18
  ↓
test
  ↓
deploy v18

continuous integration (CI)

• developers integrate changes frequently, and automated checks determine whether those changes are healthy

git push
   ↓
GitHub
   ↓
CI WORKFLOW
   ├── install dependencies
   ├── lint
   ├── unit tests
   ├── integration tests
   ├── security checks
   └── perhaps build container

• CI gives a repeatable quality gate

continuous delivery/deployment (CD)

• automating the process of taking a validated artifact toward deployment

git push
   ↓
CI
   ↓
tests pass
   ↓
build image
   ↓
push image to registry
   ↓
deploy staging
   ↓
integration checks
   ↓
deploy production

• GitHub Actions allows describing automated workflows in YAML

on:
  push:

jobs:
  test:
    steps:
      - checkout-code
      - install-dependencies
      - run-tests

• Actions need to be authenticated and authorised
• secrets should be stored and injected securely, and not hardcoded by committing as code or adding to YAML directly

infrastructure as code (IaC)

• ML system needs infrastructure such as:

  • resource group
  • storage account
  • Azure ML workspace
  • compute cluster
  • container registry
  • key Vault
  • monitoring resources

• they could be manually created, but reproducing may lead to problems

• so infrastructure can be described in code

• declarative infrastructure describes the wanted state rather than manually scripting every mouse click

• if the production environment disappears, IaC allows quick recreation

• infrastructure is versioned, reviewable and repeatable

• Bicep is a declarative language for deploying Azure resources

resource workspace 'Microsoft.MachineLearningServices/workspaces@...' = {
  name: 'my-ml-workspace'
  location: 'uksouth'
}

• Azure CLI, which allows issuing commands to Azure, is different

az something create ...
az something show ...
az something delete ...

GitHub Actions
      ↓
Azure CLI
      ↓
deploy Bicep template
      ↓
Azure resources

project/
│
├── src/
│   └── train.py  # application/ML code
│
├── tests/
│   └── test_model.py
│
├── infra/
│   └── main.bicep  # infrastructure definition
│
├── .github/
│   └── workflows/
│       └── deploy.yml  # automation/workflow
│
├── Dockerfile  # runtime/env definition
└── requirements.txt

# secrets stored elsewhere