H2 - advanced fine-tuning

supervised fine-tuning (SFT)

examples are used to teach the model a specialized task pattern

full fine-tuning

update a large portion or potentially all trainable model parameters

it can be expensive for large foundation models

parameter-efficient fine-tuning (PEFT)

adapts a relatively small set of additional/trainable parameters

it is cheaper
Low-Rank Adaptation (LoRA) is a PEFT technique which keeps the large base model intact while small learned low-rank adaptations are made for a task-adapted model

base model
   ↓
training dataset
   ↓
fine-tuning job
   ↓
candidate fine-tuned model
   ↓
evaluation
   ↓
quality/safety gates
   ↓
registration/versioning
   ↓
staging
   ↓
deployment
   ↓
monitoring

lineage must be maintained, so record:
- base model/version
- training dataset/version
- fine-tuning method
- hyperparameters
- code/config
- evaluation results
- resulting model version
fine-tuning a model too aggressively or narrowly can potentially degrade capabilities it previously had, which is called catastrophic forgetting