F3 - prompting, RAG & fine-tuning

prompting

• a request can contain various pieces that do different jobs

• system instructions establish high-level behaviour

• eg: You are an assistant for neutron detector engineers. Answer using the supplied technical documentation. If the documentation doesn't contain the answer, say that the available documentation is insufficient.

• conversation/user prompt is how the user provides the task

• eg: What is the max operating voltage of detector X?

• retrieved context

• few-shot examples can be helpful to ensure the responses are as needed

• eg: providing some example responses in the required format

• it can help model infer the expected task pattern

• prompting is usually cheap, so before building a fine-tuning dataset, try:

  • clearer instructions
  • explicit constraints
  • output schema
  • examples
  • better context
  • structured output mechanisms where available

RAG

DOCUMENTS
    ↓
chunk
    ↓
embed/index
    ↓
USER QUERY
    ↓
retrieve relevant chunks
    ↓
MODEL CONTEXT
    ↓
response

• RAG changes inference context, not model parameters
• the model can access updated information without being retrained every week
• it can also provide provenance, enabling applications to associate generated answers with retrieved sources
• it does not guarantee correctness
• it has at least two broad failure layers:
  • retrieval, investigate: chunking, embeddings, search strategy, filters, top-k, reranking
  • generation, investigate: instructions, context handling, model behaviour, grounding, model capability

fine-tuning

FOUNDATION MODEL
       +
TRAINING EXAMPLES
       ↓
additional parameter updates
       ↓
FINE-TUNED MODEL

• it changes model parameters
• fine-tuning may help adapt the model to the repeated task pattern
• it is not a database update mechanism
• it also does not eliminate RAG, they are complementay