11 - LLM fundamentals

LLMs take a sequence, $x_{1,} x_{2}, \dots, x_{n}$ and estimate the probabilities for what comes next, $P (x_{n + 1} | x_{1,} x_{2}, \dots, x_{n})$
some decoding procedure selects the next token, it gets appended, and the cycle repeats
this is called autoregressive generation

token

text: "neutron scattering experiment", may be tokenized as: "neutron", " scattering", " experiment"
a word: "unbelievable" $\to$ "un", "believ", "able", so a token is not necessarily a work, but depends on the model's tokenizer
a neural network doesn't directly manipulate the word, but rather works numerically

text
 ↓
tokenizer
 ↓
token IDs
 ↓
numerical representations
 ↓
neural network

token IDs are numerical values that identify tokens in a vocabulary by mapping them into learned vector representations
the "large" in LLM refers to the scale in things such as the number of model parameters, the training data, or the computational requirements
training the model works the same as discussed in 09 - training, where a loss is calculated and the parameters are updated using gradient descent

tokens
  ↓
model
  ↓
prediction
  ↓
compare with target
  ↓
loss
  ↓
backpropagation
  ↓
optimizer
  ↓
update parameters

as training a huge model from scratch is extraordinarily expensive, usually organisations start with a pretrained foundational model which already learned broad capabilities from large-scale training and can then be used/adapted for many downstream applications
model families eg: GPT, Llama, Phi, Gemini, Claude, etc
prompts are instructions/context provided to a trained model
the context window is the text relevant to the current inference call, which may include system instructions, previous conversation, latest message, documents, etc
note that it is finite
always choosing $\arg max P (t o k e n)$ , which is the returned token with the highest probability, makes the generations deterministic
therefore, the final token is often sampled from the probability distribution, allowing a varied response

temperature

the inference/generation configuration that controls the randomness and creativity of the model's output

for low temperatures, the most likely outcome dominates, so it is highly deterministic
high temperatures mean creative, diverse and unpredictable responses
temperature does not make the model smarter but simply changes the sampling behaviour
a model may generate text that is linguistically plausible even though it doesn't actually possess reliable access to that memo, which is called a hallucination
it is not inherently performing a database lookup for every factual claim

fine-tuning