G1 - Azure AI search & RAG architecture

• RAG has two halves
• indexing pipeline that prepares the knowledge

DOCUMENTS
    ↓
extract content
    ↓
clean/process
    ↓
chunk
    ↓
create embeddings
    ↓
INDEX

• query pipeline that answers the users

USER QUERY
    ↓
search/retrieve
    ↓
top relevant chunks
    ↓
add chunks to model context
    ↓
LLM
    ↓
ANSWER

• Azure AI Search can provide the search/indexing layer for RAG

                Azure AI Search

DOCUMENTS ───────► INDEX
	                     ▲
	                     │
USER QUERY ──────────┘
	                     │
	                     ▼
		            search results
	                     │
	                     ▼
	                  Foundry app
	                     │
	                     ▼
	                   model

• an index may contain content along with searchable representation and metadata, eg:

content:
"The detector must not exceed 850 V..."

document:
detector_manual_v7.pdf

section:
4.2

page:
37

detector_type:
D17

last_updated:
2026-08-14

embedding:
[0.032, -0.172, 0.091, ...]

• metadata can help constrain retrieval, making it more precise

• embeddings represent as a vector, $\overrightarrow{x}\in {\mathbb{R}}^d$, where semantically related phrases are nearby

• queries are embedded and then a similarity search is performed

• the embedding space matters, eg: models A and B may have incompatible embeddings

• the query and indexed content need compatible embedding representations

• new documents can simply be indexed, so no need to retrain the LLM

USER QUERY
    ↓
query representation
    ↓
Azure AI Search
    ↓
retrieve:
section 4.2
"must not exceed 850 V"
    ↓
construct model input
 
SYSTEM:
Answer using supplied documentation.

CONTEXT:
D17 manual section 4.2:
"The detector must not exceed 850 V."

USER:
What voltage should D17 never exceed?

LLM
 ↓
"The D17 should not exceed 850 V."

• search usually returns a specified number of the most similar chunks, ie. top-$k$

• $k$ is an evaluation/tuning decision

• search needn't be vector-only

• keyword-style search can be excellent for exact terms, such as error codes, names, identities, etc

• so retrieval can be:

  • keyword
  • vector
  • hybrid

• hybrid search captures both exact lexical matches, and semantic similarities

• semantic rankings and re-rankings can be done to filter retrieved chunks

• eg: initial search can be retrieve 20 candidate chunks, and a more sophisticated ranking stage can reorder/select the most useful ones

• this helps make retrieval cheap and efficient by not requiring efficient but resource-hungry scoring to run on the entire corpus

• so retrieve broadly, rank precisely

• index freshness is also important to receive up-to-date information

• a good retriever should not simply expose everything to every use

• security needs to be considered at retrieval/application level

• retrieval should respect appropriate authorization boundaries rather than handing unauthorized content to the LLM and hoping it doesn't mention it

USER IDENTITY
     ↓
authorization
     ↓
SEARCH / FILTER
     ↓
only permitted chunks
     ↓
LLM

• security should be enforced before sensitive data enters the generation context