A robust study context document for LLMs, RAG, agents, evaluation, safety, and production AI engineering
Tailored for Shubhojeet Bera - Full-stack / AI Product Engineer track - July 2026
Purpose: Use this PDF as a source map for your own revision and as input context for research agents, mentors, or study partners. The goal is not to become a research scientist overnight; the goal is to speak and build like a serious AI engineer who understands systems, theory, evals, and production tradeoffs.
Best for
AI engineer interviews, RAG/agent system design, project explanations, and deep research planning.
Profle ft
You already know full-stack. This focuses on AI-nativefundamentals around LLM behavior, retrieval, tooling,evals, safety, and infra.
Full_Stack_Engineering_Fundamentals_Master_Map.md
How to read
Do not memorize every line. Use each topic as a launch point: understand the mental model, build a tiny experiment, then write one interview answer.
Depth target
Enough depth to explain tradeofs like a 4-5 year product- minded engineer, while staying honest about your actual experience level.
AI Engineering Fundamentals Master Map - Shubhojeet Bera - July 2026
Study order and operating method
Read in this order. It mirrors how real AI products are built: model behavior first, retrieval second, agents third, production quality last.
Fine-tuning, LoRA, model routing, multimodal RAG, synthetic data, monitoring.
How to use with research agents
For each topic, ask: explain the concept simply, show one production example, show one failure mode, show how to test it, and give me one interview answer using my projects Edward and Agentic Chat as context.
1. AI engineering mindset and role clarity
These topics help you position yourself correctly: not as a pure ML researcher, but as an engineer who can turn LLM capability into reliable product workflows.
AI product engineer vs ML researcher [Must know] - An AI product engineer ships systems around models: prompts, retrieval, tools, evals, UI, infra, safety, and observability. An ML researcher mainly invents or trains model architectures and learning methods.
Study keywords: product AI, LLM applications, systems thinking
Model capability vs product reliability [Must know] - A model can look impressive in a demo but fail in production because of missing evals, poor grounding, bad UX, weak tool safety, or high latency. Your job is to convert raw model capability into repeatable, observable, user-trustworthy behavior.
Study keywords: reliability, evals, traces, user trust
Probabilistic systems thinking [Must know] - LLMs are not deterministic rule engines; they produce likely outputs from distributions. Good AI engineering assumes uncertainty and adds constraints, validation, retrieval, tools, and feedback loops.
Study keywords: uncertainty, validation, sampling
The AI application stack [Must know] - A modern AI feature usually has UI, orchestration, model calls, retrieval, tools, storage, queues, observability, and evaluation. The model is only one part of the stack, not the whole product.
Study keywords: LLM app stack, backend, orchestration
Capability boundaries [Must know] - Know what the model can do from parametric memory and what needs external grounding through docs, APIs, or tools. This separates serious AI engineers from people who just prompt and hope.
Study keywords: parametric memory, external knowledge
Tradeoff language [Must know] - AI engineering interviews reward tradeoff clarity: accuracy vs latency, cost vs quality, recall vs precision, creativity vs determinism, autonomy vs control. Build the habit of explaining every design choice as a tradeoff.
Study keywords: system design, tradeoffs
Demo quality vs production quality [Must know] - A demo proves possibility; production proves durability under messy inputs, concurrent users, failures, and changing data. Always mention logging, retries, evals, guardrails, and failure handling when explaining your work.
Study keywords: production readiness
Your honest positioning [Must know] - For your experience level, say you are strongest in AI-native full-stack systems: RAG, agent workflows, queues, streaming, and product UX. Do not pretend to be a model-training researcher; instead show you understand enough fundamentals to make good engineering decisions.
AI Engineering Fundamentals Master Map - Shubhojeet Bera - July 2026
Study keywords: career positioning
2. Math, statistics, and classic ML basics
You do not need PhD-level math, but you should understand the vocabulary behind model behavior, evaluation metrics, optimization, and similarity search.
Vectors and high-dimensional space [Foundation] - Vectors are lists of numbers used to represent text, images, users, or features. Embeddings place semantically similar items near each other in this space.
Study keywords: vectors, dimensionality, embedding space
Dot product and cosine similarity [Foundation] - Dot product measures directional alignment and magnitude; cosine similarity measures angle between vectors. In retrieval, cosine similarity is commonly used to compare query and document embeddings.
Study keywords: cosine, dot product, similarity
Probability distribution [Must know] - An LLM predicts a probability distribution over the next possible tokens. Decoding settings decide how strictly or creatively to sample from that distribution.
Study keywords: next-token prediction, sampling
Logits and softmax [Must know] - Logits are raw model scores before probabilities. Softmax converts logits into a probability distribution where likely tokens get higher probability.
Study keywords: logits, softmax
Loss function [Foundation] - Loss measures how wrong a model prediction is during training or fine-tuning. Lower loss usually means the model better fits training examples, but not always better real-world behavior.
Study keywords: cross-entropy, optimization
Gradient descent [Foundation] - Gradient descent updates model weights in the direction that reduces loss. You do not need to derive it, but you should know it is the basic engine behind training neural networks.
Study keywords: optimization, backpropagation
Overfitting and generalization [Foundation] - Overfitting means the model memorizes training data or narrow patterns and performs poorly on new cases. Generalization means it handles unseen inputs well.
Study keywords: train/test split, generalization
Precision, recall, and F1 [Must know] - Precision asks how many selected items were correct; recall asks how many correct items were found. F1 balances them and is useful when both false positives and false negatives matter.
Study keywords: classification metrics, retrieval metrics
Confusion matrix [Foundation] - A confusion matrix breaks predictions into true positives, false positives, true negatives, and false negatives. It helps you reason clearly about failure types instead of saying accuracy was bad.
Study keywords: TP, FP, TN, FN
Calibration [Advanced] - A calibrated model expresses confidence that matches reality; if it says 80 percent confidence, it should be right about 80 percent of the time. LLM confidence text is not automatically calibrated, so you need external checks.
Study keywords: confidence, uncertainty
Bias and variance [Foundation] - Bias is systematic error from oversimplified assumptions; variance is instability from overreacting to training data. The idea helps you understand why some models underfit and some overfit.
Study keywords: bias-variance tradeoff
Data leakage [Must know] - Data leakage happens when information from the test set or future data sneaks into training or evaluation. In AI apps, leakage can also happen when eval answers appear in prompts or retrieved context.
Study keywords: evaluation hygiene
Embedding dimensionality [Must know] - Embedding models output vectors with fixed dimensions such as hundreds or thousands of numbers. Higher dimension can represent richer patterns, but storage, speed, and index behavior also matter.
Study keywords: vector size, storage, ANN
Nearest neighbor search [Must know] - Nearest neighbor search finds vectors closest to a query vector. In production, approximate nearest neighbor indexes speed this up for large datasets.
Study keywords: ANN, HNSW, IVF
Statistical significance [Advanced] - One good result from ten examples does not prove quality. For evals, use enough examples and compare changes against a baseline so you do not optimize on noise.
Study keywords: eval dataset, confidence intervals
AI Engineering Fundamentals Master Map - Shubhojeet Bera - July 2026
3. NLP and language fundamentals
These are the concepts behind text processing, retrieval, generation, and evaluation before you even reach modern LLMs.
Corpus [Foundation] - A corpus is a collection of texts used for training, search, or evaluation. In RAG, your corpus could be documents, tickets, code, emails, PDFs, or product knowledge.
Study keywords: dataset, document collection
Normalization [Foundation] - Normalization cleans text by standardizing casing, spacing, punctuation, Unicode, and noisy formatting. It improves retrieval and parsing, especially for PDFs and scraped web pages.
Study keywords: cleaning, preprocessing
Stop words [Foundation] - Stop words are very common words like the, is, and of. Classic search may remove or downweight them, but modern semantic retrieval may preserve them depending on the embedding model.
Study keywords: keyword search, preprocessing
Stemming and lemmatization [Foundation] - Stemming crudely cuts words to roots; lemmatization maps words to dictionary forms. They matter more in classic NLP/search than in modern LLM prompting, but the concepts still help.
Study keywords: text normalization
Named entity recognition [Must know] - NER detects entities like people, companies, locations, products, dates, and amounts. In AI apps, entity extraction helps metadata filters, routing, compliance, and structured outputs.
Study keywords: entities, extraction
Part-of-speech and syntax [Foundation] - POS tagging identifies grammatical roles like noun, verb, and adjective; syntax describes sentence structure. You rarely implement this now, but it explains why language understanding is not only keyword matching.
Study keywords: grammar, parsing
Semantic meaning [Must know] - Semantics is meaning beyond exact words. Embeddings and transformers help models connect phrases like revenue workflow and sales pipeline even when the words differ.
Study keywords: meaning, semantic search
Pragmatics and intent [Advanced] - Pragmatics is meaning in context, including what the user actually wants. Good AI products detect intent, ambiguity, urgency, and task boundaries before responding.
Study keywords: intent detection, context
Information retrieval [Must know] - IR is the field of finding relevant information from a collection. RAG is basically modern IR plus generation, so classic IR concepts still matter a lot.
Study keywords: IR, ranking, search
BM25 [Must know] - BM25 is a strong keyword-based ranking method used in search. It is still useful in RAG because exact terms, IDs, error codes, and product names may beat semantic similarity.
Study keywords: sparse retrieval, keyword search
Dense retrieval [Must know] - Dense retrieval uses embeddings to retrieve semantically similar text. It is powerful for natural language questions but can miss exact constraints if used alone.
Study keywords: vector retrieval, embeddings
Hybrid retrieval [Must know] - Hybrid retrieval combines sparse keyword search and dense vector search. It is often stronger than either alone because it catches both exact terms and semantic meaning.
Study keywords: BM25 plus vectors
Text classification [Foundation] - Classification maps text to labels such as spam, sentiment, priority, or intent. Even in LLM apps, classification is useful for routing, moderation, and workflow decisions.
Study keywords: intent, labels
Sequence labeling [Foundation] - Sequence labeling assigns labels to parts of text, such as entities or slots. It is relevant when extracting structured fields from messy documents or conversations.
Study keywords: slots, extraction
Summarization [Must know] - Summarization compresses text while preserving key information. In RAG and agents, summarization is used for memory, context compression, thread summaries, and long-running workflows.
Study keywords: compression, memory
Question answering [Must know] - QA systems answer questions from knowledge sources or model memory. RAG-style QA should ground claims in retrieved context instead of relying only on the model.
Study keywords: open-domain QA, grounded QA
AI Engineering Fundamentals Master Map - Shubhojeet Bera - July 2026
4. Tokenization, context, and LLM input mechanics
This section explains how raw text becomes model-readable input and why token budgets, context placement, and formatting affect quality and cost.
Tokenization [Must know] - Tokenization splits text into model-readable pieces such as words, subwords, punctuation, or bytes. Token count controls cost, latency, and how much context you can send.
Study keywords: BPE, tokens, token IDs
Token IDs [Must know] - After tokenization, each token becomes an integer ID from the model vocabulary. The model does not directly see words; it sees these IDs converted into embeddings.
Study keywords: vocabulary, IDs
Subword tokens [Must know] - Most LLMs use subword tokenization so rare words can be represented as smaller pieces. This is why names, code, Hindi, and unusual strings may consume more tokens than expected.
Study keywords: BPE, multilingual text
Context window [Must know] - The context window is the maximum input plus output tokens the model can handle in one call. A larger window helps with long documents but does not remove the need for relevance filtering.
Study keywords: context length, token budget
Prompt sections [Must know] - A good prompt separates system rules, developer instructions, user input, retrieved context, examples, and output format. Clear separation reduces instruction confusion and prompt injection risk. Study keywords: prompt structure, delimiters
Context placement [Advanced] - Models may pay different attention to different parts of a long context, and important details can get lost in the middle. Put critical instructions and final task constraints in stable, easy-to-find places.
Study keywords: lost in the middle, prompt layout
Token budget management [Must know] - Every request has a budget split across instructions, context, chat history, tool results, and output. Production systems should trim, summarize, retrieve, and compress instead of blindly appending everything.
Study keywords: budgeting, trimming
Chat history handling [Must know] - Chat history can help continuity but can also bring irrelevant or stale context. Serious systems summarize or select history instead of sending the full conversation forever.
Study keywords: conversation memory
System prompt [Must know] - The system prompt defines high-priority behavior and boundaries. It should be stable, concise, tested, and separated from user-controlled content.
Study keywords: instructions, behavior
Developer/task prompt [Must know] - A task prompt describes the specific workflow, output contract, examples, and failure behavior. Treat it like an API contract, not like a casual paragraph.
Study keywords: prompt contract
Retrieved context block [Must know] - Retrieved context should be clearly marked as evidence, not as instructions. This helps the model answer from documents while ignoring malicious text inside documents.
Study keywords: grounding, prompt injection
Output token limit [Foundation] - The output token limit caps the response size. Too low can truncate structured outputs; too high can increase cost and allow rambling.
Study keywords: max tokens
Latency from token count [Must know] - More input and output tokens usually increase latency. For product UX, reduce irrelevant context, stream outputs, cache stable prefixes, and avoid unnecessary model calls.
Study keywords: latency, cost
5. Transformer and LLM internals
You do not need to derive every equation, but you should be able to explain how tokens flow through a transformer and why attention matters.
Transformer architecture [Must know] - A transformer is a neural network architecture based on attention instead of recurrence. It became the foundation of modern LLMs because it scales well and processes context efficiently.
Study keywords: attention, decoder, layers
Embedding layer [Must know] - The embedding layer maps token IDs into dense vectors. These vectors are the model internal representation before attention layers contextualize them.
Study keywords: token embeddings
AI Engineering Fundamentals Master Map - Shubhojeet Bera - July 2026
Self-attention [Must know] - Self-attention lets each token look at other tokens in the same sequence and decide what information matters. This is how the same word can mean different things in different contexts.
Study keywords: attention weights, context
Query, key, and value [Must know] - In attention, a query asks what a token is looking for, a key describes what another token offers, and a value contains the information to pass along. Their interactions decide how tokens influence each other.
Study keywords: QKV
Multi-head attention [Must know] - Multi-head attention runs multiple attention patterns in parallel. Different heads can learn different relationships such as syntax, entity links, position, or long-range dependency.
Study keywords: heads, parallel attention
Positional encoding [Must know] - Because transformers process tokens in parallel, they need a way to represent order. Positional encodings or rotary embeddings inject sequence position into token representations. Study keywords: RoPE, order
Feed-forward network [Foundation] - After attention, each token representation passes through a feed-forward network. This adds non-linear transformation and helps the model store learned patterns.
Study keywords: MLP, transformer block
Layer normalization [Foundation] - Layer normalization stabilizes activations inside deep networks. It helps training and inference remain numerically stable across many transformer layers.
Study keywords: normalization, stability
Residual connections [Foundation] - Residual connections let layers add improvements while preserving previous information. They make very deep networks easier to train.
Study keywords: skip connections
Decoder-only models [Must know] - GPT-style LLMs are usually decoder-only transformers trained to predict the next token. They generate by repeatedly predicting one token after another.
Study keywords: autoregressive generation
Encoder-only models [Foundation] - Encoder-only models like BERT are strong for understanding, classification, and embedding-style tasks. They are not usually used for open-ended text generation.
Study keywords: BERT, understanding
Encoder-decoder models [Foundation] - Encoder-decoder models read an input with an encoder and generate output with a decoder. They were common in translation and summarization before decoder-only LLMs dominated chat apps. Study keywords: seq2seq, T5
Pretraining [Must know] - Pretraining teaches a model broad language and world patterns from massive data. It creates general capability but does not automatically make the model safe or instruction-following.
Study keywords: foundation models
Instruction tuning [Must know] - Instruction tuning trains a model to follow user instructions and produce helpful outputs. It makes a raw language model more usable as an assistant.
Study keywords: SFT, instructions
RLHF and preference tuning [Advanced] - Preference tuning aligns outputs with human preferences using feedback signals. You should know it improves helpfulness and style but does not guarantee factual correctness.
Study keywords: RLHF, DPO, alignment
Attention complexity [Advanced] - Standard attention can become expensive as context length grows because every token attends to many other tokens. This is why long-context efficiency and retrieval remain important.
Study keywords: quadratic cost, long context
Model size and capability [Must know] - More parameters can improve capability, but data quality, training method, architecture, tools, and inference strategy also matter. Bigger is not automatically better for every product task.
Study keywords: parameters, scale
6. Generation behavior and decoding controls
These settings shape how an LLM chooses output tokens. They are production levers for reliability, creativity, cost, and consistency.
Next-token prediction [Must know] - LLMs generate by predicting the next token repeatedly. This simple mechanism can produce complex behavior, but it also explains why outputs can vary or drift.
Study keywords: autoregressive, prediction
Temperature [Must know] - Temperature controls randomness in token selection. Use low temperature for extraction, coding, QA, and RAG; use higher temperature for brainstorming or creative variation.
AI Engineering Fundamentals Master Map - Shubhojeet Bera - July 2026
Study keywords: determinism, randomness
Top-k sampling [Foundation] - Top-k restricts generation to the k most likely next tokens. It reduces weird low-probability choices but can be too rigid if k is poorly chosen.
Study keywords: sampling filter
Top-p sampling [Must know] - Top-p chooses from the smallest token set whose cumulative probability crosses p. It is adaptive because the candidate set expands or shrinks based on model confidence.
Study keywords: nucleus sampling
Greedy decoding [Foundation] - Greedy decoding always picks the most likely token. It is deterministic but can produce dull, repetitive, or locally optimal outputs.
Study keywords: argmax decoding
Beam search [Advanced] - Beam search explores multiple likely sequences and keeps the best scoring ones. It is common in classic seq2seq tasks but less central for modern chat-style generation.
Study keywords: sequence decoding
Frequency penalty [Foundation] - Frequency penalty discourages repeating tokens already used. It can reduce repetition but may harm precise tasks where repeated terms are necessary.
Study keywords: anti-repetition
Presence penalty [Foundation] - Presence penalty encourages the model to introduce new topics or terms. It is more useful for creative generation than grounded factual answers.
Study keywords: diversity
Seed and determinism [Advanced] - Some APIs expose seed-like controls to make outputs more reproducible. Even then, provider updates and distributed systems can limit perfect repeatability.
Study keywords: reproducibility
Structured outputs [Must know] - Structured outputs force or validate output against a schema such as JSON. They are critical when LLM output feeds code, databases, tools, or UI components.
Study keywords: JSON schema, parsers
Tool/function calling [Must know] - Tool calling lets the model request a typed function call instead of only writing text. Production systems must validate arguments, execute safely, and feed results back clearly.
Study keywords: tools, schemas
Streaming [Must know] - Streaming sends partial output as it is generated. It improves perceived latency and is also useful for progress logs in long-running agents.
Study keywords: SSE, WebSockets
Stop sequences [Foundation] - Stop sequences tell the model where generation should end. They are useful for parsers, multi-part prompts, and preventing extra text after structured output. Study keywords: output control
Model routing [Advanced] - Model routing chooses different models for different tasks based on quality, cost, latency, or safety. A strong AI engineer does not use the biggest model for every call by default.
Study keywords: router, cost optimization
7. Prompt engineering and context design
Prompting is interface design for a probabilistic model. Good prompts define task, evidence, constraints, examples, schema, and failure behavior.
Instruction clarity [Must know] - The model should know exactly what task to perform, what inputs matter, and what output shape is expected. Vague prompts create vague behavior.
Study keywords: task definition
Role prompting [Foundation] - Role prompting frames the model as a specific expert or worker. It helps style and focus, but concrete instructions and examples matter more than fancy role text.
Study keywords: role, persona
Few-shot examples [Must know] - Few-shot prompting shows input-output examples so the model can imitate the desired pattern. It is useful for classification, extraction, formatting, and edge-case behavior.
Study keywords: examples, demonstrations
Zero-shot prompting [Foundation] - Zero-shot prompting gives only instructions without examples. It is fast to write, but less reliable for specialized or strict-output tasks.
Study keywords: instruction only
AI Engineering Fundamentals Master Map - Shubhojeet Bera - July 2026
Prompt templates [Must know] - Prompt templates are reusable prompt structures with variables. They make prompts testable, versionable, and consistent across requests.
Study keywords: template variables
Context separation [Must know] - Keep instructions, user text, documents, tool outputs, and examples separated with labels or delimiters. This reduces confusion and makes prompt injection defenses easier.
Study keywords: delimiters, sections
Grounded answering [Must know] - Grounded prompts require the model to answer only from provided context or tools. They reduce hallucination when combined with good retrieval and refusal behavior.
Study keywords: evidence, citations
Refusal behavior [Must know] - A robust prompt tells the model what to do when evidence is missing, ambiguous, or conflicting. This is more production-safe than forcing an answer every time.
Study keywords: uncertainty, not enough info
Decomposition [Must know] - Decomposition breaks a complex task into smaller steps such as retrieve, extract, validate, then answer. It improves reliability and makes failures easier to debug.
Study keywords: task breakdown
Prompt chaining [Advanced] - Prompt chaining uses multiple model calls where each call has a narrow responsibility. It can improve quality but increases latency, cost, and orchestration complexity.
Study keywords: chains, pipelines
Hidden reasoning vs final answer [Advanced] - For many products, you do not need to expose reasoning traces to users. Instead, ask for concise rationales, evidence, checks, or structured intermediate artifacts.
Study keywords: reasoning control, UX
Prompt injection awareness [Must know] - Any user text, webpage, PDF, or retrieved document can contain malicious instructions. Treat external content as data, not authority.
Study keywords: prompt injection, untrusted input
Prompt versioning [Advanced] - Prompts should be versioned like code because small changes can alter behavior. Keep evals tied to prompt versions so you can detect regressions.
Study keywords: prompt registry, evals
Output contracts [Must know] - For production, define exact output contracts with schemas, enums, required fields, and validators. A beautiful natural language answer is useless if downstream code cannot trust its structure.
Study keywords: schema, validation
System prompt minimalism [Advanced] - A huge system prompt can become expensive, brittle, and hard to test. Keep stable rules concise and move task-specific detail into smaller, evaluated components.
Study keywords: maintainability
Prompt caching [Advanced] - Prompt caching reuses stable prompt prefixes to reduce latency and cost in supported systems. It is useful for long system prompts, tool lists, or repeated agent contexts. Study keywords: cache keys, cost
8. Embeddings, vector databases, and retrieval
This is the technical core behind semantic search and RAG. You should understand embedding quality, indexing, metadata, filters, and ranking tradeoffs.
Embeddings [Must know] - Embeddings convert text or other content into dense vectors that capture semantic meaning. They make search possible even when the query and document use different wording.
Study keywords: semantic vectors
Embedding model choice [Must know] - Different embedding models vary by language support, dimensionality, domain fit, cost, speed, and retrieval quality. Always evaluate on your own corpus instead of choosing by hype.
Study keywords: model selection
Query embedding [Must know] - A user query is embedded into the same vector space as document chunks. Retrieval finds chunks whose embeddings are close to the query embedding.
Study keywords: query vector
Document embedding [Must know] - Each chunk or document is embedded and stored with metadata. If chunking is poor, even a strong embedding model will retrieve weak context.
Study keywords: chunk vectors
Vector database [Must know] - A vector database stores embeddings and supports similarity search at scale. Examples include pgvector, Qdrant, Pinecone, Weaviate, Milvus, and Elasticsearch vector search.
AI Engineering Fundamentals Master Map - Shubhojeet Bera - July 2026
Study keywords: pgvector, ANN
Metadata filters [Must know] - Metadata filters restrict retrieval by tenant, date, source, document type, permissions, project, or language. They are essential for correctness, privacy, and relevance.
Study keywords: filters, access control
Approximate nearest neighbor [Advanced] - ANN search trades tiny accuracy loss for major speed improvements on large vector collections. Common index families include HNSW and IVF-style approaches.
Study keywords: HNSW, IVF
Embedding normalization [Advanced] - Normalization scales vectors so similarity is based on direction rather than magnitude. Some models and databases expect normalized vectors for cosine-like comparisons.
Study keywords: L2 norm, cosine
Sparse retrieval [Must know] - Sparse retrieval uses lexical signals like exact words and term frequency. It is strong for names, IDs, error codes, logs, and rare technical terms.
Study keywords: BM25, keyword search
Dense retrieval [Must know] - Dense retrieval uses semantic vectors and is strong for natural language similarity. It can fail on exact terms, negation, numbers, and domain-specific vocabulary without tuning.
Study keywords: semantic search
Hybrid retrieval [Must know] - Hybrid retrieval combines dense and sparse signals. This is often the best default for production RAG because it handles both meaning and exact matching.
Study keywords: BM25 plus embeddings
Reranking [Must know] - Reranking takes initially retrieved candidates and uses a stronger model to reorder them. It improves precision by judging query-document relevance more deeply than vector distance alone.
Study keywords: cross-encoder, reranker
Bi-encoder vs cross-encoder [Must know] - A bi-encoder embeds query and documents separately for fast search. A crossencoder reads query and document together for slower but more accurate reranking.
Study keywords: retrieval architecture
MMR [Advanced] - Maximal marginal relevance balances relevance with diversity. It helps avoid retrieving many nearduplicate chunks when the answer needs broader coverage.
Study keywords: diversity, redundancy
Deduplication [Must know] - Duplicate chunks waste context budget and can bias generation. Deduping at ingestion and retrieval time improves context quality.
Study keywords: duplicates, context budget
Freshness handling [Must know] - Retrieval should prefer current sources when the domain changes over time. Use metadata, timestamps, source priority, and stale-document policies.
Study keywords: recency, stale data
Access-controlled retrieval [Must know] - The retriever must enforce permissions before context reaches the model. Never rely on the model to ignore documents the user should not see.
Study keywords: authorization, tenant isolation
9. RAG pipeline fundamentals
RAG is not just vector search. It is a complete pipeline from ingestion to answer evaluation, with many places where quality can fail.
RAG definition [Must know] - Retrieval-augmented generation combines external retrieval with LLM generation. It helps models answer from current or private knowledge instead of relying only on training data.
Study keywords: retrieval plus generation
Ingestion [Must know] - Ingestion brings data into the system from PDFs, docs, databases, web pages, emails, tickets, code, or APIs. Ingestion quality controls everything downstream.
Study keywords: loaders, connectors
Parsing [Must know] - Parsing extracts usable text, tables, headings, metadata, and structure from source files. Bad PDF parsing is one of the most common reasons RAG fails.
Study keywords: PDF parsing, OCR
Cleaning [Must know] - Cleaning removes boilerplate, repeated headers, broken OCR, navigation, tracking text, and irrelevant content. Cleaner data means fewer hallucinations and better retrieval.
Study keywords: preprocessing
AI Engineering Fundamentals Master Map - Shubhojeet Bera - July 2026
Chunking [Must know] - Chunking splits documents into retrievable pieces. The chunk should be small enough to retrieve precisely and large enough to preserve meaning.
Study keywords: document splitting
Chunk overlap [Must know] - Overlap repeats some text between neighboring chunks to avoid cutting context at boundaries. Too much overlap wastes storage and context; too little can lose meaning.
Study keywords: sliding window
Structure-aware chunking [Must know] - Structure-aware chunking follows headings, sections, Markdown, HTML, pages, or document hierarchy. It is usually better than naive fixed-size chunking for serious docs.
Study keywords: headings, hierarchy
Semantic chunking [Advanced] - Semantic chunking splits where topic meaning changes. It can improve retrieval but is more complex and needs evaluation.
Study keywords: topic shifts
Parent-child retrieval [Advanced] - Parent-child retrieval indexes small child chunks but returns a larger parent section for context. It balances precise search with complete answer context.
Study keywords: small-to-large retrieval
Indexing [Must know] - Indexing stores vectors, text, metadata, and source references so retrieval can happen quickly. Good indexing also supports deletion, updates, versioning, and permissions.
Study keywords: vector store, metadata
Query rewriting [Must know] - Query rewriting turns a vague or conversational question into a clearer search query. It is especially useful in multi-turn chat where pronouns and context matter.
Study keywords: contextualization
Query expansion [Advanced] - Query expansion adds synonyms, acronyms, product names, or related terms. It can improve recall but can also add noise if uncontrolled.
Study keywords: recall improvement
Retrieval top-k [Must know] - Top-k controls how many candidates are fetched. Higher k improves recall but increases noise, reranking cost, and context pressure.
Study keywords: recall vs noise
Context construction [Must know] - Context construction decides which retrieved chunks enter the final prompt and in what order. This step has a major effect on answer quality and hallucination risk.
Study keywords: packing, ordering
Context compression [Advanced] - Context compression summarizes or filters retrieved chunks before generation. It saves tokens but can remove details if the compressor is weak.
Study keywords: summarization, filtering
Citation grounding [Must know] - Citation grounding links answer claims to source chunks or documents. It improves user trust and gives evaluators something concrete to verify.
Study keywords: sources, evidence
Conflict handling [Advanced] - Documents may disagree because of versioning, policy changes, or source quality. A robust RAG system detects conflicts and reports uncertainty instead of blending them silently.
Study keywords: source priority, contradictions
No-answer handling [Must know] - Sometimes the correct answer is not in the corpus. A strong system says it lacks enough evidence instead of hallucinating from general knowledge.
Study keywords: abstention, refusal
Feedback loop [Must know] - Production RAG should log bad answers, missing chunks, user corrections, and retrieval traces. Those signals drive improvements to chunking, prompts, and eval sets.
Study keywords: iteration, logs
RAG observability [Must know] - You need to inspect query, rewritten query, retrieved chunks, scores, reranked order, final context, answer, citations, cost, and latency. Without traces, you are debugging blind.
Study keywords: tracing, LangSmith
10. RAG patterns and types
Know these patterns so you can choose the right architecture for a task instead of saying every problem needs plain vector search.
Naive RAG [Must know] - Naive RAG embeds chunks, retrieves top-k, and asks the model to answer. It is the simplest baseline and should be improved only after you measure where it fails.
AI Engineering Fundamentals Master Map - Shubhojeet Bera - July 2026
Study keywords: baseline RAG
Advanced RAG [Must know] - Advanced RAG adds better parsing, metadata filters, query rewriting, hybrid search, reranking, and eval loops. Most serious production systems eventually move here.
Study keywords: production RAG
Modular RAG [Advanced] - Modular RAG treats ingestion, retrieval, reranking, compression, generation, and eval as replaceable modules. This makes experimentation and regression testing easier.
Study keywords: components
Hybrid RAG [Must know] - Hybrid RAG combines keyword and vector retrieval. It is useful when documents contain exact technical terms, IDs, names, logs, or legal language.
Study keywords: sparse+dense
Reranked RAG [Must know] - Reranked RAG retrieves many candidates and reranks them before context construction. It improves precision when initial vector retrieval returns noisy results.
Study keywords: candidate reranking
Conversational RAG [Must know] - Conversational RAG rewrites the current user question using chat history before retrieval. It prevents follow-up questions like what about pricing from becoming ambiguous.
Study keywords: chat history
Agentic RAG [Must know] - Agentic RAG lets an agent decide when to search, what to search, and whether more retrieval is needed. It is useful for complex tasks but requires strict observability and guardrails.
Study keywords: agents, tools
Corrective RAG [Advanced] - Corrective RAG detects weak retrieval and retries, broadens search, or falls back to web/tools. It is designed for cases where initial retrieval may be incomplete.
Study keywords: retrieval validation
Self-RAG [Advanced] - Self-RAG-style approaches let the model judge when retrieval is needed and whether evidence is sufficient. Treat this as a pattern, not magic; still evaluate it. Study keywords: reflection, evidence sufficiency
Graph RAG [Advanced] - Graph RAG extracts entities and relationships into a graph to answer questions requiring connected reasoning. It is useful for enterprise knowledge, investigations, and relationship-heavy corpora.
Study keywords: knowledge graph
Multi-vector RAG [Advanced] - Multi-vector RAG stores multiple embeddings per document, such as summaries, sections, tables, and raw chunks. It improves recall for complex documents.
Study keywords: multi-representation
Parent-document RAG [Advanced] - Parent-document RAG retrieves small indexed chunks but feeds larger parent documents or sections to the LLM. It is strong when small chunks lack enough surrounding context.
Study keywords: context expansion
Multimodal RAG [Advanced] - Multimodal RAG retrieves text, images, charts, tables, audio, or video frames. It is needed when important evidence is not plain text.
Study keywords: vision, tables
Long-context RAG [Advanced] - Long-context RAG uses large context windows but still retrieves and ranks relevant content. Large context reduces some constraints but can increase noise, latency, and cost.
Study keywords: large context
Cache-augmented RAG [Advanced] - Cache-augmented RAG stores repeated query results, embeddings, summaries, or final answers. It reduces cost and latency for repeated or similar requests.
Study keywords: semantic cache
Federated retrieval [Advanced] - Federated retrieval searches across multiple systems such as docs, Slack, email, GitHub, databases, and web. The hard part is permissions, ranking, and result normalization.
Study keywords: connectors, permissions
11. Evaluation and measurement
Evaluation is the difference between a demo and an engineering system. You need offline evals, online metrics, human review, and failure analysis.
Eval dataset [Must know] - An eval dataset is a set of representative test cases with expected behavior. Build it from real user questions, edge cases, failures, and domain-specific examples.
Study keywords: test set, golden set
AI Engineering Fundamentals Master Map - Shubhojeet Bera - July 2026
Golden answers [Foundation] - Golden answers are reference answers used to judge generated responses. They help but can be brittle when multiple valid answers exist.
Study keywords: reference answers
Retrieval recall at k [Must know] - Recall@k asks whether relevant evidence appears in the top k retrieved results. If recall is bad, generation cannot be reliably grounded.
Study keywords: retrieval eval
Retrieval precision at k [Must know] - Precision@k asks how much of the retrieved context is actually useful. Low precision wastes tokens and increases hallucination risk.
Study keywords: noise, relevance
MRR [Advanced] - Mean reciprocal rank rewards systems that put the first relevant result high in the list. It is useful when the top result matters a lot.
Study keywords: ranking metric
nDCG [Advanced] - nDCG measures ranking quality when results have graded relevance levels. It is useful when some chunks are partially relevant and others are highly relevant.
Study keywords: graded relevance
Faithfulness [Must know] - Faithfulness checks whether the answer is supported by the retrieved context. A faithful answer should not add unsupported claims just because they sound plausible.
Study keywords: groundedness
Answer relevance [Must know] - Answer relevance checks whether the response actually answers the user question. A grounded answer can still be irrelevant if it uses the wrong context or misses the intent.
Study keywords: response quality
Context precision [Must know] - Context precision measures whether retrieved contexts are useful for answering the question. It helps diagnose noisy retrieval.
Study keywords: RAGAS, retrieval quality
Context recall [Must know] - Context recall measures whether the retrieved contexts include the information needed for the answer. It helps diagnose missing evidence.
Study keywords: RAGAS, coverage
Citation accuracy [Must know] - Citation accuracy checks whether cited sources actually support the claims attached to them. Bad citations are dangerous because they create false trust. Study keywords: source verification
LLM-as-judge [Must know] - LLM-as-judge uses another model to score outputs for criteria like helpfulness, correctness, or grounding. It scales evaluation but must be calibrated with human checks.
Study keywords: automated eval
Human evaluation [Must know] - Human evaluation is still needed for nuanced correctness, UX, tone, and domain-specific judgment. Use it to calibrate automated evals, not as the only quality process.
Study keywords: review, labeling
Regression testing [Must know] - Regression testing reruns evals whenever prompts, models, retrieval, or data change. This prevents silent quality drops after seemingly small updates.
Study keywords: CI for AI
A/B testing [Advanced] - A/B testing compares AI system variants with real users. Use it carefully because user satisfaction, correctness, latency, and cost may move differently.
Study keywords: online experiments
Latency metrics [Must know] - Track p50, p95, and p99 latency, not just average latency. Users feel tail latency strongly, especially in agent workflows.
Study keywords: p95, p99
Cost metrics [Must know] - Track cost per request, per successful task, per user, and per workflow. A powerful model that ruins unit economics is not production-friendly.
Study keywords: token cost
Tool success rate [Must know] - For agents, measure whether tool calls succeed, use correct arguments, and produce useful results. Tool failure is often the real agent failure.
Study keywords: tool eval
Task completion rate [Must know] - Task completion rate measures whether the AI workflow achieved the user goal. It is often more meaningful than whether the final text looked nice.
Study keywords: agent outcome
AI Engineering Fundamentals Master Map - Shubhojeet Bera - July 2026
Failure taxonomy [Must know] - Categorize failures such as retrieval miss, bad rerank, hallucination, schema error, tool error, timeout, permission issue, or bad UX. This makes improvement systematic. Study keywords: root cause analysis
12. Hallucination, grounding, and reliability
Hallucination is not one bug. It is a failure mode caused by model behavior, bad context, weak prompts, missing tools, or absent evaluation.
Hallucination [Must know] - Hallucination is when the model states unsupported or false information confidently. In production, you reduce it with grounding, retrieval, validation, and evals, not vibes.
Study keywords: unsupported claims
Parametric knowledge limits [Must know] - A model stores learned patterns in weights but may be outdated or wrong. Use tools and retrieval for private, recent, or high-stakes knowledge.
Study keywords: model memory
Retrieval miss [Must know] - A retrieval miss happens when the needed evidence exists but is not retrieved. Fix it with better chunking, hybrid search, query rewriting, metadata, or reranking.
Study keywords: RAG debugging
No-evidence hallucination [Must know] - The model may answer even when no evidence exists. Add refusal rules, evidence checks, and no-answer eval cases.
Study keywords: abstention
Context conflict [Advanced] - Retrieved documents may contain contradictory claims. A reliable answer should surface the conflict, prefer authoritative sources, or ask for clarification.
Study keywords: contradictions
Unsupported citation [Must know] - An unsupported citation points to a source that does not actually prove the claim. This is worse than no citation because it creates fake credibility.
Study keywords: citation eval
Grounding checks [Must know] - Grounding checks verify that each important answer claim is supported by retrieved context or tool output. This can be human, rule-based, or model-assisted.
Study keywords: claim verification
Claim decomposition [Advanced] - Claim decomposition breaks an answer into atomic factual claims before verification. It makes hallucination detection more precise than scoring a whole paragraph vaguely.
Study keywords: atomic claims
Uncertainty expression [Must know] - A reliable AI system can say maybe, unknown, not in sources, or conflicting evidence. Honest uncertainty is better than confident nonsense.
Study keywords: calibrated response
Temperature control [Must know] - Lower temperature reduces creative variation but does not guarantee truth. It should be combined with retrieval, tools, and validation.
Study keywords: decoding reliability
External verification [Must know] - For high-stakes or changing facts, verify through authoritative tools or sources. Do not rely only on the model memory.
Study keywords: tools, web, APIs
Guardrail limits [Advanced] - Guardrails reduce failure probability but do not make systems perfect. Treat them as layers: input filtering, prompt design, tool constraints, output validation, and monitoring.
Study keywords: defense in depth
Model disagreement [Advanced] - Comparing outputs from multiple models or judges can expose uncertainty. But agreement is not proof; models can share the same blind spots.
Study keywords: ensemble, critique
User feedback loop [Must know] - User corrections reveal real failure cases that static evals may miss. Feed them into eval datasets and failure taxonomies instead of treating them as one-off bugs.
Study keywords: feedback, iteration
13. Agents, tools, and orchestration
Agents are not magic autonomous beings. They are loops that decide actions, call tools, update state, and continue until a goal or stop condition is met.
AI Engineering Fundamentals Master Map - Shubhojeet Bera - July 2026
Agent loop [Must know] - An agent loop usually observes context, reasons/plans, chooses an action, calls a tool, observes the result, and repeats. Reliability depends on state, tools, limits, and evals.
Study keywords: observe-think-act
Tool calling [Must know] - Tool calling gives the model access to typed functions such as search, email, calendar, database, code execution, or browser actions. The system must validate arguments and enforce permissions.
Study keywords: functions, schemas
Tool schema design [Must know] - A tool schema should be narrow, explicit, typed, and safe. Bad schemas make the model guess arguments or perform unsafe actions.
Study keywords: JSON schema, API contract
Tool result formatting [Must know] - Tool results should be concise, structured, and clearly labeled as observations. Messy tool outputs increase token use and make agents reason poorly.
Study keywords: observations, result schema
Planning [Must know] - Planning breaks a goal into steps before execution. It helps complex workflows but can fail if the plan is stale, overcomplicated, or not updated after tool results.
Study keywords: task planning
ReAct pattern [Must know] - ReAct interleaves reasoning and acting so the model can use external tools while updating its plan. It is a foundational mental model for tool-using agents.
Study keywords: reasoning plus acting
Human-in-the-loop [Must know] - HITL pauses the agent for approval, correction, or confirmation before risky actions. It is essential for emails, payments, destructive changes, production deploys, or user-sensitive data.
Study keywords: approval gates
Agent state [Must know] - Agent state stores task progress, tool outputs, decisions, memory, and pending actions. Without explicit state, long-running agents become fragile and hard to resume.
Study keywords: state machine
Memory [Must know] - Memory can mean chat history, summaries, vector memory, user preferences, or durable task state. Separate these types so private preferences do not pollute factual retrieval. Study keywords: short-term, long-term
Checkpoints [Must know] - Checkpoints persist execution state so a workflow can resume after interruption or failure. This is critical for long-running agents and crash-safe orchestration.
Study keywords: durable execution
Stop conditions [Must know] - Agents need limits on steps, time, cost, retries, and tool scope. Without stop conditions, agents can loop, overspend, or keep acting after the goal is complete.
Study keywords: loop control
Idempotency [Advanced] - Idempotent operations can safely retry without duplicate side effects. This matters when agents send emails, create tickets, charge users, or modify data.
Study keywords: safe retries
LangChain [Must know] - LangChain provides abstractions for models, prompts, tools, retrievers, parsers, and chains. Use it for composable LLM app components, but understand what happens underneath.
Study keywords: runnables, integrations
LCEL / runnables [Advanced] - LCEL-style runnables compose steps like prompt, model, parser, retriever, and lambda into pipelines. This helps build testable chains instead of unstructured glue code.
Study keywords: composition
LangGraph [Must know] - LangGraph is better for stateful workflows with branching, persistence, streaming, retries, and human-in-the-loop. It fits your Agentic Chat and Edward-style systems more than simple chains.
Study keywords: graphs, nodes, edges
Nodes and edges [Must know] - In graph orchestration, nodes perform work and edges route to the next step. Conditional edges let the workflow branch based on state or tool results.
Study keywords: workflow graph
Streaming agent events [Must know] - Streaming events expose intermediate progress like plan created, tool called, file changed, or approval needed. This improves UX and debuggability for long-running tasks.
Study keywords: events, progress
Agent observability [Must know] - Agent traces should show every prompt, model call, tool call, decision, latency, cost, and error. Without this, agent failures are nearly impossible to debug.
Study keywords: LangSmith, tracing
AI Engineering Fundamentals Master Map - Shubhojeet Bera - July 2026
Autonomy level [Must know] - Agent autonomy should match risk. Low-risk tasks can run automatically; high-risk tasks need approvals, dry-runs, or constrained tool permissions. Study keywords: risk-based design
14. Production AI systems and infrastructure
This is where your full-stack background becomes a real advantage. AI systems need queues, streaming, storage, observability, cost control, and failure handling.
Async job queues [Must know] - Long-running AI work should often run through queues instead of blocking HTTP requests. Queues improve reliability, retries, concurrency control, and progress reporting.
Study keywords: BullMQ, Redis, workers
Streaming UX [Must know] - Streaming output or events keeps users informed during slow model calls and agent workflows. It can use SSE, WebSockets, or polling depending on complexity.
Study keywords: SSE, WebSockets
Retries and backoff [Must know] - Model APIs and tools fail due to rate limits, network errors, and transient provider issues. Use bounded retries with backoff and idempotency, not infinite retry loops.
Study keywords: rate limits, resilience
Timeouts [Must know] - Every model, tool, and workflow step needs a timeout. Timeouts protect user experience, worker capacity, and cost.
Study keywords: timeouts, cancellation
Cancellation [Advanced] - Users should be able to stop expensive or long-running tasks. Cancellation must propagate to workers, tool calls, and state updates where possible.
Study keywords: abort, cleanup
Concurrency limits [Must know] - Concurrency limits prevent one user or workflow from exhausting model budget, workers, database connections, or vector search capacity. They are important for multi-tenant AI apps.
Study keywords: rate limiting, queues
Caching [Must know] - Caching can store embeddings, retrieval results, prompt prefixes, tool results, or final answers. Good caching reduces cost and latency but needs freshness and permission awareness.
Study keywords: semantic cache, prompt cache
Semantic caching [Advanced] - Semantic caching reuses answers or results for similar queries based on embedding similarity. It is powerful but risky if user context, permissions, or freshness differ. Study keywords: similar query cache
Model fallback [Advanced] - Fallbacks route to another model or degraded workflow when a provider fails. Fallback outputs should still pass validation and be monitored separately.
Study keywords: provider reliability
Cost accounting [Must know] - Track token cost, embedding cost, retrieval cost, tool cost, and per-workflow cost. Cost visibility helps you choose the right model and architecture.
Study keywords: unit economics
Latency optimization [Must know] - Reduce latency with smaller models, fewer calls, parallel retrieval, streaming, caching, context trimming, and precomputed embeddings. Do not optimize blindly before measuring traces. Study keywords: performance
Observability [Must know] - Observe prompts, context, outputs, scores, tool calls, errors, cost, and latency. Traditional logs are not enough; LLM apps need semantic traces.
Study keywords: tracing, monitoring
Prompt and model versioning [Must know] - Record prompt version, model name, model version, retriever version, and index version for every output. This makes debugging and rollback possible.
Study keywords: reproducibility
Data versioning [Advanced] - When documents change, old answers may no longer be valid. Store document versions, ingestion timestamps, and source metadata.
Study keywords: index version, freshness
Multi-tenancy [Must know] - Multi-tenant AI apps must isolate data, indexes, logs, permissions, and caches. A retrieval bug can become a serious privacy incident.
Study keywords: tenant isolation
Audit logs [Advanced] - Audit logs record who asked what, what data was accessed, which tools were called, and what actions were taken. They matter for enterprise trust and debugging.
AI Engineering Fundamentals Master Map - Shubhojeet Bera - July 2026
Study keywords: compliance, accountability
Secrets and BYOK [Must know] - API keys, user tokens, and BYOK secrets must be encrypted and scoped. Never expose secrets to prompts, logs, or model-readable context.
Study keywords: encryption, secret handling
Sandboxing [Must know] - Code-generation or tool-execution agents need isolated sandboxes. Sandboxing prevents generated code or commands from harming the host system or other users.
Study keywords: Docker, isolation
Preview and deployment safety [Advanced] - When agents generate apps or code, preview environments should be isolated from production. Add scanning, approval, and rollback before deployment.
Study keywords: app builder safety
SLOs for AI features [Advanced] - Define service-level objectives for availability, latency, task success, and hallucination rate. AI quality should be operationalized, not judged only by demos.
Study keywords: reliability targets
15. Security, safety, and governance
AI systems create new attack surfaces because natural language can influence model behavior. Security must be part of architecture, not an afterthought.
Prompt injection [Must know] - Prompt injection is when user or document text tries to override instructions or manipulate the model. Treat all external content as untrusted data.
Study keywords: malicious instructions
Indirect prompt injection [Must know] - Indirect prompt injection comes from retrieved documents, web pages, emails, PDFs, or tool outputs. It is dangerous because the user may not know the malicious instruction exists.
Study keywords: RAG security
Data exfiltration [Must know] - A compromised prompt or tool flow may leak private data. Enforce permissions before retrieval and never rely on the model to decide what data is allowed.
Study keywords: privacy, access control
Tool permissioning [Must know] - Tools should be scoped by user permission, task, environment, and risk. The model should not get broad write access when it only needs read access.
Study keywords: least privilege
Approval gates [Must know] - Approval gates require human confirmation before risky operations. They are needed for sending messages, deleting data, financial actions, deployments, or external side effects.
Study keywords: human approval
Input validation [Must know] - Validate user inputs and model-generated tool arguments. LLM output should never be trusted as safe just because it came from your own system.
Study keywords: schemas, sanitization
Output validation [Must know] - Validate generated JSON, URLs, SQL, code, email content, and structured fields before use. Output validators are a key guardrail for AI apps.
Study keywords: parsers, validators
SSRF risk [Advanced] - If an AI system can fetch URLs, it may be tricked into accessing internal services. Restrict outbound network access and validate URLs.
Study keywords: network security
SQL/tool injection [Must know] - If model output is used in queries or commands, it can create injection vulnerabilities. Use parameterized queries, allowlists, and constrained tool APIs.
Study keywords: command safety
PII handling [Must know] - Personally identifiable information should be minimized, encrypted, access-controlled, and logged carefully. Do not send unnecessary PII to models or third-party tools.
Study keywords: privacy
Content safety [Foundation] - Content safety filters harmful, illegal, abusive, or policy-violating outputs. Safety is contextdependent and should not be treated as only a model-provider problem.
Study keywords: moderation
Policy grounding [Advanced] - Policy grounding makes the model follow product, legal, or safety rules from authoritative policy sources. Policies should be versioned and evaluated like other prompts.
Study keywords: governance
AI Engineering Fundamentals Master Map - Shubhojeet Bera - July 2026
Red teaming [Must know] - Red teaming tests the system with adversarial prompts, malicious documents, weird edge cases, and unsafe tool attempts. It finds failures before users do.
Study keywords: adversarial testing
Auditability [Advanced] - A serious AI system should explain which sources, tools, prompts, and decisions produced an output. Auditability builds trust and supports incident response.
Study keywords: traceability
Compliance awareness [Advanced] - Depending on product domain, AI features may need data retention rules, privacy controls, consent, and audit logs. You do not need to be a lawyer, but you must design for constraints.
Study keywords: enterprise requirements
16. Fine-tuning, adaptation, and model customization
Most product teams should start with prompting and RAG before fine-tuning. But you should know when customization makes sense.
Fine-tuning [Must know] - Fine-tuning updates model weights on task-specific examples. It is useful for style, format, domain behavior, or classification, but not the first fix for missing knowledge.
Study keywords: SFT, customization
RAG vs fine-tuning [Must know] - Use RAG when the problem is missing, private, or changing knowledge. Use fine-tuning when the problem is behavior, style, format, or repeated task pattern.
Study keywords: knowledge vs behavior
LoRA [Advanced] - LoRA is a parameter-efficient fine-tuning method that trains small adapter weights instead of all model weights. It reduces compute and storage requirements.
Study keywords: adapters
QLoRA [Advanced] - QLoRA combines quantization with LoRA to fine-tune large models more cheaply. It is useful in resource-constrained training setups.
Study keywords: quantized fine-tuning
Distillation [Advanced] - Distillation trains a smaller model to imitate a larger model. It can reduce cost and latency for repeated tasks with predictable behavior.
Study keywords: student model
Synthetic data [Advanced] - Synthetic data is generated by models to create training or eval examples. It can scale datasets but must be quality-checked to avoid amplifying errors.
Study keywords: data generation
Instruction dataset [Advanced] - An instruction dataset contains user tasks and ideal responses. Its quality matters more than raw size for many fine-tuning jobs.
Study keywords: SFT dataset
Preference data [Advanced] - Preference data captures which output humans prefer between alternatives. It supports alignment methods like reward modeling or direct preference optimization.
Study keywords: DPO, RLHF
Evaluation before tuning [Must know] - Before fine-tuning, build evals to prove the current system fails and later prove tuning helped. Without evals, fine-tuning becomes expensive guessing.
Study keywords: baseline, regression
Catastrophic forgetting [Advanced] - Fine-tuning can make a model worse at general tasks if training is narrow or poor. Always evaluate broad behavior and safety after tuning.
Study keywords: forgetting
Quantization [Advanced] - Quantization reduces numerical precision to make models smaller and faster. It can reduce cost but may affect quality, especially on complex reasoning or rare tokens.
Study keywords: int8, int4
Model serving [Advanced] - Serving self-hosted models requires GPUs, batching, memory management, autoscaling, monitoring, and security. It is an infra project, not just loading a model. Study keywords: vLLM, inference
17. Multimodal, code, and emerging AI patterns
These topics are increasingly common in AI products. Learn the mental models even if you do not implement all of them immediately.
AI Engineering Fundamentals Master Map - Shubhojeet Bera - July 2026
Multimodal models [Advanced] - Multimodal models process more than text, such as images, audio, video, PDFs, and screenshots. They are useful when important context is visual or spoken.
Study keywords: vision-language models
Document AI [Must know] - Document AI extracts structured information from PDFs, forms, scans, tables, and contracts. It often combines OCR, layout parsing, multimodal models, and validation.
Study keywords: OCR, forms, tables
Table understanding [Advanced] - Tables need special handling because meaning depends on rows, columns, headers, and units. Flattening tables into plain text can destroy important structure.
Study keywords: tabular RAG
Code generation [Must know] - Code-generation agents need planning, file edits, tests, sandboxing, dependency handling, and rollback. The hard part is not generating code; it is making safe, working changes.
Study keywords: coding agents
Program synthesis eval [Must know] - Code AI should be evaluated by running tests, type checks, linting, build commands, and security scans. Natural language judgment is not enough.
Study keywords: tests, CI
Computer-use agents [Advanced] - Computer-use agents interact with browsers or desktop interfaces. They need strong observation handling, step limits, confirmation gates, and recovery from UI changes.
Study keywords: browser agents
Voice agents [Advanced] - Voice agents add speech recognition, latency pressure, interruption handling, and conversational turn-taking. UX quality depends heavily on speed and error recovery.
Study keywords: ASR, TTS
Personalization [Advanced] - Personalization adapts behavior using user preferences, history, and context. It must separate stable preferences from temporary conversation context and respect privacy.
Study keywords: memory, preferences
Model context protocol style integrations [Advanced] - Tool and context integration standards aim to connect models with external systems in safer, reusable ways. The important idea is standardized access to tools, data, and permissions. Study keywords: MCP, connectors
AI-native UX [Must know] - AI-native UX designs around uncertainty, progress, edits, approvals, and recoverability. Great AI products do not only expose a chat box; they build workflows. Study keywords: product design
18. Interview language and project mapping
Use these topics to connect fundamentals back to your actual work so you sound grounded, not theoretical.
Explaining Edward [Must know] - Describe Edward as a prompt-to-app system with streaming planning, queue-backed execution, Docker sandboxes, persisted project state, preview routing, and recovery. Tie it to agents, tool execution, state machines, streaming UX, and production reliability.
Study keywords: app builder, agent infra
Explaining Agentic Chat [Must know] - Describe Agentic Chat as a LangGraph-style multi-agent workflow with RAG, tool orchestration, human approvals, checkpointing, and Google Workspace integrations. Tie it to agent state, tool safety, memory, and evals.
Study keywords: LangGraph, RAG, tools
Explaining hardened RAG [Must know] - Describe hardened RAG as hybrid retrieval, reranking, BYOK, prompt-injection/SSRF defenses, semantic caching, and document-format support. Tie it to grounding, security, cost, and retrieval quality.
Study keywords: retrieval, security
Your strongest sentence [Must know] - Say: I build AI-native product systems around LLMs - RAG pipelines, agent workflows, tool orchestration, queues, streaming, and production reliability. I am not a model researcher, but I understand the fundamentals needed to make reliable engineering decisions.
Study keywords: positioning
Transformer interview answer [Must know] - Keep it simple: tokens become embeddings, attention mixes context across tokens, transformer layers update representations, and the decoder predicts the next token. Mention QKV, multi-head attention, positional information, and next-token prediction.
Study keywords: LLM internals
AI Engineering Fundamentals Master Map - Shubhojeet Bera - July 2026
RAG interview answer [Must know] - Explain RAG as ingestion, parsing, chunking, embedding, indexing, retrieval, reranking, context construction, generation, citations, and eval. Then discuss failure modes: bad parsing, bad chunks, retrieval miss, noisy context, unsupported answer.
Study keywords: RAG system design
Hallucination interview answer [Must know] - Say hallucination is not fixed by one trick. You reduce it with better retrieval, grounding, no-answer behavior, low temperature, tool verification, citation checks, and evals.
Study keywords: reliability
Evaluation interview answer [Must know] - Mention retrieval metrics, generation metrics, task success, human review, regression tests, latency, cost, and tool success rate. The key is measuring every component, not only judging the final answer.
Study keywords: quality engineering
Agent interview answer [Must know] - Explain agents as stateful tool-using workflows with planning, tool calls, observations, checkpoints, memory, approvals, and stop conditions. Mention that more autonomy needs more observability and safety.
Study keywords: agentic systems
Production interview answer [Must know] - Mention queues, streaming, retries, rate limits, caching, tracing, prompt/model versioning, permissions, and cost accounting. This shows you can build beyond demos.
Study keywords: infra
AI Engineering Fundamentals Master Map - Shubhojeet Bera - July 2026
Deep research prompts to use with agents or mentors
Use these prompts with a research agent, mentor, or study partner. Replace the topic placeholder and ask for examples tied to Edward, Agentic Chat, or a realistic production RAG system.
Concept deep dive prompt
Explain TOPIC from first principles for an AI product engineer. Use simple language first, then technical depth. Include one production example, one common misconception, one failure mode, and one interview-quality answer.
Build-it prompt
Give me a small TypeScript or Python experiment to understand TOPIC practically. The experiment should be runnable locally and should show what changes when I vary the key parameters.
RAG debugging prompt
Given a RAG system that gives a wrong answer, walk me through how TOPIC could be the root cause. Show the trace fields I should inspect and the fix I should try first.
Agent reliability prompt
Explain how TOPIC affects a tool-using agent. Include state, retries, tool schemas, human approval, observability, and stop conditions where relevant.
Evaluation prompt
Create an eval plan for TOPIC. Include offline tests, online metrics, golden examples, failure taxonomy, and what regression would look like.
Interview drill prompt
Ask me five interview questions about TOPIC, increasing difficulty from junior to senior. After I answer, grade me on correctness, clarity, tradeoff thinking, and production awareness.
Project mapping prompt
Map TOPIC to my projects: Edward, Agentic Chat, and hardened RAG. Tell me exactly where the concept appears, how to explain it honestly, and what not to overclaim.
30-day revision plan
Days
Focus and output
Days 1-4
Tokens, embeddings, transformer, decoding, prompting. Output: one-page explanation of how an LLM generates text.
