Senior-principal Q&A bank for ai.shubhojeet.me + revise-ai.shubhojeet.me
Compiled from the two PDFs plus deep research across Towards AI, MyEngineeringPath, The HLD Handbook, and the dipakkr/ai-engineering-guide repo.
Use this as a self-test. For each question, answer out loud in 1–2 minutes, then check the senior answer. If your answer misses the common fuck-up, drill that topic.
Part 1 — AI Agent vs Agentic AI (10 questions)
Q1. What is an AI agent?
Senior answer: A system that runs an observe → plan/reason → act → observe loop. The model chooses actions (usually tool calls), the system executes them, and the loop continues until a goal is reached, a budget is exhausted, or a human stops it. It needs state, tools, stop conditions, and observability.
Common fuck-up: Calling any LLM call an "agent." A chatbot that answers one question is not an agent; it has no action loop.
Senior answer: The paradigm of building AI systems that act autonomously or semi-autonomously over multiple steps: planning, tool use, memory, reflection, and multi-agent coordination. It is a design philosophy; an AI agent is one concrete instance.
Common fuck-up: Using "agentic" as a buzzword for any LLM feature. Agentic means the system takes actions over time, not just generates text.
Q3. What is the minimum viable agent loop?
Senior answer:
An LLM that can emit structured tool calls.
Tools with JSON schemas and clear descriptions.
A scratchpad / state of past thoughts and observations.
A loop that feeds tool results back and checks for termination (goal, max steps, budget).
Common fuck-up: Building reflection, planning, and multi-agent orchestration before the basic loop is reliable.
Q4. What is the ReAct pattern?
Senior answer: Reason + Act: the model interleaves reasoning traces ("I need to find X") with actions (tool calls) and observations (tool results). The scratchpad grounds the model in what actually happened and prevents error propagation.
Common fuck-up: Letting the model "reason" privately without a structured scratchpad. Hidden reasoning can hallucinate tool results and is un-auditable.
Q5. When should you use an agent vs. a simple chain or workflow?
Senior answer: Use a chain/workflow when the steps are known in advance and deterministic. Use an agent when the next step depends on the result of previous tool calls, or when the number of steps is not fixed. Most production systems are workflows, not agents.
Common fuck-up: Reaching for LangGraph or "agents" for a deterministic ETL pipeline. That adds cost, latency, and non-determinism for no benefit.
Q6. What is the difference between a workflow and an agent?
Senior answer: Anthropic's framing: a workflow is code-defined — the orchestrator decides the path. An agent is LLM-directed — the model decides the next action. LangChain/LangGraph can implement either; the label depends on who controls the control flow.
Common fuck-up: Calling a hard-coded DAG in LangGraph an "agent" just because it uses an LLM.
Q7. What are the four layers of agent memory?
Senior answer:
Scratchpad / working memory: current turn's Thought/Action/Observation.
Episodic memory: past attempts and reflections (what worked/failed).
Semantic / long-term memory: facts, documents, user preferences (often RAG).
Procedural memory: how to use tools, encoded in prompts/tool schemas.
Common fuck-up: Dumping the entire conversation into the context window and calling it memory. Real memory requires retrieval, summarization, and state management.
Q8. What is reflection (Reflexion) and when does it help?
Senior answer: After a failure, the agent writes a natural-language critique into an episodic buffer and retries. It helps when there is a verifiable oracle: tests pass, compiler succeeds, answer matches ground truth. It does not help for open-ended creative tasks.
Common fuck-up: Adding reflection to a chatbot with no external feedback signal. The model just agrees with itself and burns tokens.
Q9. What is tool/function calling in agents?
Senior answer: The model outputs a structured JSON request (tool name + arguments). The client validates schema, checks authorization, executes the tool, and returns the result. The model never executes code directly; the system does, with proper auth and sandboxing.
Common fuck-up: Letting the model execute tools without validation. Tool calls are a security boundary; parse, validate, authorize, then execute.
Q10. What is the Model Context Protocol (MCP)?
Senior answer: An open JSON-RPC protocol (Anthropic, 2024) that standardizes how clients discover and call tools/resources from servers. It decouples the agent from specific integrations: one protocol for Slack, GitHub, Postgres, Puppeteer, etc.
Common fuck-up: Treating MCP as a security boundary. MCP is interoperability, not authorization. You still need least-privilege access controls and human approval.
Part 2 — RAG & Retrieval (10 questions)
Q11. What is RAG and why use it instead of fine-tuning?
Senior answer: RAG retrieves relevant documents at query time and injects them into the prompt. Use it when knowledge changes, is private, or is too large to memorize. Fine-tuning changes behavior/style; it is a poor way to add or update factual knowledge.
Common fuck-up: Fine-tuning on company docs to "make the model know them." That gives no provenance, no citations, and is hard to update.
Q12. What is the full RAG pipeline?
Senior answer: Ingest → parse → clean → chunk → embed → index → query rewrite/expand → retrieve → rerank → context construction → generate → cite → evaluate. Each stage is a hypothesis and a failure point.
Common fuck-up: Treating RAG as "vector search + LLM." Most failures happen at chunking, query understanding, and context construction, not embedding quality.
Q13. What is the difference between dense and sparse retrieval?
Common fuck-up: Using only dense retrieval for technical/legal queries with exact terminology. Hybrid is the production default.
Q14. What is hybrid retrieval and how do you combine results?
Senior answer: Run dense and sparse in parallel, then fuse ranked lists with Reciprocal Rank Fusion (RRF) or a learned linear combination. Normalize because the score scales differ.
Common fuck-up: Adding BM25 and cosine scores directly. One scale dominates; use RRF or calibrate weights on a validation set.
Q15. What is a reranker and when is it worth the latency?
Senior answer: A cross-encoder scores query-document pairs jointly, producing a more accurate relevance score than a bi-encoder. It is worth it when accuracy matters and you can retrieve 20–100 candidates first and rerank to top-5.
Common fuck-up: Reranking only 3 candidates. Reranking cannot rescue a poor first-stage retriever.
Q16. What is chunking and what strategies matter?
Senior answer: Chunking splits documents for indexing. Strategies:
Fixed-size with overlap: simple, fast, may split meaning.
Semantic: split at topic/paragraph boundaries.
Parent-child: index small child chunks, return large parent chunks for context.
Structure-aware: by headings, tables, code AST.
Common fuck-up: Believing embedding model choice matters more than chunking. Bad chunk boundaries are the #1 RAG silent killer.
Q17. What is the lost-in-the-middle problem?
Senior answer: LLMs recall facts at the start and end of a long context better than the middle. For 128K contexts, middle positions can lose 15–20% accuracy.
Common fuck-up: Stuffing a full document into a long-context window instead of retrieving the right chunks and placing critical facts at the start/end.
Q18. What is HyDE?
Senior answer: Hypothetical Document Embeddings. Generate a hypothetical answer to the query, embed that instead of the short query, then retrieve documents similar to the hypothetical answer. Helps with vocabulary mismatch between short queries and long documents.
Common fuck-up: Using HyDE for every query. It adds an LLM call and can retrieve false-premise documents if the hypothetical answer is wrong.
Q19. What are context precision and context recall?
Senior answer (RAGAS):
Context precision: fraction of retrieved chunks that are relevant (noise).
Context recall: fraction of relevant information that was retrieved (completeness).
Common fuck-up: Optimizing recall alone. High recall with low precision drowns the generator in noise.
Q20. When would you NOT use RAG?
Senior answer:
The domain is well-covered by the model's training data and does not change.
Latency is too tight (retrieval adds 50–200 ms).
The corpus is small enough to fit directly in the context window.
The task is creative, not factual.
Common fuck-up: Adding RAG to a product where the model already knows the answer and retrieval only adds latency and cost.
Part 3 — LLM Foundations (10 questions)
Q21. How does a transformer compute self-attention?
Senior answer: For each token, compute query Q, key K, value V by linear projections. Attention scores = softmax((QK^T) / sqrt(d_k)). Output = weighted sum of V. Multi-head attention runs multiple independent attention operations in parallel.
Common fuck-up: Describing attention as "the model looks at important words." You need Q/K/V, scaling, and softmax to show you understand the mechanism.
Q22. What is the KV cache and why does it matter for inference?
Senior answer: During autoregressive generation, keys and values from previous tokens are stored so they are not recomputed. Without KV cache, generating token N costs O(N^2); with it, O(N) per step. It dominates memory at long context / high concurrency.
Common fuck-up: Thinking a model "fits in GPU memory" means it can serve many concurrent users. KV cache memory often exceeds weight memory for long contexts.
Q23. What is the difference between prefill and decode?
Senior answer:Prefill processes the entire input prompt in parallel and fills the KV cache. It is compute-bound. Decode generates one token at a time, reading the KV cache; it is often memory-bandwidth-bound. Time-to-first-token is prefill; inter-token latency is decode.
Common fuck-up: Reporting "tokens per second" without separating prefill and decode. They have different bottlenecks and optimization strategies.
Q24. What is temperature and how does it work?
Senior answer: Temperature T scales logits before softmax: p_i = exp(z_i/T) / Σ exp(z_j/T). T→0 = greedy; T=1 = raw distribution; T>1 = flatter (more diverse); T<1 = sharper (more deterministic).
Common fuck-up: Calling temperature a "creativity knob." It is a sampling policy parameter; high T does not make the model smarter or more creative, it just increases variance.
Q25. What is top-p (nucleus) sampling?
Senior answer: Top-p selects the smallest set of tokens whose cumulative probability exceeds p, then samples from that set. It is adaptive: confident predictions use a small set; uncertain ones use a larger set.
Common fuck-up: Treating top-p and temperature as independent. They interact: low temperature with high top-p is effectively greedy.
Q26. What is the difference between encoder, decoder, and encoder-decoder transformers?
Senior answer: Encoders (BERT) use bidirectional attention for understanding tasks. Decoders (GPT) use causal/left-to-right masking for generation. Encoder-decoders (T5) separate input understanding from output generation.
Common fuck-up: Saying decoders cannot do classification. With the right prompt/head, decoder-only models can classify, but they are optimized for generation.
Q27. What is tokenization and why does it matter?
Senior answer: Tokenization splits text into subword pieces the model processes. It determines cost (per token), context-window usage, and how the model handles rare words, multilingual text, and code.
Common fuck-up: Estimating cost by characters. Different languages and code tokenize very differently; non-Latin scripts can use 2–4× more tokens.
Q28. What are embeddings and how are they used?
Senior answer: Embeddings are dense vectors that capture semantic meaning. They are used for retrieval (separate embedding model) and as the model's input representation (lookup layer). Do not confuse the two.
Common fuck-up: Assuming the embedding model "understands" text the same way the LLM does. They have different training objectives and vocabularies.
Q29. What is positional encoding?
Senior answer: A mechanism to inject token order into the transformer, since attention itself is order-invariant. Classic: sinusoidal; modern: learned positional embeddings or RoPE (rotary).
Common fuck-up: Forgetting that some position methods do not generalize beyond training length, so long-context models need extrapolation-aware RoPE or similar.
Q30. What is the context window and why is it a budget, not a strategy?
Senior answer: The maximum tokens the model can process in one call. A larger window lets you fit more, but it raises cost and latency and may degrade middle-of-context recall. Retrieval, chunking, and compression are still needed.
Common fuck-up: "We have a 128K context window, so we don't need RAG." Long context raises the ceiling; it does not replace retrieval.
Part 4 — Fine-tuning, Training, Inference (8 questions)
Q31. What is fine-tuning and what does it actually change?
Senior answer: Continued training on a smaller task-specific dataset to adapt style, format, tone, or task behavior. It changes behavior, not a reliable truth database.
Common fuck-up: Fine-tuning to inject new facts. Use RAG/tools for facts; fine-tune for form.
Q32. What is LoRA and why use it?
Senior answer: Low-Rank Adaptation adds small trainable rank-decomposition matrices to frozen base weights. It trains ~1–5% of parameters, cutting compute and storage. Adapters can be swapped per task.
Common fuck-up: Expecting LoRA to learn new knowledge or fix a broken retrieval pipeline. It is efficient behavioral adaptation.
Q33. What is QLoRA?
Senior answer: LoRA + 4-bit/8-bit quantization of the base model. It reduces VRAM so a 70B model can be fine-tuned on a single consumer GPU.
Common fuck-up: Using QLoRA without measuring quantization impact on reasoning and rare-token tasks. It can degrade quality.
Q34. What is RLHF vs DPO?
Senior answer: Both align models with human preferences. RLHF trains a reward model and optimizes with PPO. DPO directly optimizes from preference pairs without a reward model. DPO is simpler and often the modern default.
Common fuck-up: Believing RLHF/DPO fix hallucination. They improve helpfulness, style, and safety; they do not guarantee facts.
Q35. What is distillation?
Senior answer: Train a smaller "student" model to imitate a larger "teacher" model's outputs or logits. It reduces latency/cost for well-defined tasks.
Common fuck-up: Distilling for open-ended tasks where the teacher's behavior is too broad; the student fails on edge cases.
Q36. What is quantization and what are the trade-offs?
Senior answer: Reduces numerical precision of weights/activations (INT8, INT4, FP8, GGUF). It lowers memory and can increase throughput, but may degrade quality on reasoning and rare tokens.
Common fuck-up: Assuming "4-bit = 4× speedup." Speedup depends on kernel support; there are also scales, metadata, and non-quantized layers.
Q37. What is continuous batching?
Senior answer: The serving system adds and removes requests between decode iterations instead of waiting for a whole batch. It improves GPU utilization but can hurt P99 latency.
Common fuck-up: Maximizing throughput without latency SLOs. A large batch can saturate throughput while making the user experience terrible.
Q38. What is speculative decoding?
Senior answer: A smaller draft model proposes several tokens; the target model verifies them in parallel. Speedup depends on draft acceptance rate and overhead.
Common fuck-up: Applying speculative decoding to all workloads. It helps most on low-entropy, repetitive text, not on every task.
Part 5 — Evaluation & Metrics (7 questions)
Q39. What is the difference between precision@k and recall@k in retrieval?
Senior answer:Recall@k: of all relevant documents, how many are in the top k. Precision@k: of the top k documents, how many are relevant. High recall without precision means noise; high precision without recall means missing answers.
Common fuck-up: Reporting recall@k only. A system can retrieve everything and have low precision.
Q40. What is MRR?
Senior answer: Mean Reciprocal Rank = average of 1 / rank_of_first_relevant across queries. It rewards placing the first correct answer high.
Common fuck-up: Using MRR when multiple relevant answers matter. It only cares about the first one.
Q41. What is nDCG?
Senior answer: Normalized Discounted Cumulative Gain. It grades relevance (0, 1, 2, 3...) and discounts by rank, then normalizes by the ideal ordering. It rewards ranking highly-relevant items first.
Common fuck-up: Using arbitrary relevance grades without calibration. nDCG is only as good as the grading rubric.
Q42. What is faithfulness in RAG evaluation?
Senior answer: Faithfulness checks whether every claim in the generated answer is supported by the retrieved context. It is the core defense against hallucination in RAG.
Common fuck-up: Confusing fluency with faithfulness. A smooth, confident answer can still be unsupported.
Q43. What is LLM-as-judge and how do you make it trustworthy?
Senior answer: Using an LLM to score outputs. To make it trustworthy: calibrate against human labels, measure inter-annotator agreement, randomize answer order (pairwise), and audit for length/position/self-preference bias.
Common fuck-up: Treating LLM-as-judge as ground truth without calibration. It is a proxy, not oracle.
Q44. What is RAGAS?
Senior answer: An open framework measuring RAG quality: context precision, context recall, faithfulness, answer relevancy.
Common fuck-up: Running RAGAS once and declaring victory. Metrics must be tracked over time with regression gates.
Q45. Why do offline metrics not guarantee product success?
Senior answer: Offline metrics are proxies. User value depends on latency, UX, trust, safety, and actual task completion rate. A/B tests and product metrics are needed to close the loop.
Common fuck-up: Optimizing BLEU/Rouge/LLM-judge score while users hate the output.
Part 6 — Production, Security, MLOps (8 questions)
Q46. What is prompt injection?
Senior answer: An attack where user or retrieved content contains instructions that override the system prompt. Direct injection comes from the user; indirect injection comes from documents, tools, or external data.
Common fuck-up: Relying on the system prompt to defend against injection. The model is not a security boundary.
Q47. How do you defend against prompt injection?
Senior answer: Defense in depth: label and isolate untrusted content, least-privilege tools, deterministic auth outside the model, output validation, human approval for consequential actions, sandboxing, monitoring, red teaming.
Common fuck-up: One defense. There is no single perfect defense.
Q48. What is least privilege in agentic systems?
Senior answer: Give the agent only the tools and permissions it needs for the current task. A support agent should not have write access to orders or user deletion.
Common fuck-up: Giving the agent broad API access because "it makes the demo more powerful."
Q49. What is MLOps for LLMs?
Senior answer: The control system for versioning and deploying code, data, prompts, indexes, models, and evaluation sets, with reproducibility, CI/CD, canary/shadow releases, monitoring, and rollback.
Common fuck-up: Treating prompt or retrieval-index changes as "not a deployment." They change behavior and need regression gates.
Q50. What is observability for AI systems?
Senior answer: Four dimensions: operational health (errors, latency, saturation), model/system quality (task success, groundedness, retrieval quality), economics (tokens, GPU, cost per success), and risk (policy violations, access, PII).
Common fuck-up: Logging only the final answer. You cannot debug retrieval, tool, or prompt failures without traces.
Q51. What is an SLO and an error budget in AI?
Senior answer: SLOs are targets for availability, p95 latency, task success, hallucination rate, tool success, etc. An error budget decides when to slow feature rollout and focus on reliability.
Common fuck-up: Only tracking uptime and cost while answer quality silently degrades.
Q52. What is drift and why does it matter?
Senior answer: Model drift (provider updates), data drift (query distribution changes), and concept drift (world changes). Detect with weekly golden-set evals and trace analysis.
Common fuck-up: Assuming a "frozen" prompt/index/model stays stable. Providers change models, users change behavior, and knowledge ages.
Q53. What is a guardrail?
Senior answer: A filter that validates input before the model or output before the user. Input guardrails catch injection and policy violations; output guardrails catch harmful content, PII, and format errors.
Common fuck-up: One-layer guardrail. Use multiple independent checks and never rely solely on the model refusing.
Part 7 — System Design & Leadership (7 questions)
Q54. What is the quality-risk-cost envelope?
Senior answer: The first step in senior AI system design is defining what "good enough" means across quality, risk, and cost. Every decision flows from that envelope.
Common fuck-up: Picking the best model first without constraints. The best model is the one that meets quality at the cost and risk budget.
Q55. When should you use synchronous vs asynchronous AI processing?
Senior answer: Synchronous for user-facing chat/tools. Asynchronous for batch jobs, long-running agents, and heavy analysis. Async is cheaper (Batch API) and avoids rate limits but adds minutes/hours of latency.
Common fuck-up: Forcing long-running agent tasks into a synchronous HTTP request and hitting timeouts.
Q56. How do you decide self-hosted vs managed API LLMs?
Senior answer: Managed API is right for most teams. Self-host when monthly spend exceeds ~$50K, you have an ML infra team, and you have strict data residency or latency requirements.
Common fuck-up: Self-hosting to "save money" while underestimating 24/7 GPU ops, monitoring, and on-call cost.
Q57. What is multi-tenancy in AI systems?
Senior answer: Multiple users/organizations share the same infrastructure while remaining isolated in data, retrieval, quotas, and cost. Includes rate limits, prefix-cache isolation, and data access controls.
Common fuck-up: Sharing prefix caches or vector indexes across tenants and leaking data.
Q58. What is model routing?
Senior answer: Choosing which model to call based on task, latency, cost, risk, or tenant. Examples: small model for classification, large model for generation, cheap model for summarization.
Common fuck-up: Sending every request to the strongest model. Routing can cut cost 5–10× without hurting quality.
Q59. What is a cascade in AI systems?
Senior answer: Try a cheap/fast stage first, escalate to a stronger/more expensive stage only when confidence is low or validation fails. Example: rules → small classifier → LLM.
Common fuck-up: Using confidence alone to escalate. A weak model can be confidently wrong; use task-specific checks and disagreement.
Q60. What makes a senior AI engineer different from an ML researcher?
Senior answer: The ML researcher invents/trains models. The senior AI engineer ships systems around models: prompts, retrieval, tools, orchestration, evaluation, UI, infra, observability, security. They understand the engine enough to tune it, but their output is the whole car.
Common fuck-up: Optimizing model metrics while ignoring latency, cost, UX, and failure recovery.
Quick self-test: 60-second drill
Pick any five questions above. Set a timer. Answer each in 60 seconds, then check:
Did you mention a system concern (latency, cost, security, eval)?
Did you mention a failure mode?
Did you avoid the common fuck-up?
Did you tie the answer to a real project (Edward / Agentic Chat / Fieldcraft / hardened RAG)?
If not, revisit that topic.
Sources
ai.shubhojeet.me — Production AI Engineering (2026 edition)
revise-ai.shubhojeet.me — AI Engineering Fundamentals: In-Depth End-to-End Guide (July 2026)
Towards AI — "Senior AI Interviews Don’t Test What You Know. They Test What Breaks at 2am."
