LLM Foundations interview questions & answers

Think of a kid who has read 100 crore books. Every language, every topic — this kid has seen it all. But here's the twist: this kid does not actually "understand" meaning like you do. The kid has just seen so many patterns that when you say "Mera naam ___", the kid instantly says "hai" — not because it understood your sentence, but because it has seen that pattern a billion times before. A Large Language Model (LLM) is exactly this kid, built out of math.

In an LLM, the core job is simple: given some text so far, predict the single most likely next token — over and over, one token at a time, until the reply is complete. This is called being autoregressive. It is your phone's autocomplete ("Good mor" → "morning"), scaled up a million times, trained on a huge chunk of the internet instead of your own texts.

Under the hood, LLMs use a transformer architecture. At a high level (no heavy math needed): text is broken into tokens, then an attention mechanism figures out which words in the sentence relate to each other — e.g. in "The cat sat on the mat because it was tired", attention links "it" strongly to "cat", not "mat". That's how the model tracks context inside a sentence.

Models don't read words — they read tokens (≈4 characters, roughly ¾ of a word). "ChatGPT" alone might be 3 tokens. You pay per token (input + output), so token count = cost + speed. Every model also has a context window — the max tokens it can "see" at once (system prompt + chat history + your message + the reply, all counted together). Cross that limit and the API rejects the request with an error (a 400/413, or a response with stop_reason: "model_context_window_exceeded") — it does not silently drop your older messages. Keeping the history under the limit, by trimming or summarising old turns, is your application's job.

Two knobs reshape how the next token is sampled: temperature (low = the model sticks close to its highest-probability tokens, so replies repeat more closely; ~1 = a flatter distribution, more varied wording; too high = nonsense) and top_p (nucleus sampling — restricts the pool to the most probable tokens before it picks). Tune one of these, not both — usually temperature. Two caveats interviewers like: temperature is not a determinism switch (the same settings can still produce different text), and it does not make answers more accurate — a confidently wrong answer just becomes a consistently wrong one; accuracy comes from grounding the model in real data. These knobs are also provider-scoped: they are OpenAI API parameters, while current Claude models reject temperature and top_p with a 400 error.

Finally, models split into closed (GPT-4o, Claude Sonnet/Haiku/Opus, Gemini — API-only, you pay per token, very capable) and open (Llama, Mistral — free weights, you host them yourself on your own GPU).

🌍 Real-world example: Ask the same factual question at temperature 0 three times — the three replies usually land on very similar wording, but that is a tendency, not a guarantee, and it says nothing about whether the answer is correct. Ask a "write me a poem" prompt at temperature 1 three times — you get three different poems. Same model, different knob.

💡 Token = a chunk of text (~4 characters); the unit an LLM actually reads and bills by.

💡 Context window = the max tokens a model can hold in memory during one conversation turn.

💡 Attention = the mechanism that lets the model relate words in a sentence to each other.

💡 Hallucination = when an LLM confidently outputs something wrong, because it is pattern-completing, not fact-checking.

When an LLM is the right tool: open-ended language work — summarising, rewriting, classifying messy text, extracting fields from prose, drafting, answering from provided context. Anywhere the input varies too much to write rules for.

When it is the wrong tool: anything with one correct answer you can compute. Arithmetic, sorting, dates, business rules, lookups in your own database. A regex, a SQL query or three lines of code are exact, free and auditable; a model is approximate, billed and unexplainable. "Use the model to call the calculator" is the pattern, not "use the model as the calculator".

When to be careful even where it fits: anything where a fluent wrong answer is expensive — medical, legal, financial. Being confidently wrong is not an edge case; it is a property of next-token prediction.

Trade-off: you trade determinism for flexibility. The same prompt can produce different output on two calls, output cannot be unit-tested by equality, and there is no stack trace when it is wrong — only a prompt you can change and hope. That flexibility is worth a great deal on messy input and worth nothing on structured input you already control.

Standard definition: A Large Language Model (LLM) is a neural network — built on the transformer architecture — trained to predict the next token in a sequence of text, and it generates responses by repeatedly predicting one token at a time based on all the tokens seen so far.

from openai import OpenAI

client = OpenAI()  # reads your API key from the environment

prompt = "Suggest one good programming language for beginners, in 5 words."

for temp in [0.0, 0.0, 1.2]:
    response = client.responses.create(
        model="gpt-4o-mini",
        input=prompt,
        temperature=temp
    )
    print(f"temperature={temp} -> {response.output_text}")

Think of an LLM like a genie. Say "give me food" and it hands you something — maybe undercooked, maybe not what you wanted. Say "give me Hyderabadi biryani, medium spicy, dum-pukht style, for 2 people, in a bowl" and the genie delivers exactly that. Prompt engineering is asking the genie the right way — the model didn't get smarter, your instructions got clearer.

In an LLM, the only thing it has to work with is the text you send it. It cannot read your mind, see your screen, or guess the format you secretly wanted. Every constraint you don't state is a constraint the model is free to ignore. So a vague prompt doesn't just risk a worse answer — it risks an answer that is technically correct but useless to your program (wrong length, wrong tone, wrong shape).

Five habits fix most bad prompts: (1) be specific — numbers, audience, length, not "write something good"; (2) give it a role — "You are a senior Python developer" pulls the model toward expert-level, relevant vocabulary and depth; (3) specify the output format — "return ONLY valid JSON with keys x, y" so your code can actually parse the reply; (4) give examples (few-shot) — 2-3 input→output pairs teach the pattern faster than any explanation; (5) use delimiters — wrap the actual data in """, ---, or <text> tags so the model is far less likely to confuse your instructions with the content you handed it. Delimiters are ordinary tokens in the prompt, not an enforced parser boundary — content that itself contains the delimiter can still break out — but they greatly reduce mix-ups and prompt injection.

🌍 Real-world example: "Summarize this" followed directly by a paragraph is ambiguous — where does the instruction end and the text begin? "Summarize the text between the triple quotes in one sentence: """..."""" makes the boundary much clearer — though if the text itself contains triple quotes, the boundary can still blur.

💡 Zero-shot = asking directly, no examples given. 💡 Few-shot = giving 2-3 example input→output pairs before the real question, so the model copies the pattern. 💡 Delimiter = a marker (""", ---, XML tags) that clearly separates your instructions from the data being processed.

Compare these side by side:

BAD prompt: "Tell me about this product." → Model has no idea what "good" looks like: length? tone? audience? You get something generic.

GOOD prompt: "You are a marketing copywriter. Write a 40-word product description for wireless earbuds, targeting college students. Tone: casual. Return plain text only, no markdown." → Role + specificity + length + audience + format — the model has almost no room to go wrong.

This matters in real apps because your code downstream (a UI, a database, a parser) expects a predictable shape. A vague prompt today is a broken feature tomorrow.

Beyond those five habits, four more techniques are worth knowing well — they show up constantly in interviews because the trade-offs matter as much as the definitions.

Role prompting is habit #2 taken seriously: you assign the model an identity — "You are a senior security engineer", "You are a patient teacher explaining to a 10-year-old" — and the reply visibly shifts in vocabulary, depth, and default assumptions. Why does this work at all? An LLM has seen enormous amounts of text written by doctors, by teachers, by engineers — each with its own conventions (a doctor hedges and lists differentials; a teacher explaining to a child avoids jargon and uses short sentences). Naming the role is a cheap, reliable way to steer the model toward the region of its training data that matches. The honest caveat: a role changes style and framing, not ground truth. "You are a senior doctor" does not make the medical facts more correct — it only makes the answer sound and read like one written by a doctor. Never rely on a role alone where correctness matters; combine it with grounding or verification.

Prompt chaining splits one big task into several smaller prompts, where each prompt's output becomes the next prompt's input — instead of "summarize this article, pull out 5 key points, translate them to Hindi, and write a tweet" in one shot, you run four separate calls: summarize → extract points → translate → tweet. Chaining beats one giant prompt when the task has genuinely distinct sub-steps, because (a) each individual prompt is simpler and more likely to be followed correctly, (b) when something goes wrong you can tell which step produced the bad output and fix that step alone, and (c) you can validate or edit the intermediate output before it goes any further. The cost is real: N prompts mean N API calls, so more latency and more total tokens billed than the single mega-prompt — and an error in an early step (a bad summary) silently poisons every step downstream. Use chaining when steps are logically separable; skip it for genuinely simple, one-shot tasks.

🌍 Real-world example: a "generate a blog post from these bullet points, SEO-optimize it, and write 3 social captions" prompt is really three jobs wearing a trenchcoat — chaining them means you can regenerate just the captions if only those are weak, instead of re-rolling the whole post.

Negative prompting states what you do not want — "do not use jargon", "do not exceed 5 sentences", "do not start with 'In conclusion'". It is a real, usable technique, but it is honestly a weaker constraint than positive prompting. "Don't use jargon" rules out one category of failure and leaves an unlimited number of other ways to miss the mark (too long, too casual, wrong audience); "explain this in one simple sentence a 10-year-old could follow" hands the model an actual target to aim for. The practical rule: use negative instructions as guardrails layered on top of a positive instruction, not as a substitute for one.

Iterative refinement treats a prompt the way you'd treat any other piece of code you're debugging: write it, run it, look at what came back, and tighten the prompt based on the actual gap — not write once and accept whatever you get. "Write a product description for headphones" → too generic. Add audience and format → better, but too long. Add a word limit and tone → now it's usable. Expect 2-3 rounds on anything that matters; a prompt is a first draft, not a final answer.

When prompt engineering is the right fix: the model can do the task but is doing it inconsistently — wrong format, wrong length, wrong tone, missing a constraint you never stated. That is nearly every early problem, and it costs one edit and no infrastructure.

When it is NOT the fix: the model does not have the facts (it needs RAG), or you need the same voice across thousands of replies and prompting keeps drifting (that is fine-tuning). Climb the ladder in that order — prompting, then RAG, then fine-tuning — because each rung costs far more setup than the one below.

When to stop adding to a prompt: when it has become a document nobody can reason about. A prompt with fifteen rules will have some ignored, and you cannot tell which. At that point split the task into two calls, or move the rules into structured output and validation.

Trade-off: a prompt is fast to change and impossible to test properly. Improving it for one input can silently break another, and you only find out from a user — which is why anything important needs a small set of saved examples you re-run after every prompt edit. Without that, prompt work is guessing with extra steps.

Standard definition: Prompt engineering is the practice of crafting clear, specific instructions — including role, format, examples, and delimiters — to reliably get the output you want from an LLM.

from anthropic import Anthropic

client = Anthropic()  # reads ANTHROPIC_API_KEY from the environment

review_text = "The delivery was late and the box was damaged, but the product itself works great."

prompt = f"""You are a customer support analyst.

Classify the sentiment of the review below as Positive, Negative, or Mixed.
Also extract the single main complaint, if any.

Review:
\"\"\"
{review_text}
\"\"\"

Return ONLY valid JSON in this exact shape:
{{"sentiment": "...", "main_complaint": "..."}}
"""

response = client.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=200,
    messages=[{"role": "user", "content": prompt}]
)

print(response.content[0].text)

What we're building: A real terminal chatbot in ~20 lines of Python — one you type into, that replies, and that remembers everything said so far in the conversation. The trick behind every chatbot you've ever used (ChatGPT, Claude, your bank's support bot) is almost embarrassingly simple: a Python list that grows by two items every turn. By the end of this project you'll have built one yourself, and you'll know exactly why it "remembers" and exactly where that memory runs out.

---

Step 0 — The mental model (read first, 30 seconds)

A chatbot is not a mysterious brain that remembers your chat. It's a program that, on every single turn, resends the entire conversation so far to a stateless LLM — and the LLM (which has zero memory of its own between calls) reads the whole thing fresh and replies to the last line.

💡 Stateless = the LLM API itself remembers NOTHING between calls. Every client.chat.completions.create(...) call is a blank slate. "Memory" is an illusion your Python code creates by resending the full history each time.

So the entire chatbot is really just: one growing list + one loop. That's Step 0 — keep it in your head through every step below.

---

Step 1 — Set up (recap of Chapter 1)

pip install openai python-dotenv

# .env file
OPENAI_API_KEY=sk-your-real-key-here

What's happening: exactly the Chapter-1 project's setup — a .env file holding your secret key, never hardcoded, never committed to git. Nothing new here; this whole project stands on that foundation.

---

Step 2 — The system prompt (the bot's personality)

SYSTEM_PROMPT = "You are a friendly Hinglish tutor. Keep answers short, encouraging, and beginner-friendly."

What's happening: the system prompt is an instruction the developer sets, not the user — it shapes the bot's role, tone, and rules for the whole conversation. Change this one string and you get a totally different bot (a sarcastic coding reviewer, a formal HR interviewer, a recipe assistant) with zero other code changes.

🌍 Real-world example: every branded AI chatbot you've used — a bank's support bot, a shopping assistant — is the SAME underlying model as ChatGPT, just given a different system prompt ("You are Acme Bank's support agent. Only answer questions about Acme accounts...").

---

Step 3 — Start the messages list

messages = [
    {"role": "system", "content": SYSTEM_PROMPT}
]

What's happening: this is the ONE list that holds the entire conversation, from now until the user quits. Right now it has exactly one item — a dict with two keys, role and content, same dict shape you've used since Chapter 1. role tells the model WHO is "speaking" this entry: "system" (developer instructions), "user" (the human), or "assistant" (the model's own past replies).

💡 Message = one {"role": ..., "content": ...} dict. A conversation = a Python list of messages, in order.

---

Step 4 — The loop, and the exit condition

while True:
    user_input = input("You: ")
    if user_input.lower() == "quit":
        print("Bot: Bye! Keep practicing.")
        break

What's happening: while True is an infinite loop — the exact same loop type from Chapter 1, just with no fixed end. input("You: ") pauses the program and waits for the human to type a line, returning it as a str. We check the exit condition BEFORE doing any API work — no point burning a paid API call on the word "quit". break is the only way out of this infinite loop.

---

Step 5 — Grow the list: add the user's turn

messages.append({"role": "user", "content": user_input})

What's happening — this is the whole trick, beat 1 of 2: .append() is the exact list method from Chapter 1 — it adds one item to the END of a list, in place. The list that started with 1 item (just the system prompt) now has 2. Every single turn adds exactly one user entry here.

---

Step 6 — Call the LLM with the WHOLE list

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=messages
)
reply = response.choices[0].message.content

What's happening: notice we pass messages — the FULL list, every turn ever said, not just the newest line. This is the one thing that makes the illusion of memory work: the model has no memory of its own, so we hand it the whole transcript, every single time, and it replies to whatever's most recent while having full context of everything before. response.choices[0].message.content walks the response object exactly like data["output"][0]["content"][0]["text"] did in Chapter 1's project — nested indexing into the API's reply shape, just with openai's chat-completions structure this time.

---

Step 7 — Print it, then grow the list again: beat 2 of 2

print("Bot:", reply)
messages.append({"role": "assistant", "content": reply})

What's happening — THIS is the line that gives the bot memory: if you skip this .append(), the bot would answer your question fine once, but the NEXT time through the loop, messages would still only have the system prompt + your latest question — the model would never see its own previous answer, and the conversation would fall apart (it'd contradict itself, forget names you mentioned, repeat itself). By appending the assistant's own reply, the list now has the system prompt + turn 1's user message + turn 1's assistant reply — 3 items — ready for turn 2 to add a 4th.

---

Step 8 — Trace the list growing, turn by turn

After...	messages list length	Contents
Setup (Step 3)	1	[system]
Turn 1, you speak (Step 5)	2	[system, user1]
Turn 1, bot replies (Step 7)	3	[system, user1, assistant1]
Turn 2, you speak	4	[system, user1, assistant1, user2]
Turn 2, bot replies	5	[system, user1, assistant1, user2, assistant2]

What's happening: every turn adds exactly 2 items — one user, one assistant. This table IS the chatbot. Every call to client.chat.completions.create(...) resends this whole growing list, so the model "remembers" turn 1 when answering turn 5 — purely because turn 1 is still sitting right there in the list you're sending.

---

Step 9 — Context window awareness (why the list can't grow forever)

💡 Context window = the maximum number of tokens (roughly ¾ of a word each) a model can read in one call — the ENTIRE messages list must fit inside it, every single time.

A long chat session — 50, 100, 200 turns — makes messages bigger and bigger. Eventually one of two things happens: you hit the context window limit (the call fails with a 400 context_length_exceeded error — nothing is dropped for you), or you just pay more per call, since every call re-sends the entire growing history as input tokens. A real chatbot needs a strategy to trim, summarize, or cap this list once it gets long — that's exactly what Chapter 7 (Memory Management) is about. For now: know that this simple "append forever" approach works great for a short session and is exactly where production systems add a limit.

---

🔎 The full flow (recap the wiring)

1. SYSTEM_PROMPT — a plain string, the bot's personality/rules.
2. messages = [{"role": "system", ...}] — the ONE list that holds the whole conversation, starting with 1 item.
3. while True + input() — the loop that reads one line from the human each turn; quit breaks out.
4. messages.append({"role": "user", ...}) — beat 1: the human's turn joins the list.
5. client.chat.completions.create(model=..., messages=messages) — the WHOLE list is sent, every time, because the LLM itself is stateless.
6. messages.append({"role": "assistant", ...}) — beat 2: the bot's own reply joins the list too — THIS is what makes it "remember" next turn.
7. Repeat — the list keeps growing by 2 per turn, until context window limits (Ch7) say otherwise.

---

✅ What you just learned — and what's next

• A chatbot = a growing Python list, not a magic memory. The LLM behind it is stateless; your code creates the illusion of memory by resending the full history every call.
• role (system/user/assistant) tells the model who said what, in order.
• Every turn appends exactly 2 items — one from the user, one from the assistant — right after the API call returns.
• Context window is a hard ceiling on how big that list can get — foreshadowing Chapter 7's memory-management techniques (trimming, summarizing).
• This project ties together Chapter 1 (loops, lists, input(), .env/API keys) and Chapter 2 (system prompts, the SDK, tokens) into one working program.

Standard definition: A CLI chatbot maintains a single growing list of role-tagged messages (system, user, assistant); each turn appends the user's input, sends the ENTIRE list to a stateless LLM API call, appends the model's reply back onto the list, and repeats — the resend-the-whole-history pattern is what gives a stateless LLM the appearance of memory within a session, bounded by the model's context window.

import os
from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

SYSTEM_PROMPT = "You are a friendly Hinglish tutor. Keep answers short, encouraging, and beginner-friendly."

messages = [
    {"role": "system", "content": SYSTEM_PROMPT}
]

print("Chatbot ready! Type 'quit' to exit.\n")

while True:
    user_input = input("You: ")
    if user_input.lower() == "quit":
        print("Bot: Bye! Keep practicing.")
        break

    messages.append({"role": "user", "content": user_input})

    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=messages
    )
    reply = response.choices[0].message.content

    print("Bot:", reply)
    messages.append({"role": "assistant", "content": reply})