LLM Foundationsinterview questions & answers

Think of a kid who has read 100 crore books. Every language, every topic — this kid has seen it all. But here's the twist: this kid does not actually "understand" meaning like you do. The kid has just seen so many patterns that when you say "Mera naam ___", the kid instantly says "hai" — not because it understood your sentence, but because it has seen that pattern a billion times before. A Large Language Model (LLM) is exactly this kid, built out of math.

In an LLM, the core job is simple: given some text so far, predict the single most likely next token — over and over, one token at a time, until the reply is complete. This is called being autoregressive. It is your phone's autocomplete ("Good mor" → "morning"), scaled up a million times, trained on a huge chunk of the internet instead of your own texts.

Under the hood, LLMs use a transformer architecture. At a high level (no heavy math needed): text is broken into tokens, then an attention mechanism figures out which words in the sentence relate to each other — e.g. in "The cat sat on the mat because it was tired", attention links "it" strongly to "cat", not "mat". That's how the model tracks context inside a sentence.

Models don't read words — they read tokens (≈4 characters, roughly ¾ of a word). "ChatGPT" alone might be 3 tokens. You pay per token (input + output), so token count = cost + speed. Every model also has a context window — the max tokens it can "see" at once (system prompt + chat history + your message + the reply, all counted together). Cross that limit and older messages get dropped or you get an error.

Two knobs control randomness: temperature (0 = deterministic/robot, good for facts and code; ~1 = creative/artist, good for stories and brainstorming; too high = nonsense) and top_p (nucleus sampling — restricts the model to only the most probable words before it picks). You normally tune one of these, not both — usually temperature.

Finally, models split into closed (GPT-4o, Claude Sonnet/Haiku/Opus, Gemini — API-only, you pay per token, very capable) and open (Llama, Mistral — free weights, you host them yourself on your own GPU).

🌍 Real-world example: Ask the same factual question at temperature 0 three times — you get the identical answer every time. Ask a "write me a poem" prompt at temperature 1 three times — you get three different poems. Same model, different knob.

💡 Token = a chunk of text (~4 characters); the unit an LLM actually reads and bills by.

💡 Context window = the max tokens a model can hold in memory during one conversation turn.

💡 Attention = the mechanism that lets the model relate words in a sentence to each other.

💡 Hallucination = when an LLM confidently outputs something wrong, because it is pattern-completing, not fact-checking.

Standard definition: A Large Language Model (LLM) is a neural network — built on the transformer architecture — trained to predict the next token in a sequence of text, and it generates responses by repeatedly predicting one token at a time based on all the tokens seen so far.

from openai import OpenAI

client = OpenAI()  # reads your API key from the environment

prompt = "Suggest one good programming language for beginners, in 5 words."

for temp in [0.0, 0.0, 1.2]:
    response = client.responses.create(
        model="gpt-4o-mini",
        input=prompt,
        temperature=temp
    )
    print(f"temperature={temp} -> {response.output_text}")

Think of an LLM like a genie. Say "give me food" and it hands you something — maybe undercooked, maybe not what you wanted. Say "give me Hyderabadi biryani, medium spicy, dum-pukht style, for 2 people, in a bowl" and the genie delivers exactly that. Prompt engineering is asking the genie the right way — the model didn't get smarter, your instructions got clearer.

In an LLM, the only thing it has to work with is the text you send it. It cannot read your mind, see your screen, or guess the format you secretly wanted. Every constraint you don't state is a constraint the model is free to ignore. So a vague prompt doesn't just risk a worse answer — it risks an answer that is technically correct but useless to your program (wrong length, wrong tone, wrong shape).

Five habits fix most bad prompts: (1) be specific — numbers, audience, length, not "write something good"; (2) give it a role — "You are a senior Python developer" pulls the model toward expert-level, relevant vocabulary and depth; (3) specify the output format — "return ONLY valid JSON with keys x, y" so your code can actually parse the reply; (4) give examples (few-shot) — 2-3 input→output pairs teach the pattern faster than any explanation; (5) use delimiters — wrap the actual data in """, ---, or <text> tags so the model never confuses your instructions with the content you handed it.

🌍 Real-world example: "Summarize this" followed directly by a paragraph is ambiguous — where does the instruction end and the text begin? "Summarize the text between the triple quotes in one sentence: """..."""" removes all doubt.

💡 Zero-shot = asking directly, no examples given. 💡 Few-shot = giving 2-3 example input→output pairs before the real question, so the model copies the pattern. 💡 Delimiter = a marker (""", ---, XML tags) that clearly separates your instructions from the data being processed.

Compare these side by side:

BAD prompt: "Tell me about this product." → Model has no idea what "good" looks like: length? tone? audience? You get something generic.

GOOD prompt: "You are a marketing copywriter. Write a 40-word product description for wireless earbuds, targeting college students. Tone: casual. Return plain text only, no markdown." → Role + specificity + length + audience + format — the model has almost no room to go wrong.

This matters in real apps because your code downstream (a UI, a database, a parser) expects a predictable shape. A vague prompt today is a broken feature tomorrow.

Standard definition: Prompt engineering is the practice of crafting clear, specific instructions — including role, format, examples, and delimiters — to reliably get the output you want from an LLM.

from anthropic import Anthropic

client = Anthropic()  # reads ANTHROPIC_API_KEY from the environment

review_text = "The delivery was late and the box was damaged, but the product itself works great."

prompt = f"""You are a customer support analyst.

Classify the sentiment of the review below as Positive, Negative, or Mixed.
Also extract the single main complaint, if any.

Review:
\"\"\"
{review_text}
\"\"\"

Return ONLY valid JSON in this exact shape:
{{"sentiment": "...", "main_complaint": "..."}}
"""

response = client.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=200,
    messages=[{"role": "user", "content": prompt}]
)

print(response.content[0].text)

What we're building: A real terminal chatbot in ~20 lines of Python — one you type into, that replies, and that remembers everything said so far in the conversation. The trick behind every chatbot you've ever used (ChatGPT, Claude, your bank's support bot) is almost embarrassingly simple: a Python list that grows by two items every turn. By the end of this project you'll have built one yourself, and you'll know exactly why it "remembers" and exactly where that memory runs out.

---

Step 0 — The mental model (read first, 30 seconds)

A chatbot is not a mysterious brain that remembers your chat. It's a program that, on every single turn, resends the entire conversation so far to a stateless LLM — and the LLM (which has zero memory of its own between calls) reads the whole thing fresh and replies to the last line.

💡 Stateless = the LLM API itself remembers NOTHING between calls. Every client.chat.completions.create(...) call is a blank slate. "Memory" is an illusion your Python code creates by resending the full history each time.

So the entire chatbot is really just: one growing list + one loop. That's Step 0 — keep it in your head through every step below.

---

Step 1 — Set up (recap of Chapter 1)

pip install openai python-dotenv

# .env file
OPENAI_API_KEY=sk-your-real-key-here

What's happening: exactly the Chapter-1 project's setup — a .env file holding your secret key, never hardcoded, never committed to git. Nothing new here; this whole project stands on that foundation.

---

Step 2 — The system prompt (the bot's personality)

SYSTEM_PROMPT = "You are a friendly Hinglish tutor. Keep answers short, encouraging, and beginner-friendly."

What's happening: the system prompt is an instruction the developer sets, not the user — it shapes the bot's role, tone, and rules for the whole conversation. Change this one string and you get a totally different bot (a sarcastic coding reviewer, a formal HR interviewer, a recipe assistant) with zero other code changes.

🌍 Real-world example: every branded AI chatbot you've used — a bank's support bot, a shopping assistant — is the SAME underlying model as ChatGPT, just given a different system prompt ("You are Acme Bank's support agent. Only answer questions about Acme accounts...").

---

Step 3 — Start the messages list

messages = [
    {"role": "system", "content": SYSTEM_PROMPT}
]

What's happening: this is the ONE list that holds the entire conversation, from now until the user quits. Right now it has exactly one item — a dict with two keys, role and content, same dict shape you've used since Chapter 1. role tells the model WHO is "speaking" this entry: "system" (developer instructions), "user" (the human), or "assistant" (the model's own past replies).

💡 Message = one {"role": ..., "content": ...} dict. A conversation = a Python list of messages, in order.

---

Step 4 — The loop, and the exit condition

while True:
    user_input = input("You: ")
    if user_input.lower() == "quit":
        print("Bot: Bye! Keep practicing.")
        break

What's happening: while True is an infinite loop — the exact same loop type from Chapter 1, just with no fixed end. input("You: ") pauses the program and waits for the human to type a line, returning it as a str. We check the exit condition BEFORE doing any API work — no point burning a paid API call on the word "quit". break is the only way out of this infinite loop.

---

Step 5 — Grow the list: add the user's turn

messages.append({"role": "user", "content": user_input})

What's happening — this is the whole trick, beat 1 of 2: .append() is the exact list method from Chapter 1 — it adds one item to the END of a list, in place. The list that started with 1 item (just the system prompt) now has 2. Every single turn adds exactly one user entry here.

---

Step 6 — Call the LLM with the WHOLE list

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=messages
)
reply = response.choices[0].message.content

What's happening: notice we pass messages — the FULL list, every turn ever said, not just the newest line. This is the one thing that makes the illusion of memory work: the model has no memory of its own, so we hand it the whole transcript, every single time, and it replies to whatever's most recent while having full context of everything before. response.choices[0].message.content walks the response object exactly like data["output"][0]["content"][0]["text"] did in Chapter 1's project — nested indexing into the API's reply shape, just with openai's chat-completions structure this time.

---

Step 7 — Print it, then grow the list again: beat 2 of 2

print("Bot:", reply)
messages.append({"role": "assistant", "content": reply})

What's happening — THIS is the line that gives the bot memory: if you skip this .append(), the bot would answer your question fine once, but the NEXT time through the loop, messages would still only have the system prompt + your latest question — the model would never see its own previous answer, and the conversation would fall apart (it'd contradict itself, forget names you mentioned, repeat itself). By appending the assistant's own reply, the list now has the system prompt + turn 1's user message + turn 1's assistant reply — 3 items — ready for turn 2 to add a 4th.

---

Step 8 — Trace the list growing, turn by turn

After...	messages list length	Contents
Setup (Step 3)	1	[system]
Turn 1, you speak (Step 5)	2	[system, user1]
Turn 1, bot replies (Step 7)	3	[system, user1, assistant1]
Turn 2, you speak	4	[system, user1, assistant1, user2]
Turn 2, bot replies	5	[system, user1, assistant1, user2, assistant2]

What's happening: every turn adds exactly 2 items — one user, one assistant. This table IS the chatbot. Every call to client.chat.completions.create(...) resends this whole growing list, so the model "remembers" turn 1 when answering turn 5 — purely because turn 1 is still sitting right there in the list you're sending.

---

Step 9 — Context window awareness (why the list can't grow forever)

💡 Context window = the maximum number of tokens (roughly ¾ of a word each) a model can read in one call — the ENTIRE messages list must fit inside it, every single time.

A long chat session — 50, 100, 200 turns — makes messages bigger and bigger. Eventually one of two things happens: you hit the context window limit (the call errors out or the model truncates and silently forgets your earliest messages), or you just pay more per call, since every call re-sends the entire growing history as input tokens. A real chatbot needs a strategy to trim, summarize, or cap this list once it gets long — that's exactly what Chapter 7 (Memory Management) is about. For now: know that this simple "append forever" approach works great for a short session and is exactly where production systems add a limit.

---

🔎 The full flow (recap the wiring)

1. SYSTEM_PROMPT — a plain string, the bot's personality/rules.
2. messages = [{"role": "system", ...}] — the ONE list that holds the whole conversation, starting with 1 item.
3. while True + input() — the loop that reads one line from the human each turn; quit breaks out.
4. messages.append({"role": "user", ...}) — beat 1: the human's turn joins the list.
5. client.chat.completions.create(model=..., messages=messages) — the WHOLE list is sent, every time, because the LLM itself is stateless.
6. messages.append({"role": "assistant", ...}) — beat 2: the bot's own reply joins the list too — THIS is what makes it "remember" next turn.
7. Repeat — the list keeps growing by 2 per turn, until context window limits (Ch7) say otherwise.

---

✅ What you just learned — and what's next

• A chatbot = a growing Python list, not a magic memory. The LLM behind it is stateless; your code creates the illusion of memory by resending the full history every call.
• role (system/user/assistant) tells the model who said what, in order.
• Every turn appends exactly 2 items — one from the user, one from the assistant — right after the API call returns.
• Context window is a hard ceiling on how big that list can get — foreshadowing Chapter 7's memory-management techniques (trimming, summarizing).
• This project ties together Chapter 1 (loops, lists, input(), .env/API keys) and Chapter 2 (system prompts, the SDK, tokens) into one working program.

Standard definition: A CLI chatbot maintains a single growing list of role-tagged messages (system, user, assistant); each turn appends the user's input, sends the ENTIRE list to a stateless LLM API call, appends the model's reply back onto the list, and repeats — the resend-the-whole-history pattern is what gives a stateless LLM the appearance of memory within a session, bounded by the model's context window.

import os
from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

SYSTEM_PROMPT = "You are a friendly Hinglish tutor. Keep answers short, encouraging, and beginner-friendly."

messages = [
    {"role": "system", "content": SYSTEM_PROMPT}
]

print("Chatbot ready! Type 'quit' to exit.\n")

while True:
    user_input = input("You: ")
    if user_input.lower() == "quit":
        print("Bot: Bye! Keep practicing.")
        break

    messages.append({"role": "user", "content": user_input})

    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=messages
    )
    reply = response.choices[0].message.content

    print("Bot:", reply)
    messages.append({"role": "assistant", "content": reply})