Memory Management interview questions & answers

Picture a call-center agent with amnesia: every time you dial in, they have zero memory of your last call — no ticket history, nothing. If the call actually flows well, it's because a different person (a supervisor) is silently sliding your entire past-conversation transcript across the desk to the agent right before they pick up the phone, every single time you call. The agent isn't remembering anything — someone is re-feeding them the whole history on every call.

In LLMs, the model is that amnesiac agent. A model API call is stateless — it has zero memory of any previous call, ever. When a chatbot "remembers" what you said three messages ago, the model didn't retain anything; your application re-sent those three messages back to it, bundled into this call's prompt. "Memory" is not a model feature — it's application-side state that you carry across turns and feed back in.

This creates a hard trade-off: every old message you resend costs real input tokens, on every single turn, and the context window is finite. Carry too little and the model "forgets" useful context; carry too much and you burn tokens (and money) re-sending stuff that barely matters. This whole chapter is techniques for managing that trade-off well — trimming, summarising, retrieving only what's relevant, and persisting state so a restart doesn't wipe it.

LangGraph splits memory into two clean buckets. Short-term memory = one conversation's state, held by a checkpointer and keyed by a thread_id. Same thread → the conversation continues; a new thread_id → a totally blank slate, even with the exact same graph. Long-term memory = facts about a user that should survive across every conversation ("the user's name is Fraz") — that's a separate object called the Store, not the checkpointer. Later topics in this chapter go deep on both; here, just get the map straight.

One more thing worth knowing up front: older LangChain tutorials (0.x) taught a whole langchain.memory module — ConversationBufferMemory, ConversationSummaryMemory, and friends. That entire module is deleted in the current 1.x line; importing it throws ModuleNotFoundError before your program even runs. The modern replacement for all of it is LangGraph's checkpointer + Store system, which is what this chapter teaches.

🌍 Real-world example: a customer-support chatbot answers "what did I just ask you?" correctly not because the model remembers — it's because your backend pulled this conversation's last few messages from a database/checkpointer and pasted them into the prompt before calling the model again.

💡 Stateless = each API call to the model is independent; it carries no memory of any earlier call.

💡 Short-term memory = one conversation's state, scoped to a thread_id, managed by a checkpointer.

💡 Long-term memory = facts about a user/world that outlive any single conversation, managed by a Store.

💡 Token budget = the finite space (measured in tokens) available per call; every resent message eats into it.

Standard definition: An LLM is stateless — it retains nothing between API calls — so "memory" is application-side state (prior messages, summaries, or stored facts) that your code resends or retrieves and feeds back into each new call; LangGraph implements this with a thread-scoped checkpointer for short-term memory and a separate Store for long-term memory, replacing the deleted 0.x langchain.memory module.

from typing import TypedDict
from langgraph.graph import StateGraph, START, END, MessagesState
from langgraph.checkpoint.memory import InMemorySaver
from langchain_core.messages import HumanMessage, AIMessage


def fake_model_node(state: MessagesState):
    # Stand-in for a real LLM call -- NO API key needed.
    # It just reports how many messages it can see, which is enough
    # to prove whether prior turns are present or not.
    msgs = state["messages"]
    reply = f"I can see {len(msgs)} message(s) in this conversation so far."
    return {"messages": [AIMessage(content=reply)]}


# 1) a BARE graph, no checkpointer -- every invoke starts from zero
bare_builder = StateGraph(MessagesState)
bare_builder.add_node("model", fake_model_node)
bare_builder.add_edge(START, "model")
bare_builder.add_edge("model", END)
bare = bare_builder.compile()  # NO checkpointer

r1 = bare.invoke({"messages": [HumanMessage(content="Hi, I am Fraz")]})
print("Bare call 1:", r1["messages"][-1].content)

r2 = bare.invoke({"messages": [HumanMessage(content="What is my name?")]})
print("Bare call 2:", r2["messages"][-1].content)

# 2) SAME kind of graph, but compiled with a checkpointer + a thread_id
mem_builder = StateGraph(MessagesState)
mem_builder.add_node("model", fake_model_node)
mem_builder.add_edge(START, "model")
mem_builder.add_edge("model", END)
remembering = mem_builder.compile(checkpointer=InMemorySaver())

config = {"configurable": {"thread_id": "user_123"}}
t1 = remembering.invoke({"messages": [HumanMessage(content="Hi, I am Fraz")]}, config)
print("Thread user_123, turn 1:", t1["messages"][-1].content)

t2 = remembering.invoke({"messages": [HumanMessage(content="What is my name?")]}, config)
print("Thread user_123, turn 2:", t2["messages"][-1].content)

# a DIFFERENT thread_id = a fresh conversation, no memory of user_123
new_config = {"configurable": {"thread_id": "user_999"}}
t3 = remembering.invoke({"messages": [HumanMessage(content="Hello")]}, new_config)
print("Thread user_999 (new thread), turn 1:", t3["messages"][-1].content)

# confirm the old memory module really is gone
try:
    from langchain.memory import ConversationBufferMemory
except ModuleNotFoundError as e:
    print("langchain.memory is gone:", e)

Picture your WhatsApp chat with a friend who has 3 years of history. When you open the chat, WhatsApp doesn't reload all 20,000 messages into view — it shows you the last 10-ish, and that's enough to follow what's happening right now. If someone brings up something from 2 years ago, you'd have to scroll back manually; the app doesn't carry it forward for you automatically.

Buffer memory is that exact idea applied to an LLM conversation: keep only the last N messages, and drop everything older. It is the simplest possible memory strategy — no summarising, no searching, just "keep the recent tail, forget the rest."

The manual pattern is a plain Python list capped at a size: append a new message, and if the list grows past max_messages, slice it back down (usually keeping a system prompt pinned at index 0, then the most recent messages). This is exactly what a chatbot does under the hood before it ever calls the model — it decides which messages actually get sent this turn.

In LangChain (1.x) there is a real tool that does this window-trimming for you: trim_messages from langchain_core.messages. It takes a list of messages, a token budget (max_tokens), a strategy ("last" = keep the most recent), and a token_counter (in production a real tokenizer; a placeholder like len for demos), and returns the trimmed list. Older LangChain tutorials taught a class called ConversationBufferWindowMemory for this — that class is DELETED in the current 1.x line; importing from langchain.memory import ConversationBufferWindowMemory raises ModuleNotFoundError before your program even starts. trim_messages is the modern, working replacement, and it is typically combined with a checkpointer (the thread-scoped memory mechanism) so the trimmed list is what actually gets resent to the model each turn.

🌍 Real-world example: a customer-support bot that only needs to know the last 2-3 things the customer said ("my order number is X", "it hasn't arrived") doesn't need the entire chat history re-sent every turn — a buffer of the last N messages is enough context to answer, and it keeps every API call cheap.

💡 Buffer memory = keep only the last N messages of a conversation, discard the rest — the simplest memory strategy.

💡 trim_messages = the 1.x langchain_core.messages function that trims a message list down to a token/message budget using a chosen strategy.

💡 token_counter = the function trim_messages uses to "count" each message's size; in production this is a real tokenizer, not a stand-in like len.

Standard definition: Buffer memory keeps only the most recent N messages of a conversation and drops older ones to stay within the token budget; in current LangChain (1.x) the working tool for this is trim_messages(messages, max_tokens=N, strategy="last", token_counter=...) — the older ConversationBufferWindowMemory class is deleted.

# --- 1. The manual last-N pattern (plain Python, no library needed) ---
class BufferMemory:
    def __init__(self, max_messages=4):
        self.messages = []
        self.max_messages = max_messages

    def add(self, role, content):
        self.messages.append({"role": role, "content": content})
        if len(self.messages) > self.max_messages:
            # keep the system prompt (index 0) + the most recent (max_messages - 1)
            self.messages = [self.messages[0]] + self.messages[-(self.max_messages - 1):]

    def get_messages(self):
        return self.messages

memory = BufferMemory(max_messages=4)
memory.add("system", "You are a helpful assistant.")
memory.add("user", "Hi, I am Fraz")
memory.add("assistant", "Hello Fraz!")
memory.add("user", "What is 2+2?")
memory.add("assistant", "4")
memory.add("user", "What is my name?")
for m in memory.get_messages():
    print(m)

# --- 2. The real 1.x tool: trim_messages ---
from langchain_core.messages import trim_messages, HumanMessage

msgs = [HumanMessage(content=f"m{i}") for i in range(1, 6)]
trimmed = trim_messages(msgs, max_tokens=3, strategy="last", token_counter=len)
print([m.content for m in trimmed])

# --- 3. Proof the old 0.x class is gone ---
try:
    from langchain.memory import ConversationBufferWindowMemory
except ModuleNotFoundError as e:
    print("dead:", e)

What we're building: a chatbot that remembers you — a LangGraph graph with TWO separate memories wired in at once: an InMemorySaver checkpointer that carries THIS session's messages (short-term, keyed by thread_id), and an InMemoryStore fact that greets you by name even in a BRAND-NEW session (long-term, keyed by user_id). This is the chapter's whole mental model — checkpointer vs Store — fused into one runnable project you can put on a resume.

---

Step 0 — The mental model (read first)

Turn arrives with (thread_id, user_id)
        |
        v
[chatbot node] -- reads state["messages"] (checkpointer: THIS thread only)
        |
        +-- reads store.get(("users", user_id), "profile") (Store: ACROSS all threads)
        |
        +-- if message says "I am <Name>" -> store.put(...) saves the fact FOREVER
        |
        v
   reply: "Hi <Name>!" if the Store has a name, else "Hi stranger!"

Two DIFFERENT storage systems, two DIFFERENT lifetimes: the checkpointer's messages list resets to empty the moment thread_id changes. The Store's profile fact does NOT reset when thread_id changes — only when user_id changes. That contrast is the entire chapter, made concrete.

💡 Checkpointer = short-term memory, one conversation's message history, keyed by thread_id. 💡 Store = long-term memory, facts about a user that outlive any single conversation, keyed by user_id (namespaced).

---

Step 1 — Define the state

from typing import Annotated
from typing_extensions import TypedDict
from langgraph.graph.message import add_messages

class ChatState(TypedDict):
    messages: Annotated[list, add_messages]
    user_id: str

What's happening: messages is tagged with the add_messages reducer, so every node's returned message list is APPENDED to history, never overwrites it — the exact rule from checkpointers. user_id rides along in the state on every turn so the node knows WHICH user's long-term facts to look up in the Store; it is not touched by the checkpointer at all, it is just a plain field.

---

Step 2 — Create the Store (long-term memory)

from langgraph.store.memory import InMemoryStore

store = InMemoryStore()

What's happening: InMemoryStore is a separate object from the checkpointer, created once and closed over by the node function below. It is namespaced key-value storage: store.put(namespace_tuple, key, value_dict) writes, store.get(namespace_tuple, key) reads back an object whose .value is that dict. We will namespace by ("users", user_id) so each user's facts are isolated from every other user's, exactly the pattern from long-term-memory-store.

---

Step 3 — The chatbot node (reads BOTH memories)

def chatbot(state: ChatState):
    last_human = state["messages"][-1].content
    user_id = state["user_id"]
    namespace = ("users", user_id)
    remembered = store.get(namespace, "profile")

    # extract a name the first time we see "I am <Name>" (pure Python, no LLM needed to run this)
    if "i am" in last_human.lower():
        name = last_human.lower().split("i am")[-1].strip().split()[0].capitalize()
        store.put(namespace, "profile", {"name": name})
        remembered = store.get(namespace, "profile")

    if remembered:
        name = remembered.value["name"]
        reply = f"Hi {name}! (this session has {len(state['messages'])} messages so far)"
    else:
        reply = f"Hi stranger! (this session has {len(state['messages'])} messages so far)"

    from langchain_core.messages import AIMessage
    return {"messages": [AIMessage(content=reply)]}

What's happening: state["messages"] is the CHECKPOINTER's job — LangGraph already restored this thread's prior turns into state before calling the node, so len(state["messages"]) genuinely grows turn over turn WITHIN one thread_id. store.get(...) / store.put(...) are the STORE's job — they read/write a fact keyed by user_id, completely independent of which thread this turn is on. A real build would swap the "i am" in last_human.lower() extraction for an LLM call that pulls structured facts out of free text (mark that illustrative); the store/namespace WIRING underneath does not change either way — this exact pure-Python version is what we run and verify below.

---

Step 4 — Wire the graph with a checkpointer

from langgraph.graph import StateGraph, START, END
from langgraph.checkpoint.memory import InMemorySaver

graph = StateGraph(ChatState)
graph.add_node("chatbot", chatbot)
graph.add_edge(START, "chatbot")
graph.add_edge("chatbot", END)

checkpointer = InMemorySaver()
app = graph.compile(checkpointer=checkpointer)

What's happening: passing checkpointer=InMemorySaver() to .compile() is the ONE line that turns a plain graph into a memory-carrying one — every .invoke() call against app will, given a thread_id in its config, load that thread's prior messages before running the node and save the updated list after. No thread_id in config means no persistence at all. Verified on C:/lcv: this exact graph compiles with zero API keys and zero network calls — nothing in this project needs a key to construct or invoke, which is the whole point of a pure-Python node.

---

Step 5 — Turn 1 and Turn 2, SAME thread (proves the checkpointer)

from langchain_core.messages import HumanMessage

config1 = {"configurable": {"thread_id": "abc123"}}
r1 = app.invoke({"messages": [HumanMessage(content="Hi, I am Fraz")], "user_id": "user_1"}, config1)
r2 = app.invoke({"messages": [HumanMessage(content="What's the weather like?")], "user_id": "user_1"}, config1)

This ACTUALLY RAN on C:/lcv — zero API keys — and produced this real state, turn by turn:

--- Turn 1 (thread abc123) ---
HumanMessage | Hi, I am Fraz
AIMessage    | Hi Fraz! (this session has 1 messages so far)
msg count: 2

--- Turn 2 (SAME thread abc123) ---
HumanMessage | Hi, I am Fraz
AIMessage    | Hi Fraz! (this session has 1 messages so far)
HumanMessage | What's the weather like?
AIMessage    | Hi Fraz! (this session has 3 messages so far)
msg count: 4

Trace it beat by beat: Turn 1 sees ONLY its own HumanMessage (checkpointer had nothing yet for abc123), extracts "Fraz" from it, writes the fact to the Store, and replies. Turn 2 calls .invoke() with a NEW HumanMessage and the SAME config1 — the checkpointer restores turn 1's two messages FIRST, add_messages appends the new turn's two messages on top, so the node sees len(state["messages"]) == 3 (the fresh count right before its own reply is appended) and the final list has 4. This is short-term memory, genuinely carried, with real numbers.

---

Step 6 — A brand-new session, SAME user (proves the Store)

config2 = {"configurable": {"thread_id": "xyz789"}}
r3 = app.invoke({"messages": [HumanMessage(content="Hello again")], "user_id": "user_1"}, config2)

Also ran for real:

--- New thread xyz789 (new session, same user_1) ---
HumanMessage | Hello again
AIMessage    | Hi Fraz! (this session has 1 messages so far)
msg count: 2

This is the whole chapter in one line of output. thread_id changed from "abc123" to "xyz789", so the CHECKPOINTER has nothing — state["messages"] starts fresh at 1 message, exactly like a brand-new conversation, with zero memory of turn 1 or turn 2 above. But user_id is still "user_1", so the STORE lookup on ("users", "user_1") finds the same {"name": "Fraz"} fact written in Step 5 and the bot greets Fraz by name anyway. Short-term memory reset to zero; long-term memory did not.

---

Step 7 — A different user (proves isolation)

config3 = {"configurable": {"thread_id": "thread_new_user"}}
r4 = app.invoke({"messages": [HumanMessage(content="Hey")], "user_id": "user_999"}, config3)

--- New thread + new user_999 ---
HumanMessage | Hey
AIMessage    | Hi stranger! (this session has 1 messages so far)

What this proves: ("users", "user_999") has never had anything written to it, so store.get(...) returns None and the node falls back to "Hi stranger!". The Store namespace really does isolate one user's facts from another's — this is not a global variable, it is per-user storage.

---

Step 8 — Where a real LLM would slot in (illustrative)

Two spots in this project are stubbed with plain Python so the WIRING runs with zero API keys; a production build swaps them for real model calls without touching the graph:

1. Fact extraction — "i am" in last_human.lower() is a toy string match. A real build sends the message to an LLM with a small extraction prompt ("pull out the user's name if mentioned") and stores whatever structured fact comes back. Mark this illustrative — it needs a key.
2. The reply itself — f"Hi {name}! ..." is an f-string. A real build passes state["messages"] (plus the recalled fact injected into a system message) to a chat model and returns its AIMessage instead of a hand-built string.

Neither swap changes a single line of Step 4's graph wiring, Step 5's checkpointer behaviour, or Step 6/7's Store behaviour — those are ALL genuinely verified above with no key. Only the WORDING the model would produce is illustrative:

AI (illustrative, real LLM in the node): "Welcome back, Fraz! Last time you asked about
the weather -- anything else I can help with today?"

---

✅ What you just built — and resume framing

You built a chatbot with two INDEPENDENT, correctly-scoped memories: a checkpointer (InMemorySaver, keyed by thread_id) that genuinely carries a conversation's messages turn to turn, and a Store (InMemoryStore, namespaced ("users", user_id)) that genuinely survives a brand-new thread_id and still recalls a fact about the user. You proved BOTH with real .invoke() calls and real message counts, not a mocked example — same thread grew from 2 to 4 messages; a new thread reset to 1 message but still greeted the user by name; a new user got the stranger fallback. That is the exact short-term/long-term split every LangGraph memory question is really asking about.

For your resume/interview: "Built a memory-aware chatbot in LangGraph — an InMemorySaver checkpointer scoped to thread_id for per-conversation short-term memory, plus an InMemoryStore namespaced by user_id for long-term facts that persist across sessions, with add_messages correctly appending state instead of overwriting it." That sentence shows you understand the checkpointer-vs-Store distinction, not just that you called .invoke().

Bonus features to extend this (do NOT need to build all three to claim the project):

1. Real fact extraction — replace the "i am" string match with an LLM call that extracts multiple facts (name, role, preferences) from free text into the Store.
2. Durable persistence — swap InMemorySaver/InMemoryStore for a SqliteSaver-backed setup (separate package, langgraph-checkpoint-sqlite) so both memories survive a process restart, not just RAM.
3. Trim the checkpointer's history — once a thread grows long, run trim_messages on state["messages"] before sending it to a real model, so the checkpointer keeps everything but the model only sees a bounded window (see buffer-memory).

Standard definition: a memory-aware LangGraph chatbot combines two distinct persistence layers — a checkpointer (short-term, keyed by thread_id, carries one conversation's message history and resets on a new thread) and a Store (long-term, keyed/namespaced by user_id, holds facts that survive across every thread) — so the same application can both remember the last few turns of THIS conversation and recognise a returning user in a conversation it has never seen before.

from typing import Annotated
from typing_extensions import TypedDict
from langgraph.graph import StateGraph, START, END
from langgraph.graph.message import add_messages
from langgraph.checkpoint.memory import InMemorySaver
from langgraph.store.memory import InMemoryStore
from langchain_core.messages import HumanMessage, AIMessage

class ChatState(TypedDict):
    messages: Annotated[list, add_messages]  # reducer: APPENDS, never overwrites
    user_id: str

store = InMemoryStore()  # long-term memory: facts survive across every thread_id

def chatbot(state: ChatState):
    last_human = state["messages"][-1].content
    user_id = state["user_id"]
    namespace = ("users", user_id)
    remembered = store.get(namespace, "profile")

    if "i am" in last_human.lower():
        name = last_human.lower().split("i am")[-1].strip().split()[0].capitalize()
        store.put(namespace, "profile", {"name": name})
        remembered = store.get(namespace, "profile")

    if remembered:
        name = remembered.value["name"]
        reply = f"Hi {name}! (this session has {len(state['messages'])} messages so far)"
    else:
        reply = f"Hi stranger! (this session has {len(state['messages'])} messages so far)"

    return {"messages": [AIMessage(content=reply)]}

graph = StateGraph(ChatState)
graph.add_node("chatbot", chatbot)
graph.add_edge(START, "chatbot")
graph.add_edge("chatbot", END)

checkpointer = InMemorySaver()  # short-term memory: carries THIS thread_id's messages
app = graph.compile(checkpointer=checkpointer)

config1 = {"configurable": {"thread_id": "abc123"}}
r1 = app.invoke({"messages": [HumanMessage(content="Hi, I am Fraz")], "user_id": "user_1"}, config1)
r2 = app.invoke({"messages": [HumanMessage(content="What's the weather like?")], "user_id": "user_1"}, config1)

config2 = {"configurable": {"thread_id": "xyz789"}}  # NEW thread, SAME user
r3 = app.invoke({"messages": [HumanMessage(content="Hello again")], "user_id": "user_1"}, config2)

config3 = {"configurable": {"thread_id": "thread_new_user"}}  # NEW thread, NEW user
r4 = app.invoke({"messages": [HumanMessage(content="Hey")], "user_id": "user_999"}, config3)

for label, result in [("Turn 1", r1), ("Turn 2 (same thread)", r2), ("New thread, same user", r3), ("New thread, new user", r4)]:
    print(f"--- {label} ---")
    for m in result["messages"]:
        print(type(m).__name__, "|", m.content)