AI Memory interview questions
AI Memory questions a AI engineer can be asked, from first job to senior. Open one to see the strongest answer and why it works.
What is the difference between conversation memory and user memory in an assistant?
The strongest answer: Conversation memory is this session’s turns, resent to the model as context; user memory is facts the app stores across sessions and retrieves when relevant. The application builds both — the model keeps nothing
Why it works: Right — the model is stateless between calls. Short-term memory is what you resend; long-term memory is a store you design, write to and retrieve from.
- The app builds memory, not the model.
- Conversation memory: this session, resent as context.
- User memory: stored facts across sessions.
- Select what is kept; retrieve what matters.
Why would you give a stored memory an expiry date or a “last confirmed” date?
The strongest answer: Facts go stale — a job search ends, a city changes — and stale memory gives confident wrong answers. A date lets you expire, down-rank or re-confirm it, and not hold personal data longer than needed
Why it works: Exactly — staleness is a correctness problem and a privacy problem at once. Different kinds of memory deserve different lifetimes.
- Stale memory gives confident wrong answers.
- Date every memory when it is written.
- Set lifetimes by the kind of fact.
- Expire, down-rank or re-confirm.
How do you decide what gets written to a user’s long-term memory, and when?
The strongest answer: Only durable, useful facts and preferences the user stated, saved as short dated statements with their source, extracted after the turn, not in the reply path. Never every message, and no sensitive details by accident
Why it works: Right — memory is a write policy before it is a store. Durable, stated, dated, sourced; and the write happens off the latency path.
- Memory needs a write policy.
- Keep durable, stated facts and preferences.
- Record source and date; write asynchronously.
- Filter sensitive data before it lands.
When do you store a memory as a structured field instead of as an embedded text snippet?
The strongest answer: When the fact has one current value that must be exact — name, language, plan, time zone. Fields can be overwritten and read deterministically; vector memory suits open-ended recollections where fuzzy recall is acceptable
Why it works: Exactly — overwrite semantics and exact reads are what fields give you. Similarity search returns “something related”, which is wrong for a user’s name.
- Exact, single-valued facts go in fields.
- Fields overwrite; vectors accumulate.
- Vectors suit fuzzy, open-ended recall.
- Most products need both.
Your assistant retrieves the top five memories by similarity on every message, and users say it drags an old project into unrelated chats. What do you change?
The strongest answer: Top-k always returns k, relevant or not. Add a similarity threshold so “nothing” is a valid result, weight by recency and importance, scope by topic or type, and test cases where no memory should surface
Why it works: Right — retrieval needs a way to return nothing. Threshold, recency and scope, then an eval set that includes the “no memory applies” case.
- Retrieval must be able to return nothing.
- Add a relevance threshold.
- Weight recency and importance; scope by type.
- Test the no-memory case.
In March a user told the assistant they work in Berlin. In June they mention having moved to Lisbon. Both statements are in memory, and answers now mix the two.
How should memory writes handle this?
The strongest answer: Make the write an update, not an append: look up related memories first, let the newer statement supersede the older with timestamps and provenance kept, and ask the user when the conflict is unclear
Why it works: Exactly — contradiction is resolved at write time. Supersede rather than delete, so you can explain and undo, and confirm with the user when in doubt.
- Resolve contradictions at write time.
- Look up related memories before writing.
- Supersede with dates; keep provenance.
- Ask the user when unsure.
Product wants the assistant to “remember everything about the user” so it feels personal. You own the memory design — what is your position?
The strongest answer: Not everything: named categories, visible to the user with edit and delete, sensitive topics off unless the user asks, a retention period, and deletion that reaches the index, caches and logs
Why it works: Right — memory that users can see and control is the kind they trust. Scope, visibility, sensitive-data rules, retention and real deletion are the design, not extras.
- Remember on purpose, not everything.
- Users can see, edit and delete.
- Sensitive topics off by default.
- Deletion reaches every copy.
Your assistant reads web pages and emails for the user, and writes long-term memories from whatever passes through the conversation.
What risk do you raise in the design review, and what do you require?
The strongest answer: Memory poisoning: a page or email can plant false facts or instructions that persist across sessions. Write only from the user’s own statements, record provenance, treat recalled memory as data, and make it reviewable and purgeable
Why it works: Exactly — memory turns a one-off injection into a permanent one. Provenance and a “user statements only” write rule are the controls; recalled memory is never an instruction.
- Memory makes an injection permanent.
- Write only from the user’s statements.
- Store provenance with every memory.
- Recalled memory is data, never instructions.