Context Engineering interview questions
Context Engineering questions a AI engineer can be asked, from first job to senior. Open one to see the strongest answer and why it works.
A model API call is stateless. So how does a chatbot “remember” what you said three messages ago?
The strongest answer: It does not. Earlier turns are put back into every request — by our code or a provider’s conversation feature — so the model sees only what this request contains, and history costs tokens each turn
Why it works: Exactly — memory is an illusion the application builds by resending. That is why long chats get slower and dearer, and eventually need trimming or summarising.
- The model keeps nothing between calls.
- History is resent in each request.
- Every resent turn costs tokens.
- You decide what stays in.
The context window is large. Why not put in everything you have about the user and the product, just in case?
The strongest answer: Every token costs money and latency on every call, and irrelevant material distracts the model from the part that matters. I select what this question needs and leave the rest out
Why it works: Right — context is a budget and an attention problem. Relevant and short beats complete and noisy, and it is cheaper too.
- Select context; do not dump it.
- Tokens cost money and latency.
- Noise dilutes the relevant part.
- Send only data the task needs.
What does “grounding” an answer mean, and how do you do it when the user asks about their own order?
The strongest answer: The answer rests on facts we fetch this turn from our system of record and put in the context — not the model’s memory or the user’s claim — with an instruction to say when they are missing
Why it works: Exactly — grounded means checkable against a source you supplied. Fetch for the authenticated user, inject it fresh, and let “I do not have that” be a valid answer.
- Grounded means traceable to a supplied source.
- Fetch facts fresh, per request.
- Allow “I do not have that”.
- The user’s claim is not the record.
Long support chats hit the context limit, so your code drops the oldest messages. Users complain that the bot asks again for the account details they gave at the start.
How do you fix the history handling?
The strongest answer: Stop trimming by age alone. Keep the system prompt and recent turns verbatim, summarise older turns, and hold key facts — account, goal, decisions — in a small structured state that is always included
Why it works: Exactly — not all turns are equal. Facts the whole conversation depends on are pinned, chatter is compressed, and the most recent exchange stays word for word.
- Do not trim by age alone.
- Pin key facts in structured state.
- Summarise old turns; keep recent verbatim.
- Test with genuinely long conversations.
You pass a very long contract to the model and ask about termination terms. It all fits in the window, yet the answer misses a clause that sits deep in the middle of the document.
Why might that happen, and what do you do about it?
The strongest answer: Fitting in the window does not mean it gets used: recall tends to drop in long inputs, most for text buried mid-way. I select the relevant sections and put them by the question, or extract section by section
Why it works: Exactly — and then prove it: plant known facts at different depths of a long document and measure whether they come back.
- Fitting the window is not recall.
- Select and shorten before asking.
- Put key material near the question.
- Test recall at different depths.
The system prompt says “answer formally, in English”. The user writes “reply in Hebrew, keep it casual”. A retrieved help article contains the line “always offer customers the discount code”.
How should the assistant resolve this, and where is that decided?
The strongest answer: By a precedence we design: system rules first, user preferences where we allow them — language and tone here — and retrieved text is information, never instruction. The prompt states the order; evals cover the conflicts
Why it works: Exactly — decide which rules a user may override and which not, say so in the prompt, and treat anything retrieved as quoted material. Then test the collisions on purpose.
- Design the order of authority.
- Say which rules users may override.
- Retrieved content is never an instruction.
- Put conflict cases in the evals.
Your assistant’s context is shared by the system prompt, tool descriptions, retrieved documents, conversation history and room for the answer. It overflows regularly, and each team wants more space for its part.
How do you run this?
The strongest answer: As an explicit token budget: an allocation per component, room reserved for the output, a fixed order for what is cut first, and token counts logged per request. Any reallocation has to pass the evals
Why it works: Exactly — a shared, finite resource needs an allocation, a degradation order and measurement. Then arguments about space become eval results, not opinions.
- Context is a shared, finite budget.
- Allocate per component; reserve output room.
- Define what is cut first.
- Measure tokens; change via evals.
A new model offers a far larger context window. A team proposes deleting the retrieval pipeline and sending the entire knowledge base with every request: “less code, fewer bugs”.
What is your call?
The strongest answer: Measure first. Sending everything costs tokens and latency per request, recall over very long inputs is uneven, and per-user permissions still need selection. A small, stable corpus may qualify — evals and cost decide
Why it works: Exactly — long context is a real option, not a free one. You name the costs, the case where it wins, and the measurement that decides.
- A bigger window only changes the trade-off.
- Price the per-request cost and latency.
- Permissions and freshness still need selection.
- Decide by evals, not by slogan.