All jobs / AI engineer

Embeddings & Semantic Search interview questions

Embeddings & Semantic Search questions a AI engineer can be asked, from first job to senior. Open one to see the strongest answer and why it works.

1. A user types “how do I get my money back” and semantic search finds the page titled “Refund policy”. Why does that work, and where does it not?

   The strongest answer: Both texts are turned into vectors by an embedding model, and similar meanings land close together, so they match without sharing words. It is weaker on exact tokens — product codes, ids, rare names

   Why it works: Exactly — nearness in vector space stands in for nearness in meaning. Knowing where it fails, on exact strings, is what separates using it from trusting it.

   • Meaning becomes distance between vectors.
   • Matches without shared words.
   • Weak on exact codes, ids, names.
   • Keep keyword search for exact matches.

2. What does cosine similarity measure between two embeddings, and what does a score of 0.8 tell you?

   The strongest answer: The angle between the vectors — direction, not length. By itself 0.8 tells you little: scores only compare within one model, so I use them to rank and tune any cut-off on our own data

   Why it works: Right — cosine compares direction, and what counts as “close” differs from model to model. Rank by it; never read it as an absolute grade.

   • Cosine is angle, not length.
   • Scores rank; they are not grades.
   • Comparable only within one model.
   • Tune thresholds on your own data.

3. Users search “Acme invoices from last quarter” and get similar-looking invoices from other customers and other years. What do you change?

   The strongest answer: Customer and date are facts, not meaning. I store them as metadata on each vector and filter on them alongside the similarity search — and anything that is access control gets enforced server-side, never left to similarity

   Why it works: Exactly — similarity finds “things like this”; a filter finds “only these”. Exact constraints belong in metadata, and who may see what is never a ranking question.

   • Similarity ranks; filters constrain.
   • Store customer, date, type as metadata.
   • Filter in the search, not afterwards.
   • Access rules are enforced server-side.

4. Your embedding model can output vectors at several sizes. How do you decide which dimension to use?

   The strongest answer: More dimensions can hold finer distinctions but cost memory, index size and query time on every vector. I measure retrieval quality on our labelled queries at each size and take the smallest that holds up

   Why it works: Exactly — it is a cost-and-quality trade you can measure. The cost applies to every vector stored and compared, so corpus size decides how much it matters.

   • Dimensions trade quality against cost.
   • Cost applies to every stored vector.
   • Measure on your own labelled queries.
   • Take the smallest size that holds up.

5. Vector databases mostly use approximate nearest-neighbour indexes. What are you giving up, and when does it bite?

   The strongest answer: Exactness for speed: an index such as HNSW or IVF returns most of the true neighbours, not all. It bites under selective filters and when nobody measured recall, so I compare with exact search on a sample

   Why it works: Exactly — recall is a dial traded against latency and memory. A strict filter on top of an approximate index can return too few or the wrong neighbours, and only a comparison with exact search shows it.

   • Approximate search trades recall for speed.
   • Recall is tunable against latency and memory.
   • Selective filters are where it bites.
   • Measure against exact search on a sample.

6. The team is starting a retrieval feature over a few hundred thousand documents. The data already lives in PostgreSQL. One engineer wants a dedicated vector database; another wants pgvector.

   How do you make the call?

   The strongest answer: From requirements: scale, latency, filtering, update rate, and who operates it. At this size I would start in the database we already run, keep retrieval behind an interface, and name the measurements that would trigger a move

   Why it works: Right — it is an operations and requirements decision before it is a technology one. One fewer system to run, joins with your own data and a planned exit beat a guess about future scale.

   • Requirements first, product second.
   • Count the cost of a second datastore.
   • Keep retrieval behind an interface.
   • Name the numbers that trigger a move.