All jobs / Data science / ML / AI

Mathematics interview questions

Mathematics questions a Data science / ML / AI can be asked, from first job to senior. Open one to see the strongest answer and why it works.

  1. What does gradient descent actually do when a model “trains”?

    The strongest answer: It nudges the parameters a small step in the direction that lowers the loss, using the gradient as the slope, and repeats until the loss stops improving

    Why it works: Exactly — the gradient says which way is uphill, so you step the other way; the learning rate sets how far.

    • Step downhill on the loss.
    • Gradient: the slope of the loss.
    • Move parameters against it, a little.
    • Repeat until the loss levels off.

  2. Two items are represented as vectors. What does their dot product tell you, and why do people normalise before using it as a similarity?

    The strongest answer: It sums elementwise products, so it is large when vectors point the same way; length inflates it, so dividing by both norms gives the cosine, which measures direction only

    Why it works: Right — the dot product mixes angle and length; cosine keeps the angle. That is why embeddings are compared with cosine or unit-normalised first.

    • Dot product: alignment times length.
    • Large when directions agree.
    • Normalise to remove length.
    • Cosine compares direction only.

  3. Two fraud signals each fire on about 10% of transactions. A teammate says both fire together on 1%. What is being assumed, and is it safe?

    The strongest answer: Independence — multiplying probabilities only works when one signal tells you nothing about the other, and fraud signals are usually correlated, so the true joint rate must be measured, not derived

    Why it works: Exactly — P(A and B) = P(A)·P(B) only under independence. Correlated signals fire together far more often than the product suggests.

    • Multiply only when independent.
    • Ask: does A change P(B)?
    • Correlated signals fire together.
    • Measure the joint rate.

  4. Your training loss goes down for a few steps, then shoots up to NaN. What is the most likely cause, and what do you try first?

    The strongest answer: The learning rate is too large — steps overshoot the minimum and the updates blow up; lower it, or add gradient clipping and warm-up, and confirm the loss then descends smoothly

    Why it works: Right — a divergent loss is the signature of an overshooting step size. Clipping and warm-up are the standard guards.

    • Exploding loss: step too large.
    • Lower the learning rate first.
    • Clip gradients; warm up.
    • Confirm a smooth descent.

  5. A feature matrix X has shape (n, d) and a weight matrix W has shape (d, k). What does X·W give you, and why does the shape check matter?

    The strongest answer: An (n, k) matrix: one row per sample, one column per output, each entry a weighted sum of that sample’s features — and the shared d must match, which is the first bug a shape error catches

    Why it works: Exactly — rows are samples, columns are outputs, and matching the inner dimension is the contract. This is a linear layer.

    • Outer dims stay; inner dim sums.
    • (n, d)·(d, k) gives (n, k).
    • Rows: samples. Columns: outputs.
    • Shape errors are your friend.

  6. Logistic regression has a convex loss; a neural network does not. What does that difference change in practice?

    The strongest answer: Convex means one global minimum, so any sensible optimiser lands on the same solution — with a neural net, different seeds and learning rates reach different minima, so you compare runs, not a single fit

    Why it works: Right — convexity buys reproducibility and a meaningful “the” solution; non-convex training makes seeds and schedules part of the experiment.

    • Convex: one minimum, reproducible.
    • Non-convex: seeds and schedules matter.
    • Compare several runs.
    • Report variance, not a single fit.