Metrics interview questions
Metrics questions a Product manager can be asked, from first job to senior. Open one to see the strongest answer and why it works.
What is a product metric allowed to be?
The strongest answer: A behaviour we can observe and a decision we will make — not a slide of invented lift
Why it works: Exactly. Observe, then decide. Promised 20% and industry percentiles are fiction.
- Metric = observable behaviour + a decision.
- No promised lift %.
- No always-up vanity.
- No invented industry percentile.
Two reports give different churn numbers for the same month. How can that happen?
The strongest answer: Churn needs a definition before it is a number: customers or revenue, which window, and when someone counts as gone — cancelled, expired, or simply stopped using it
Why it works: Exactly — customer churn and revenue churn can move in opposite directions when one large account leaves. Put the definition next to the number.
- Define churn before you quote it.
- Customers or revenue?
- Name the window.
- Say what “gone” means.
A deck has “this will lift retention 18%.” You have no study. What do you do?
The strongest answer: Take the number off — say what we will learn and what we will watch, I do not keep a decorative 18
Why it works: Exactly. No study, no number. Precision theatre and fake industry are worse.
- Remove invented lift.
- Say what you will watch.
- No fake industry source.
- Leadership likes ≠ evidence.
A slide says we are “top quartile for onboarding.” You have no benchmark. What do you do?
The strongest answer: Delete the quartile — say what we measured on our users, I do not keep a costume ranking
Why it works: Exactly. No benchmark, no quartile. Deciles and sourceless industry are worse costumes.
- Delete unverified quartiles.
- Report our measurement.
- Do not tighten a fake rank.
- No sourceless “industry.”
You ship a setup wizard. What do you refuse to call the success metric?
The strongest answer: Page views on the wizard — I want completed setup that leads to the job, not tourism
Why it works: Exactly. Completed job, not tourism. Pre-written lifts and percentiles are fiction.
- Measure completed job, not views.
- Tourism ≠ success.
- No pre-written lift.
- No industry percentile.
After a search redesign, average time in the app is up and sessions per user are up. Someone proposes putting it in the company update.
What do you check first?
The strongest answer: Whether people are finding things faster or slower — for search, more time and more sessions can mean failure. I look at successful searches, repeat searches, and what happens next
Why it works: Exactly — engagement is only good when it is time spent getting the job done. For a search box, more time is a warning.
- Engagement is a proxy, not value.
- Ask what the job is.
- For search, faster beats longer.
- Check outcomes, not minutes.
You watch a leading indicator. What do you still need?
The strongest answer: The lagging behaviour the bet was about — I do not declare victory on clicks alone
Why it works: Exactly. Leading is a hint. Faster-only and invented lagging % are costumes.
- Keep the lagging job in view.
- Leading ≠ victory.
- Do not pick an always-up leading.
- Do not convert clicks into a fake %.
Signup to first project runs across four steps. Most of the people who fall out do so at step three, where the product asks to connect their calendar.
What do you do with that?
The strongest answer: Treat it as a question, not a cause: watch recorded sessions at that step, see whether the drop differs by device and source, and test whether the ask can be optional or later
Why it works: Right — the biggest drop tells you where to look, not what to fix. Some of those people were never going to convert.
- The drop shows where to look.
- Segment before you conclude.
- Watch real sessions.
- Ask if the step belongs here.
Teams start shipping dark patterns to move the north star. What did you miss?
The strongest answer: A guardrail on the harm, and a review of how the number is moved — I do not “just raise the target”
Why it works: Exactly. How it moves is the product. Higher, hidden, and secret percentiles are more game.
- Add harm guardrails.
- Review how the number moves.
- Do not only raise the target.
- No secret percentile.
You are shown retention curves for six monthly cohorts. Each drops steeply in the first week and then keeps sloping down; the newest cohorts start higher than the old ones.
What do you read from that?
The strongest answer: A curve that never flattens means we are not keeping a base — growth is refilling a leaky bucket. The higher starts say onboarding or targeting improved; the slope says the product has not
Why it works: Right — the question a retention curve answers is whether it flattens. A higher start with the same slope buys time, not a base.
- Read the slope, not the start.
- Ask whether it flattens.
- Compare cohorts; never average them.
- No flat tail, no base.
When is a north star vanity?
The strongest answer: When teams can move it without the user getting the job done — I change the star or add a guardrail
Why it works: Exactly. If it can be gamed away from the job, it is vanity. Sacred and averaged vanities are still vanities.
- Vanity = movable without the job.
- Change it or guardrail it.
- Not sacred.
- Do not average two vanities.
One large customer signs a three-year contract and pays the first year up front. The monthly revenue chart jumps, and an exec wants to present it as the new run rate.
How do you report it?
The strongest answer: Separate what was booked, what was billed and what recurs: normalise the contract to a monthly figure, show it as one named account, and say plainly that the run rate did not move that far
Why it works: Right — bookings, billings and recurring revenue answer three different questions, and concentration in one account is the part leadership needs to see.
- Bookings, billings and recurring differ.
- Normalise prepayments to months.
- Show account concentration.
- Fix the chart, not the footnote.
Three teams report three “the” activation numbers. What do you lock?
The strongest answer: The definition, the window, and who may change them — I do not lock the chart colour
Why it works: Exactly. Definition and change control. Themes, query-bottlenecks, and novels are not a lock.
- Lock definition, window, changer.
- Let chart taste differ.
- Do not approve every query.
- Do not write a metrics novel.
The board wants one number for “is product working?” What do you give?
The strongest answer: The live behaviour of the current bet, plus a leftover sentence — I do not invent a product-health %
Why it works: Exactly. Current bet and leftover. Forty plots, ticket counts, and awards are games.
- Live behaviour of the current bet.
- One leftover sentence.
- No invented health %.
- No award as the answer.
A staff review still has lift % with no study. What do you do in the room?
The strongest answer: Take the number off the slide in front of people — so the next room does not bring another
Why it works: Exactly. Remove it in the room. Internal-only, decimals, and “directional” are how fiction survives.
- Remove invented lift in the room.
- Internal ≠ allowed.
- Decimals ≠ care.
- “Directional” does not save an 18.