All jobs / Automation developer
Framework Design interview questions
Framework Design questions a Automation developer can be asked, from first job to senior. Open one to see the strongest answer and why it works.
Someone opens the test report at eight in the morning. What questions does it have to answer?
The strongest answer: Which tests failed and on which assertion, which build and environment it ran against, how long it took, and whether each failure is new — with the evidence attached to the failing test
Why it works: Exactly — the report exists so nobody has to rerun anything to find the cause.
- The report replaces a rerun.
- Name the test and the assertion.
- State the build and environment.
- Attach evidence to the failure.
How do you get a user into “has items in cart” for a test?
The strongest answer: Create that state through an API, a factory, or a seed I own — I do not click the whole catalogue every run to reach the page I care about
Why it works: Exactly. Setup is data work. The UI test starts at the risk.
- Seed state cheaply.
- Start the UI at the risk.
- Do not share one row.
- Never use prod accounts.
A spec file contains a staging password. What do you do?
The strongest answer: Rotate it, pull it from the secret store or CI variables, and treat the commit as a leak — I do not leave it “because staging is not prod”
Why it works: Exactly. Staging creds open staging data. Rotate and store them properly.
- Rotate the leaked secret.
- Load secrets from CI / a vault.
- Staging is not “fake enough.”
- Do not encode and call it safe.
What is Given / When / Then for in an automation suite?
The strongest answer: A way to say the story out loud so a reader knows the setup, the action, and the proof — not a requirement to buy a BDD tool
Why it works: Exactly. The words are a communication habit. The tool is optional.
- Use the story shape.
- BDD tools are optional.
- Do not hide a mess in English.
- Still talk to the team.
Where do base URLs, users, and feature flags for tests live?
The strongest answer: In config or CI variables per environment — not hardcoded staging URLs in the spec, and not a laptop-only .env nobody else has
Why it works: Exactly. Config is how the same pack hits local, staging, and a preview app.
- Config per environment.
- No URLs baked into specs.
- Share the config shape, not a laptop file.
- CI is already a second env.
A teammate wants the suite’s test data in a spreadsheet so “anyone can add a case”, starting with 400 rows for the checkout journey.
What do you push back on?
The strongest answer: Data-driven is one journey with genuinely different inputs — one row per rule, each named so a failure says which rule broke. Four hundred rows of the same path is runtime, not signal, and the file belongs in the repo
Why it works: Right — a row has to be a rule, it has to be named, and it has to be versioned with the code.
- One row per rule, not per repetition.
- Name the row so failures read.
- Keep the data in the repo.
- More rows is not more coverage.
Three squads share checkout steps and each forked a page object. What do you introduce?
The strongest answer: A shared task / actor layer for the jobs they all perform, versioned like a library, with a contract for the few page APIs underneath
Why it works: Exactly. Shared jobs want a shared language. Ownership of a library beats a turf war.
- Shared tasks for shared jobs.
- Version the library.
- Do not win by turf.
- A new tool is not a shared layer.
Your framework offers per-test, per-file and per-run fixtures. Setting up a tenant takes thirty seconds; signing a user in takes two.
How do you decide the scope of each fixture?
The strongest answer: Run scope only for things that stay read-only and are expensive, like the tenant; test scope for anything a test mutates, like the user and its data. A shared fixture that is ever written to is the next order-dependent failure
Why it works: Exactly — the question is not speed, it is whether anything writes to the shared thing.
- Share only what stays read-only.
- Mutated state is per-test, always.
- Cost decides what is worth sharing.
- A written shared fixture is a future flake.
A feature flag is off in staging and on in the PR preview. How do tests cope?
The strongest answer: They name the flag they need, set it through a supported test API, and skip with a reason when the flag cannot be set — they do not assert both worlds in one silent test
Why it works: Exactly. Flags are environment. Tests either control them or declare the skip.
- Set flags through a test API.
- Skip with a reason if you cannot.
- Do not hard-code lucky URLs.
- Do not assert two worlds silently.
Six squads share your framework, and every triage starts with somebody rerunning the test locally with extra logging turned on.
What must every run leave behind so that stops?
The strongest answer: A structured log per test, written by default: the steps taken, the requests and responses with secrets redacted, the data the test created, and the build and environment — attached to the test, not buried in one console stream
Why it works: Right — triage should read the run, not reproduce it, and default-on is the only setting that helps.
- Capture it on the first run.
- Structured log per test, by default.
- Redact secrets; keep the requests.
- Attach it to the test, not a stream.
What does a test-platform team ship?
The strongest answer: The paved road: runners, data minting, lints, traces, and docs a squad can use on Tuesday — not a rewrite of every squad’s tests
Why it works: Exactly. Platform is leverage: the road, not the destinations. Centralising every test is a bottleneck.
- Ship the road.
- Do not steal every test.
- Do not annual-rewrite.
- Docs a squad can use Tuesday.
A vendor wants all oracles inside their cloud recorder. What do you keep local?
The strongest answer: The cases, the data factories, and the right to run in our CI — a recorder can help author; it must not own the only copy of the pack
Why it works: Exactly. Authoring help is fine. Escrow for your pack is not. Passwords are a leak, not leverage.
- Keep cases and factories.
- Keep a CI you can run.
- Recorders help; they do not own the pack.
- Passwords are not leverage.
What does a paved road for tests look like on day one of a new service?
The strongest answer: A template: CI trains, data minting, linted locators, traces on fail — the squad adds cases, they do not invent a runner
Why it works: Exactly. New work starts on the road. Blank repos and vendor-only starts are how dialects multiply.
- Ship a template that runs.
- Squads add cases, not runners.
- Do not copy a monolith’s mess.
- A vendor login is not enough.
Give the one sentence you want every squad to say about test platform.
The strongest answer: “I can mint data, run the smoke, and see a trace before lunch” — if they cannot, you built a centre of excellence, not a road
Why it works: Exactly. Self-serve before lunch is the bar. Tickets and hope are a bottleneck with a nicer name.
- Self-serve data, smoke, traces.
- Before lunch, not a ticket.
- Do not be a guild queue.
- Do not make skip the escape hatch.