All jobs / Automation developer
CI/CD interview questions
CI/CD questions a Automation developer can be asked, from first job to senior. Open one to see the strongest answer and why it works.
When should the automated pack run?
The strongest answer: On every change that can break it — usually every push — with a fast smoke first and a longer pack on a slower train
Why it works: Exactly. Fast signal on the change. Nightly is extra, not the only net.
- Run on the change.
- Fast smoke, slower full pack.
- Do not wait for release eve.
- Do not make CI a quarterly ritual.
What belongs in the smoke the pipeline must pass to ship?
The strongest answer: A handful of money-path checks that finish in minutes — login, buy, the webhook you cannot afford to miss — not the whole regression encyclopaedia
Why it works: Exactly. Smoke is the gate you can afford to run always. Depth lives on a slower train.
- Smoke = scarce money paths.
- Minutes, not hours.
- 200 OK is not a buy.
- Depth belongs on a slower job.
A CI test fails and the log says “element not found.” What else should the job keep?
The strongest answer: A screenshot, a trace, and the last network or console lines — enough to debug without rerunning locally for an hour
Why it works: Exactly. Traces are how remote failures become local understanding.
- Keep screenshot + trace.
- Keep last network / console.
- Do not rely on a meeting.
- Do not assume local repro.
Your CI job prints test output to the console and nothing else. Why do teams also emit a JUnit XML report?
The strongest answer: Because CI can parse it: failed tests appear by name in the run summary and on the pull request, with the assertion and the duration, instead of somebody scrolling a console log to find them
Why it works: Exactly — the point is the machine reading it so the person does not have to.
- Reports are for machines to read.
- Failures surface by name in CI.
- Console logs do not summarise.
- The exit code is a separate thing.
The PR job is 40 minutes of UI tests. What do you change first?
The strongest answer: Split: unit + API on every push, a thin UI smoke on the PR, and the long UI pack on main or a nightly — I do not ask people to wait 40 minutes to merge a typo
Why it works: Exactly. Time-to-signal is a product. Layers and trains beat a single slog.
- Fast layers on the PR.
- Long UI on a slower train.
- Hardware is not a strategy.
- Do not hide the pack on laptops.
You are adding an end-to-end job to a GitHub Actions workflow. What goes in it beyond running the tests?
The strongest answer: Pin the runner and tool versions, cache dependencies, install the browsers the runner does not ship, take credentials from repository secrets, shard across a matrix, and upload the report and failure artefacts even when the step fails
Why it works: Right — the upload on failure and the secrets are the two people forget, and both bite on the first red run.
- Pin versions; cache installs.
- Secrets come from the secret store.
- Shard with a matrix.
- Upload artefacts even on failure.
How do you choose shards vs workers vs more jobs?
The strongest answer: By isolation cost: workers if data is namespaced, shards if the suite is a bag of files, separate jobs if the packs have different oracles and SLOs
Why it works: Exactly. Parallelism is a data and SLO design. Marketing pictures are not a plan.
- Match the knob to isolation.
- Different oracles → different jobs.
- Do not max workers blindly.
- Do not worship one log stream.
Finance asks why test minutes doubled. What is an honest answer?
The strongest answer: Which pack grew, whether it found bugs, and what we will move down or kill — not “quality is expensive” as a slogan
Why it works: Exactly. Minutes are a portfolio. Show signal per minute, then cut or move.
- Show signal per minute.
- Move or kill low-signal packs.
- Do not hide the bill.
- Do not cut blindly to last year’s number.
When do you keep Playwright traces?
The strongest answer: On failure always, and a sample of passes when you are hunting a flake — retention with a size cap, not every passing video forever
Why it works: Exactly. Failures need traces. Forever-all-passes is a cost and a privacy pile.
- Traces on fail.
- Sample passes while hunting.
- Cap retention; watch PII.
- Do not wait for a manager.
Who owns the test pipeline when it is red on main?
The strongest answer: A named owner for the paved road, plus the squad whose change broke it — “the platform team will notice” is not an owner
Why it works: Exactly. The road has an owner. The break has an owner. Folklore on-call is how red becomes weather.
- Name the road owner.
- Name the breaking squad.
- No lifestyle red.
- No last-touch folklore.
You are writing the rule that decides whether a new test runs on every pull request or only nightly. What is the rule?
The strongest answer: On the pull request if it is fast, deterministic and covers something we would block a merge for; nightly for the slow, the broad matrix and anything whose failure needs a person — and every nightly failure gets an owner by morning or the test leaves the pack
Why it works: Right — and the ownership clause is what stops nightly becoming a graveyard nobody reads.
- Block a merge only for blocking risk.
- Slow and broad runs on a train.
- Every nightly failure gets an owner.
- Never demote a test to hide a number.
How do you treat the PR test job like a product?
The strongest answer: With an SLO: time-to-signal, flake rate, and “main is green” — owned, with a budget, the same way an API has latency
Why it works: Exactly. If it has users (every engineer), it has an SLO. Hope is not an SLO.
- SLO: time, flakes, green main.
- Named owner and budget.
- Hope is not an SLO.
- Bake-offs are not ops.
A thousand UI tests must finish in 15 minutes. What do you challenge first?
The strongest answer: Whether a thousand UI tests should exist — then isolation, shards, and a cheaper layer — I do not first buy a thousand browsers
Why it works: Exactly. Scale problems are usually shape problems. Hardware last, honesty first.
- Challenge the shape first.
- Then isolation and shards.
- Hardware last.
- Do not quietly move the SLO.
The nightly suite has been red for three weeks. What do you do?
The strongest answer: Kill or quarantine with owners until nightly means something — a red ritual trains everyone to ignore the only deep pack
Why it works: Exactly. A red nightly is a dead oracle. Restore meaning or admit you do not have a deep pack.
- Restore meaning or kill it.
- Owners on every quarantine.
- No lying badges.
- Do not hide it on a slower calendar.
Audit asks for “proof of testing.” What do you hand them?
The strongest answer: The gates that ran, the residual that was signed, and the artefacts that still exist — not a folder of 4,000 unmarked screenshots
Why it works: Exactly. Evidence is gates, residual, artefacts. Badges and slogans are theatre.
- Gates + signed residual + artefacts.
- No badge zips.
- No 100% slogans.
- Do not promise next quarter’s proof.