Orchestration interview questions
Orchestration questions a Data engineer can be asked, from first job to senior. Open one to see the strongest answer and why it works.
Job B needs Job A’s table. How do you schedule them?
The strongest answer: A dependency in the orchestrator — I do not start B at 07:05 and hope A finished at 07:00
Why it works: Exactly. Dependencies, not clocks. Hope-cron and coffee-clicks are folklore.
- Declare A then B.
- Do not hope on a clock.
- Do not click it by hand.
- Empty-first is not a dependency.
Your task has retries set to three. Which failures should those retries fix, and which should they not?
The strongest answer: Transient ones — a timeout, a throttled API, a lost connection — with a backoff between attempts. A schema mismatch or bad data fails identically every time, so it should fail fast and page
Why it works: Exactly — retries buy time for the world to recover, not for the code to change; separate the two kinds in the task’s error handling.
- Retry transient; fail fast on deterministic.
- Back off between attempts.
- Schema and data errors: no retry.
- Green via retries is not health.
A sensor pokes a table every minute for two hours. What is the grown-up version?
The strongest answer: An upstream success event or a data-aware schedule — I do not poll as a personality
Why it works: Exactly. Evented or data-aware. Polling and hoping are the two amateurs.
- Depend on an event or a data check.
- Do not poll as a habit.
- Do not Slack-page a human sensor.
- Do not drop the wait.
A DAG task reads a 30 GB table into pandas inside the Airflow worker, transforms it and writes it back. What is wrong with that picture?
The strongest answer: The orchestrator’s workers coordinate; they are sized for many small tasks and share memory, so one 30 GB transform starves every other DAG. Push the transform to the warehouse or Spark and let the task submit and wait
Why it works: Right — orchestrate the engine, do not become it; a worker running one 30 GB job starves every other DAG.
- Orchestrate compute; do not perform it.
- Workers coordinate many small tasks.
- Push transforms to the engine.
- The task submits and waits.
A daily Airflow DAG is scheduled for midnight, yet the run labelled Monday actually starts on Tuesday at 00:00. Is that a bug?
The strongest answer: No: the run is labelled by the data interval it processes — Monday’s data — and starts once that interval has ended. Tasks should use the interval’s start and end, not the wall clock, so reruns process the right day
Why it works: Exactly — the interval is the contract; template the interval bounds and reruns stay correct.
- Runs are labelled by data interval.
- Start after the interval ends.
- Use interval start and end.
- Never use the wall clock.
The DAG is green and the table is empty. What was missing?
The strongest answer: A data quality check on grain or volume that can fail the task — orchestrator-green is not data-green
Why it works: Exactly. Assert the data. Timeouts, twins, and empty mail are theatre.
- Test grain/volume in the DAG.
- Task success ≠ table ok.
- Do not twin the empty job.
- Do not mail a void.
You deploy a new daily DAG with a start date three years ago. Within minutes the scheduler is running a thousand tasks and the warehouse is throttled.
What happened, and what should the DAG have said?
The strongest answer: Catch-up: the scheduler created a run for every missed interval since the start date. Decide it explicitly — catch-up off when only today matters, or a deliberate backfill with max active runs capped when history is wanted
Why it works: Right — history is a feature you opt into deliberately; a start date years back with catch-up on is a thousand runs.
- Catch-up is a decision.
- Off unless history is wanted.
- Backfill deliberately, rate-limited.
- Cap active runs.
The morning SLA slipped. 80 tasks are green at 06:00 and three are still running. What do you look at?
The strongest answer: The critical path to the SLA table, not the count of green boxes — I will not add 20 more tasks that look busy
Why it works: Exactly. Path, not theatre. Hidden tasks and optional SLAs are how mornings die.
- Optimise the path to the SLA table.
- Green count is vanity.
- Do not hide the slow three.
- SLAs are not optional when busy.
How do you treat the orchestrator like a product?
The strongest answer: SLOs on the critical path, ownership of the paved DAG, and a queue that is not “ask in Slack” — I do not let every squad invent a runner
Why it works: Exactly. Path, paved DAG, one runner. Pools, yearly rewrites, and slides are not a product.
- SLO the critical path.
- One paved runner.
- No Slack-as-queue.
- Do not yearly-rewrite.
What does a grown-up backfill API look like?
The strongest answer: A ranged, idempotent, announced job with a progress and a consumer note — not 90 clicks in the UI
Why it works: Exactly. A product. Slack, drop-all, and laptops are how you page the company.
- Range + idempotent + announce.
- Show progress.
- No 90 UI clicks.
- No laptop scripts.
Gold is late because silver waited on bronze. How do you set SLAs?
The strongest answer: Each hop has an SLO, and gold’s residual names the parent that missed — I do not give gold a 06:00 promise the parents cannot keep
Why it works: Exactly. Parents keep clocks. Slogan 06:00 and no-SLAs are how mornings surprise.
- SLO per hop.
- Gold names the missed parent.
- Do not promise the impossible 06:00.
- Do not SLA only the last hop.
Models are trained on a hobby notebook pipeline. What do you require to call it production?
The strongest answer: Scheduled, tested, point-in-time, and erasable — a notebook is exploration, not a factory
Why it works: Exactly. Factory properties. GPUs, labels, and next-quarter are not production.
- Schedule + tests + PIT.
- Erasure path.
- Notebooks explore.
- No “prod” sticker on a hobby.
Critical ops run on a cron the author left. What do you do first?
The strongest answer: Stabilize it, name an owner, and plan a paved home — I do not yank it on Friday or pretend it is not production
Why it works: Exactly. It is production. Yank, forever, and night-rewrites are the failure modes.
- Treat it as production.
- Owner now.
- Plan the paved home.
- No Friday yank, no night rewrite.
A team proposes migrating the company’s four hundred Airflow DAGs to Dagster because “asset-based orchestration is the future.”
What do you require before you agree?
The strongest answer: A named problem the task model causes us — untracked lineage, no notion of stale tables — proof the asset model fixes it on our ten worst pipelines, and a migration plan for the other four hundred
Why it works: Exactly — assets versus tasks is a real difference; whether it is worth a four-hundred-DAG rewrite is a business question you must be able to answer.
- Name the problem, prove it, then plan.
- Assets vs tasks is a real shift.
- Pilot on the worst pipelines.
- Never run two orchestrators by drift.