All jobs / Solutions / cloud architect
System Design interview questions
System Design questions a Solutions / cloud architect can be asked, from first job to senior. Open one to see the strongest answer and why it works.
An interviewer says: “Design a URL shortener.” What do you do first?
The strongest answer: Ask what it must do and at what scale — reads vs writes, expiry, custom aliases, analytics — and state the assumptions you will design against before drawing anything
Why it works: Right. Requirements and numbers first; the boxes follow from them.
- Clarify functional and non-functional requirements first.
- State assumed scale in numbers.
- Only then draw.
- Check assumptions with the interviewer.
What does a load balancer do, and what does it not do?
The strongest answer: It spreads incoming requests across several identical servers and stops sending to ones that fail health checks; it does not make one slow server faster or fix a database bottleneck behind them
Why it works: Right. Distribution and failover, nothing more.
- Distributes requests across servers.
- Removes unhealthy servers.
- Not a cache, not capacity.
- Bottlenecks behind it remain.
What is the difference between scaling up and scaling out?
The strongest answer: Up means a bigger machine — simple, but there is a ceiling and one box to lose; out means more machines behind a balancer, which needs the service to be stateless or to share state elsewhere
Why it works: Right. Up is easy until the ceiling; out needs stateless services.
- Up: bigger box, ceiling, single failure.
- Out: more boxes, needs statelessness.
- Choose by state and ceiling.
- Neither is always right.
Why do interviewers keep asking whether a service is stateless?
The strongest answer: Because a stateless service keeps nothing between requests on its own box, so any instance can serve any request — that is what lets you add instances, replace a dead one and deploy without users noticing
Why it works: Right. Statelessness is what makes scaling out and failover trivial.
- Stateless: nothing kept between requests locally.
- Any instance serves any request.
- State moves to a shared store.
- Enables scale-out and failover.
When does adding a cache help, and when does it not?
The strongest answer: It helps when the same data is read far more often than it changes and a slightly stale answer is acceptable; it does not help write-heavy paths, unique-per-request data, or when staleness is unacceptable
Why it works: Right. Read-heavy, repeated, staleness-tolerant — otherwise it adds complexity for nothing.
- Read-heavy and repeated.
- Staleness must be acceptable.
- Not for writes or unique reads.
- Name what is cached and for how long.
In a design, why do you say which columns are indexed?
The strongest answer: Because the query pattern decides the index: a lookup by short code needs an index on that column or every read scans the table; each index also slows writes and takes space, so you index what you query, not everything
Why it works: Right. Index the lookup paths; pay on writes.
- Index what you look up by.
- Each index costs writes and space.
- Query pattern decides.
- Say the key columns explicitly.
What is a read replica for?
The strongest answer: A copy of the database that receives writes from the primary and serves reads, so read traffic scales out and the primary is left for writes; reads from it can lag slightly behind the primary
Why it works: Right. Reads scale out; expect replication lag.
- Copies primary; serves reads.
- Scales reads, not writes.
- Lag exists.
- Primary stays for writes.
Why would you put a queue between two parts of a system?
The strongest answer: To decouple them: the producer finishes fast and the consumer works at its own pace, spikes are absorbed, and a consumer being down does not lose the work — at the cost of the result arriving later
Why it works: Right. Decoupling and absorbing spikes, paid for in latency.
- Decouples producer and consumer.
- Absorbs spikes.
- Survives consumer outages.
- Result becomes asynchronous.
Users get logged out whenever the platform scales the API from two instances to three.
What makes a service “stateless”, and why does the architect keep asking for it?
The strongest answer: Any instance can serve any request because session and working data live outside it; that is what lets you add, remove and replace instances freely
Why it works: Exactly. State lives in a store the instances share; the instances are interchangeable.
- Any instance, any request.
- State in a shared store.
- Enables scaling and replacement.
- Sticky sessions break it.
What is the difference between latency and throughput?
The strongest answer: Latency is how long one request takes; throughput is how many the system handles per second — you can have high throughput with bad latency, and adding servers raises throughput without shortening one request
Why it works: Right. One request's time versus requests per second.
- Latency: time per request.
- Throughput: requests per second.
- They move independently.
- Scaling out helps throughput.
An interviewer asks for p99 latency. What does that mean and why ask for it?
The strongest answer: The time under which 99% of requests finish — it shows the slow tail that averages hide, and the tail is what users on the worst path feel and what times out downstream
Why it works: Right. Tails matter more than averages.
- 99% of requests finish under it.
- Exposes the tail.
- Averages hide outliers.
- Tail drives user experience and timeouts.
Scaling up versus scaling out: what decides which you reach for?
The strongest answer: Whether the workload can be split across instances; scale up is simple but has a ceiling, scale out needs stateless design and shared storage but goes further
Why it works: Exactly. Splittability decides it. A bigger box is often the right first step; it just ends.
- Can the work be split?
- Up is simple, with a ceiling.
- Out needs stateless design.
- Often up first, out later.
A stakeholder uses “fast” to mean two different things. What are they, and why does it matter?
The strongest answer: Latency — how long one request takes — and throughput — how many we handle per second; you can improve one while making the other worse, so the requirement must say which
Why it works: Exactly. Batching raises throughput and latency together; you need to know which one the user feels.
- Latency: one request’s time.
- Throughput: requests per second.
- They trade against each other.
- Make the requirement name one.
How do you estimate storage for 100 million short URLs?
The strongest answer: Pick a per-record size — say a few hundred bytes for the code, the long URL and metadata — multiply, and round: a few hundred bytes times 100 million is tens of gigabytes, so one database handles it and the interesting problem is reads, not storage
Why it works: Right. Rough units, a multiplication, a conclusion.
- Per-record size times count.
- Round to a power of ten.
- Draw a conclusion from it.
- Say what the real constraint is.
What does a CDN do for a web application?
The strongest answer: It serves static files from servers near the user, so images, scripts and styles come from the edge with low latency and the origin only serves what changes
Why it works: Right. Static content at the edge, origin for the rest.
- Static files near the user.
- Lower latency for assets.
- Origin serves dynamic content.
- Cache headers control it.
Logs, metrics and traces: what question does each one answer?
The strongest answer: Metrics say whether something is wrong and how much; logs say what happened in one place; traces follow one request across services to show where the time went
Why it works: Exactly. Is it wrong, what happened, where did the time go. Three questions, three signals.
- Metrics: is it wrong, how much?
- Logs: what happened here?
- Traces: where did the request go?
- One incident uses all three.
How do you estimate capacity for a system that does not exist yet?
The strongest answer: Start from business numbers — users, actions per user, peak-to-average ratio — derive requests, data and bandwidth, state the assumptions, and plan to replace the estimate with measurements
Why it works: Exactly. Business inputs, derived numbers, written assumptions, measured replacement.
- Start from business numbers.
- Derive requests, data, bandwidth.
- Write the assumptions down.
- Replace with measurements early.
What observability decisions belong in the design, before any code is written?
The strongest answer: A correlation id propagated across every hop, structured logs with agreed fields, the key business and latency metrics, and which dashboards and alerts each service owns
Why it works: Exactly. Correlation, structure, key metrics, ownership. The vendor comes after.
- Correlation id on every hop.
- Structured logs with agreed fields.
- Named key metrics.
- Owned dashboards and alerts.
Twelve services, no tracing. How do you introduce distributed tracing without a year-long programme?
The strongest answer: Standardise the context propagation and instrumentation library first, instrument the gateway and the two slowest paths, sample sensibly, and let each team adopt the shared setup on its own schedule
Why it works: Exactly. Standard first, prove value on the painful paths, then spread.
- Standardise propagation first.
- Instrument the slowest paths.
- Sample deliberately.
- Spread by paved path, not decree.