Consistency ↔ Availability

This is the first axis, and the one interviewers push on hardest. Sometimes the network splits, so one part of the system cannot talk to another part. Engineers call this a partition. When a partition happens, a distributed system has only two choices: refuse to answer, or answer and risk being wrong. Good engineers pick on purpose, and can say which one they picked.

Two nodes. One goes silent. What do you do?

• Answer anyway Serve whatever this node knew last. You stay up, but you might hand back old or conflicting data.
• Refuse to answer Return an error until the nodes agree again. You stay correct, but you are down for that request.

Node A cannot reach Node B, so it has to decide alone: reject the write and stay correct, or accept the write locally and fix things up with B once the link comes back.

There is no third option during a partition. The skill is knowing which door your data should take, and it is not the same door for every table.

Consistency vs availability, without the jargon

• Consistency Everyone who reads gets the latest agreed value. Read right after a write, and you see that write, no matter which server answers.
• Availability Every request gets a real answer, not an error, even while some servers are unreachable. The system always answers, even if the answer is a little behind.

The two only clash when the network breaks. On a healthy network you get both at once. This axis is about what you do on your worst day, not your average one.

"P" is not optional, so it's really C vs A

• Healthy Nodes can talk to each other. Reads are fresh and every request gets an answer. You have both C and A.
• Partition (P) A switch dies, or a link drops. Two nodes cannot reach each other. This will happen sooner or later.
• Pick C Reject writes on the side that got cut off. Correct, but unavailable there.
• Pick A Accept writes on both sides. Available, but now the two sides disagree.

You cannot skip P. Partitions are just a fact of networks. So CAP (Consistency, Availability, Partition tolerance) really comes down to a forced choice between C and A the moment a partition happens.

Same app, different doors

The strong move here: do not just answer "is this system CP or AP?" Answer "which of my data needs C, and which can live with A?"

Same choice, in code

Here are two small stores with the same interface, but the opposite behavior during a partition. QuorumStore refuses to write unless it can reach enough copies of the data, called a quorum, to be sure they agree. EventualStore accepts the write right away and queues it to sync up later, once the partition heals.

interface WriteResult {
  ok: boolean;
  value?: number;
  error?: string;
}

interface KVStore {
  write(key: string, value: number): Promise<WriteResult>;
  read(key: string): Promise<WriteResult>;
}

// Strongly consistent: needs enough replicas (a quorum) to be reachable
// before it confirms the write. If a partition blocks that, refuse the write.
class QuorumStore implements KVStore {
  constructor(
    private replicas: string[],
    private quorum: number,
    private reachable: (id: string) => Promise<boolean>,
    private local = new Map<string, number>(),
  ) {}

  async write(key: string, value: number): Promise<WriteResult> {
    const acks = await Promise.all(this.replicas.map((r) => this.reachable(r)));
    const ackedCount = acks.filter(Boolean).length;
    if (ackedCount < this.quorum) {
      return { ok: false, error: "quorum unreachable, refusing write" };
    }
    this.local.set(key, value);
    return { ok: true, value };
  }

  async read(key: string): Promise<WriteResult> {
    const acks = await Promise.all(this.replicas.map((r) => this.reachable(r)));
    if (acks.filter(Boolean).length < this.quorum) {
      return { ok: false, error: "quorum unreachable, refusing read" };
    }
    return { ok: true, value: this.local.get(key) };
  }
}

// Eventually consistent: accept the write right away, queue it, and let
// it sync up once the partition heals. The caller gets an answer now,
// but it might be stale, or later overwritten by a conflict resolver.
interface PendingWrite {
  key: string;
  value: number;
  since: Date;
}

class EventualStore implements KVStore {
  private local = new Map<string, number>();
  private pending: PendingWrite[] = [];

  async write(key: string, value: number): Promise<WriteResult> {
    this.local.set(key, value);
    this.pending.push({ key, value, since: new Date() });
    return { ok: true, value };
  }

  async read(key: string): Promise<WriteResult> {
    return { ok: true, value: this.local.get(key) };
  }

  // Called once the partition heals. This uses last-write-wins; a real
  // system might use vector clocks or a CRDT instead.
  async reconcile(remote: PendingWrite[]): Promise<void> {
    for (const w of [...this.pending, ...remote].sort((a, b) => a.since.getTime() - b.since.getTime())) {
      this.local.set(w.key, w.value);
    }
    this.pending = [];
  }
}

Same shape, same interface, opposite answer to the question "what do I do when I can't reach the other side?" That is the whole axis, written as code.

Even with no partition, you still trade

CAP only talks about what happens during a partition. But the network is fine 99.9% of the time, and you are still making a choice then too, this time between latency (how long a request takes) and consistency.

This bigger trade-off has a name: PACELC.

• If there is a Partition, choose Availability or Consistency. That is the CAP choice.
• Else, on a normal day, choose Latency or Consistency: wait for replicas to confirm, or answer fast and risk a stale read.

PA/EL systems, like Dynamo and Cassandra, pick availability during a partition and low latency the rest of the time, so they end up eventually consistent. PC/EC systems, like a single-leader SQL setup, pick consistency both times, so they pay for it in latency. Being able to name your quadrant, PA/EL or PC/EC, is a strong signal that you understand this trade-off.

The defaults, so you can name them fast

• Lean C (CP) Postgres/MySQL (single leader), ZooKeeper, etcd, HBase, Spanner*
• Lean A (AP) Cassandra, DynamoDB, Riak, Redis (replicates asynchronously), most DNS
• Tunable Cassandra/Dynamo let you set the quorum per query; Mongo has a write/read concern setting; you pick C or A at call time

*Spanner gets strong consistency and high availability by using atomic clocks and Paxos, a consensus algorithm, to make partitions rare and short. It does not beat CAP, it just makes the cost of choosing C small enough to live with.

Axis → Position → Consequence → Trigger

A strong answer in an interview has four parts:

1. Axis "This is a consistency-vs-availability call on the orders table."
2. Position "I'm choosing consistency: single leader, synchronous commit."
3. Consequence "So a leader failover means a few seconds where writes are rejected, not wrong."
4. Trigger "If checkout latency or write volume forces it, I'd revisit with a queue in front."

That four-part answer is the difference between saying "I'd use Postgres" and sounding like you have actually run this in production.

Where this axis is fumbled

✗ "We picked CP, so we're consistent everywhere."

Consistency applies per write path, not to the whole system at once. Your cache, your read replica, and your search index can all be eventually consistent, no matter what the primary database does.

✗ "We're highly available, so CAP doesn't apply."

You can be available and consistent, right up until the first partition. CAP is about that moment, and that moment always arrives.

✗ Treating the whole system as one choice

Account balances want C. View counts want A. Both live in the same product. Decide per data type, not once for the whole system.

✗ Forgetting the EL half

Even with zero partitions, you still trade latency for consistency on every replicated read. That is the 99.9% case, the normal day.

During a partition, you either answer or you're correct. Pick per data type, and know your trigger

During a partition, you either answer or you are correct. Pick per data type, and be ready to say what would make you switch.

1. A partition will happen. So every distributed store is really choosing C or A for the moment things break.
2. The choice is per data type. Money leans C. Counts and feeds lean A, in the same app.
3. PACELC reminds you the trade is latency vs consistency, even on a healthy day.
4. Next: Latency vs Throughput, the axis that decides whether you optimize for one request or for a million.