Blog Init

PACELC Theorem - An Extension to CAP Theorem in Distributed Systems

The PACELC Theorem: Why CAP Isn't Enough for Modern Distributed Systems

Introduction: The Evolution Beyond CAP

For over two decades, the CAP theorem has been the cornerstone of distributed systems thinking. It states that in case of network partitioning ℗ in a distributed computer system, one has to choose between availability (A) and consistency ©. But as distributed systems evolved and became more sophisticated, engineers started noticing something crucial: CAP only tells half the story.

The PACELC theorem was first described by Daniel Abadi from Yale University in 2010 in a blog post, which he later clarified in a paper in 2012. The purpose of PACELC is to address his thesis that "Ignoring the consistency/latency trade-off of replicated systems is a major oversight [in CAP], as it is present at all times during system operation, whereas CAP is only relevant in the arguably rare case of network partitions.

What is PACELC?

PACELC (pronounced "pass-elk") stands for:

The theorem states that in case of network partitioning (P) in a distributed computer system, one has to choose between availability (A) and consistency (C) (as per the CAP theorem), but else (E), even when the system is running normally in the absence of partitions, one has to choose between latency (L) and loss of consistency (C).


Fig. 1: The tradeoff between availability, consistency and latency, as described by the PACELC design principle.

The Critical Gap CAP Left Behind

Why CAP Falls Short

CAP theorem has several limitations that PACELC addresses:

  1. Binary Thinking: According to the CAP theorem, a database can be considered Available if a query returns a response after 30 days. Obviously, such latency would be unacceptable for any real-world application.

  2. Partition-Only Focus: CAP assumes systems are always dealing with partitions, but the CAP theorem forces us to assume that the system has network partitions, which is not necessarily the case at all times.

  3. Performance Blindness: PACELC was developed to address a key limitation of the CAP theorem: it makes no provision for performance or latency.

The ELC Extension: What Happens During Normal Operation?

What choices does a distributed system have when there are NO network partitions? The CAP theorem doesn't answer the question. The PACELC theorem answers this.

When your system is running normally (no partitions), you still face a fundamental tradeoff between:

The Four PACELC Configurations

There are four configurations or tradeoffs in the PACELC space:

1. PA/EL Systems (High Availability + Low Latency)

ScyllaDB can be defined as PA/EL: partition tolerant, highly available, and oriented towards low latency as opposed to consistency.

2. PC/EC Systems (Strong Consistency Always)

3. PA/EC Systems (Conditional Consistency)

4. PC/EL Systems (Partition-Sensitive)

Real-World Applications

Netflix: The EL Champion

Netflix exemplifies the EL aspect of the PACELC theorem by emphasizing low latency in normal operations. By optimizing for fast content delivery while accepting eventual consistency for some non-critical data, Netflix ensures a smooth and responsive user experience, even during high demand.

When you click play on a Netflix show, you get instant streaming (low latency) even if your watchlist updates take a moment to sync across all devices (eventual consistency).

Banking Systems: PC/EC in Action

Traditional banking systems prioritize consistency above all else. In a banking application, if one user transfers money, all other users should immediately see the updated balance to avoid discrepancies. They'll sacrifice both availability and latency to ensure your account balance is always accurate.

E-commerce During Sales: PA/EL Trade-offs

Consider an e-commerce application during a sale. If a data center goes offline due to a network issue, customers can still place orders through other available nodes. Although they may not see the most up-to-date inventory (due to potential stale reads), they can still interact with the system without downtime.

Implementation Considerations

Understanding Latency in Practice

From a general point of view, latency is a time delay between the cause and the effect of some physical change in the system being observed. In distributed systems, this translates to the time between sending a request and receiving a response.

Consider a write operation in a replicated database:

Write Request → Primary Node → Replicate to Secondary Nodes → Acknowledge Success
     |              |                        |                      |
   0ms            5ms                     15ms                   20ms

Consistency Models Spectrum

PACELC recognizes that consistency isn't binary. You can choose from:

Making PACELC Decisions

Framework for System Design

Choose Wisely Between Consistency and Availability: Depending on your application requirements, you may prioritize availability (like in Cassandra) or consistency (like in Google Spanner). Assess your use case carefully.

Questions to Ask:

  1. What's the cost of stale data? (Financial systems: very high; Social feeds: low)
  2. What's the cost of downtime? (Emergency services: very high; Analytics dashboards: medium)
  3. What latency do users expect? (Gaming: <50ms; Batch reports: minutes are fine)
  4. How often do partitions actually occur? (Cross-datacenter: frequent; Single datacenter: rare)

Monitoring and Adaptation

The dynamics of distributed systems can change over time. Regularly monitor your system's performance and be ready to adjust your consistency and availability strategies based on real-world usage patterns.

Beyond the Theorems: Practical Wisdom

The Reality Check

Both CAP and PACELC are valuable tools, but neither is a step-by-step recipe for building systems. Instead, they provide mental models for evaluating tradeoffs and understanding the limits of distributed architectures.

Hybrid Approaches

Modern systems often implement different PACELC profiles for different data types:

Conclusion: Thinking Beyond CAP

PACELC mostly takes aim at the first and fourth misconceptions I listed: that CA systems exist, and that eventual consistency is all about CAP. First, it doesn't allow you to ignore partition tolerance. Second, it makes it clear that even in the absence of partitions there's a tradeoff between consistency and latency.

The PACELC theorem doesn't replace CAP, it extends it. By acknowledging that systems must make consistency-latency tradeoffs even during normal operation, PACELC provides a more complete framework for understanding distributed systems behavior.

As we build increasingly complex distributed applications, from microservices to edge computing to IoT systems, having this nuanced understanding of tradeoffs becomes essential. The question isn't just "What happens when things break?" but "What happens when everything works perfectly?"

That's the insight PACELC brings: even in perfect conditions, distributed systems are about tradeoffs. The key is making conscious, informed decisions about which tradeoffs serve your users best.

Understanding PACELC helps developers make better decisions about database selection, consistency models, and performance optimization. It's not about finding the "right" answer it's about understanding the question more completely.

References

  1. PACELC design principle - Wikipedia
  2. PACELC Theorem Explained - Lohith Chittineni
  3. PACELC Theorem - Scylla DB
  4. CAP and PACELC Theorem in Plain English
  5. CAP Twelve Years Later: How the "Rules" Have Changed
  6. Please stop calling databases CP or AP - Martin Kleppmann
  7. A Critique of CAP Theorem
  8. Notes on Distributed Systems for Young Bloods
  9. Exploring the World of Distributed Systems: PACELC Theorem or Why CAP Is Not Enough
  10. PACELC Theorem How.dev
  11. CAP vs PACELC - Design Gurus
  12. CAP and PACELC: Thinking more Clearly about Consistency