When Not to Use Kafka: The Importance of "Read Your Own Writes" Consistency

When designing systems, it's crucial to choose the right tools for the right use case. A common question in system design is whether Kafka should be used.

Here's one important piece of advice: avoid Kafka in scenarios where "read your own writes" consistency is critical.

Let’s break this down with a few examples.

Scenarios Requiring "Read Your Own Writes" Consistency

1. E-Commerce: Add to Cart
   Imagine a user on an e-commerce website who clicks "Add to Cart." The system must immediately update the cart so that the user can see the changes reflected in real-time. Any delay in this update could lead to confusion or frustration.

2. Order Creation Workflow
   When a user places an order, it’s important that the order instantly appears in their order history. Users expect to see confirmation of their actions immediately, and delays in reflecting this information can erode trust in the system.

3. Payment Processing
   After completing a payment, users expect their transaction to be instantly reflected in their account balance or transaction history. Even a slight delay in updating this critical information can cause confusion or concern, especially when financial transactions are involved.

---

Why Kafka May Not Be Suitable for These Scenarios

Kafka, as a distributed message broker, operates on a producer-consumer model. While it is excellent for handling high-throughput, fault-tolerant, and asynchronous message processing, there is an inherent delay between:

• The time a message is produced (e.g., a payment event is written to Kafka) and

• The time it is consumed (e.g., when the cart or transaction history is updated).

This delay makes Kafka unsuitable for scenarios where immediate feedback is required and where users need to see updates reflected as soon as they perform an action.

---

Alternative: Direct Database Writes

For use cases that demand real-time updates and "read your own writes" consistency, direct database writes are often a better approach. Why?

• Instant Updates: Data is written directly to the database, ensuring that users can immediately query and see the latest state.

• Consistency: Users experience the system as predictable and reliable, as their actions are instantly reflected in the user interface.

When to Use Kafka

While Kafka is not ideal for real-time, user-facing updates, it shines in other use cases, such as:

• Event-driven architectures.

• Asynchronous workflows like order fulfillment or payment reconciliation.

• Analytics and log processing, where delays are acceptable.

---

Conclusion

When designing systems, it’s important to align tools with the requirements of the use case. For scenarios requiring immediate feedback and "read your own writes" consistency, direct database writes are a better fit than Kafka. Kafka's inherent delays can lead to user frustration and inconsistencies, making it unsuitable for critical user-facing workflows like adding items to a cart, order placement, or payment updates. Always evaluate the trade-offs and design systems that meet user expectations seamlessly.