top of page
  • Writer's pictureViacheslav Miroshnychenko

Events Modeling in Distributed Systems: Common Pitfalls, Event Streams, and Exactly-One Processing.

In the complex landscape of distributed systems, event modelling and processing play pivotal roles in ensuring system robustness and efficiency. From the pitfalls of minimal-information events, often likened to "clickbait," to the nuanced intricacies of event stream processing and the challenges of achieving exactly-once processing, recent articles have shed light on these critical aspects. They explore the balance required in event design, the advantages of asynchronous event stream processing, and strategies for consistent event processing, providing essential insights for architects and developers navigating the dynamic world of distributed systems.

In the realm of distributed systems, a common occurrence is events that carry minimal information, typically just a name and a resource identifier. These events are sometimes referred to as "clickbait-like" due to their lack of context. They indicate that something has happened, but the details are unclear from the event name alone. For instance, an event like ShipmentStatusChanged might only include the shipment identifier, omitting any status or additional data.

Such events can be likened to synchronous communication in terms of their granularity and the issues they present, including potential race conditions. However, it's crucial to recognize that not all minimal-data events signify poor system design. In some cases, the mere occurrence of an event provides sufficient context, especially in business-related scenarios. An example is certain shipment events, where the event's occurrence itself is the key information.

It's essential to avoid extremes in designing these events. If there's a valid reason for the absence of additional data (for example, if it's not used within the system), then artificially adding more fields to the event's body is unnecessary. Solutions for improving such situations often involve the use of event notification patterns, which can help in providing the needed context and ensuring effective communication within the system.

In distributed systems, efficiently accessing necessary data from other services is a key challenge. There are two main approaches: synchronous (where a request is made during logic execution, blocking until a response is received) and asynchronous (continuously updating a "local" storage with data, which can be accessed without delay during logic execution). An insightful article delves into the asynchronous approach, focusing on event stream processing.

This approach is explained through its two fundamental components: storage and processing, including the role of technologies like Kafka. It contrasts event stream processing with Batch Processing, highlighting the former's advantages. The real value of the article lies in its six practical examples of event stream processing applications, ranging from GitHub notifications and IoT to online trading, payments, and logistics. These examples demonstrate the versatility and efficiency of this approach in handling complex, real-time data needs in distributed systems.

In the realm of distributed systems, ensuring event processing consistency poses significant challenges. A practical solution lies in shifting focus from exactly-once delivery, which is complex to implement, to exactly-once processing. This approach involves ensuring that an event is processed only once, essentially achieving idempotent event handling. This can be accomplished by:

  1. Utilizing unique identifiers for each event, which allows for the verification of whether an event has already been processed.

  2. Implementing retryable operations to minimize data mutations.

The article discusses these two strategies in detail, providing insights into the advantages and drawbacks of each approach. This knowledge is vital for developing robust and reliable software architectures in distributed systems.



© 2023 IT 

We are here

Novogordka 31,
Warsaw, Poland

Get in touch

bottom of page