Implementing the Saga pattern on Azure - Part 3
We will now delve into the Saga pattern, providing its definitions and exploring how it is conventionally presented in the industry.
The definitions and concepts presented in this post reflect our understanding and should not be unquestionably accepted.
What is the Saga pattern ?
The Saga pattern is a design pattern used in distributed systems to manage long-running transactions. It breaks down a transaction into a series of smaller, self-contained steps or activities, each with its own compensating transaction. This approach allows for better resilience and fault tolerance in distributed environments.
- If one step fails, the compensating transactions of the preceding steps are executed to undo the changes and maintain consistency.
- Each step in the Saga represents an individual atomic transaction, and the entire sequence of steps ensures the overall transaction's integrity.
This definition is somewhat abstract and might seem a bit elusive, but what does it actually mean in concrete terms ?
What is the Saga pattern in concrete terms ?
We will dissect each term in the definition using illustrative examples.
The Saga pattern is used in distributed systems
This constitutes the straightforward aspect of the definition. The Saga pattern is predominantly employed in systems utilizing multiple isolated services that need to interconnect and collaborate. Its relevance is less pronounced in monolithic applications where there is typically a singular point of truth and pre-existing transaction mechanisms.
The Saga pattern is particularly valuable in microservices architectures where traditional ACID transactions might be challenging to implement due to the distributed and decentralized nature of the system. It provides a more flexible and scalable way to manage complex, multi-step transactions in such environments.
It breaks down a transaction into a series of smaller steps
Instead of executing a large transaction that spans multiple systems and might impact performance, the Saga pattern depends on local transactions. Each subsystem, such as the Order microservice or the Delivery module, is tasked with initiating its own coherent mechanism. This approach is inherently logical: each module possesses an intimate understanding of the intricacies of its technology and is, therefore, best suited to determine how to execute a transaction within its environment.
However, if the Saga pattern were merely a sequence of more or less independent local transactions executed by each subsystem at its convenience, it would lack efficiency and value. This is why a mechanism called compensation is introduced.
Each step has its own compensating transaction
The compensating transaction pattern involves defining and implementing compensating transactions for each step or activity within a Saga and are designed to undo the effects of the corresponding original transactions in case of a global failure or error. When a step within the sequence encounters an issue, the compensating transaction for each preceding step is triggered. These compensating transactions are carefully crafted to reverse or compensate for the changes made during the successful execution of their corresponding steps.
The compensating transaction pattern ensures that, in the event of a failure, the system can be brought back to a consistent state by systematically applying the compensating transactions. This approach provides a way to maintain data integrity and consistency despite failures in the distributed environment.
Who is responsible for overseeing the execution throughout the entire process ?
This mechanism requires a central coordinator, often referred to as an orchestrator, which instructs other services to execute their local transactions and, if necessary, to roll them back in the event of a failure. In the context of a serverless architecture, such as on Azure, this orchestrator could be implemented as an Azure Function.
The Azure Function, functioning as the executor, executes the first local transaction, followed by the second one if the first one succeeds. In the event of a failure, the executor rolls back the first transaction using the previously implemented compensation logic.
To provide a comprehensive view, it's important to note that the Saga pattern can also be implemented not only with an orchestrator but with a mechanism known as choreography. In this approach, each local transaction publishes an event upon success, and other transactions subscribe to these events, executing when their conditions are met. However, this process won't be covered in this series.
This approach is more straightforward to comprehend and oversee since the saga execution logic is centralized. However, it can also pose a potential bottleneck and a single point of failure if not meticulously designed.
Give me an example
We persist with the previously mentioned scenario and envision once more that an order is successfully placed in the Order database, but an error hinders the recording of the delivery. The following diagram illustrates how this situation is addressed using the Saga pattern.
What happens if the compensation logic fails ?
There are instances when the compensation logic may fail. This situation can be addressed through a combination of the following approaches:
- Implementing the Retry pattern for the composating action
- Utilizing exception handling
If an automated process is unable to resolve the issue, the fallback option is to generate an exception report, which can be reviewed manually. This manual review allows for appropriate actions to be taken based on the identified issue.
Resorting to a manual process may appear cumbersome, but nothing comes without a cost in this world. We trade the advantages of microservices, such as modularity and ease of deployment, against very rare and occasional instances where we need to resort to manual methods.
But let's leave theory behind. The final post will focus on practical implementation, demonstrating the Saga pattern on a serverless architecture using Azure.