I attempted to compile an overview of microservice transactions using the TCC pattern and the Saga pattern.

Hello, I'm Wakatchi ( @wakatchi_tech )

I received this question.

When building a system using microservices architecture, if the update target spans multiple services, handling transactions becomes extremely difficult. In particular, compensation (correction) transactions are required to roll back processing between services when a failure occurs, which requires complex transaction control.

A compensation transaction is a process that negates the execution results by canceling a failure in the middle of the process. Implementing compensation transactions imposes a burden on both the service that provides the cancellation process and the service that calls it, and tends to be complex to design and implement.

There are two types of transactions: local transactions, which process only one resource (such as a DB) within one transaction, and global transactions, which process multiple resources within one transaction. Global transactions are also called distributed transactions, and known implementations include X/Open XA and WS-Transaction.

Microservices architecture encourages local transactions and discourages global transactions (distributed transactions). This is because distributed transactions are complex and prone to failure. Microservices architecture recommends using eventual consistency across multiple resources rather than distributed transactions. There are architectures called the TCC (Try-Confirm/Cancel) pattern and the Saga pattern as methods to utilize eventual consistency of multiple resources.

This article will summarize how the TCC pattern and Saga pattern can be used to suit the transactional characteristics of microservices.

Types of transactions that cross microservices

Transactions that span microservices can be categorized by service coordination methods and associations between services. There are two methods of service coordination: request-reply and event-driven. Associations between services include orchestration (centralized) and choreography (distributed). So, what are the characteristics of each?

Service coordination method

Request/Reply

• This is a collaboration method using REST communication.
• We outsource some of our processing to other services.

event driven

• This is a coordination method in which specific service processing is driven by the occurrence of an event.
• By notifying other services of things that have already occurred, they can take appropriate actions based on the events that have occurred.

Associations between services

orchestration

• A central conductor controls the call for service.
• Each service executes logic in response to a request from the center and returns the execution results as a response to the center, using a request/reply method.
• Suitable for synchronous communication.

choreography

Each service operates autonomously and interacts with other predefined services.

• Each service operates autonomously according to defined conditions and sends data to other services that meet the conditions.
• Suitable for asynchronous communication.

If we categorize the transaction types using the vertical axis as the service coordination method and the horizontal axis as the association between services, we can illustrate the TCC pattern and the Saga pattern like this.

So what are the TCC pattern and Saga pattern?

TCC pattern

The TCC pattern stands for Try operations, confirmation, and cancellation, and is an architecture consisting of a coordinator and a group of services. The TCC pattern is an architecture based on a paper published in the United States, and it seems to be implemented by some e-commerce sites and cloud vendors.

Orchestration with TCC pattern

A service that corresponds to a coordinator performs a series of processes while calling a group of services. It is similar to the image of a program that processes while calling subroutines.

The TCC pattern controls transactions through two phases: the Try phase and the Confirm/Cancel phase.

Choreography with TCC pattern

Choreography using the TCC pattern applies when a service called by the coordinator calls another service (nested service call). Transaction control for nested called services is similar to orchestration.

Points to keep in mind when implementing the TCC pattern

Implement Try/Confirm/Cancel API

The TCC pattern provides a coordinator (orchestrator) that controls a series of processes and implements call control for multiple resources (services). The called service must implement three APIs: Try/Confirm/Cancel.

Processing may take twice as long to complete

Please note that the TCC pattern may take twice as long to complete. This is because the TCC pattern communicates twice between each service, and only starts the confirmation phase when it receives try request responses from all services.

Pass the service through a provisional (pending) state instead of a final state

The TCC pattern is characterized by passing the service through a temporary (pending) state rather than a final state, making cancellation processing easy. For example, an email service's send request marks the email as ready to send, and a confirmation request sends the email. The corresponding cancellation request will only be marked.

The Saga pattern described below results in a transaction sending an email and a corresponding compensation transaction sending another email explaining the cancellation.

Saga pattern

The Saga pattern is an event-driven architecture. The Saga pattern is not unique to microservices, but is an architecture that was also adopted in SOA. Also, the idea behind the Saga pattern is old, and is based on a 1987 paper.

The Saga pattern offers the idea of compensation in exchange for not being able to do rollbacks. Since it is not possible to perform a rollback, a pseudo rollback is performed instead using an undo operation that reverses the interaction operation.

The Saga pattern is processed as follows.

[Exhibitor]

Orchestration with Saga patterns

The Saga pattern's orchestration approach allows a coordinator service to centralize decision-making and ordering of Saga's business logic. Define a coordinator (Saga Orchestrator) that controls what each service should do and when it should do it. The coordinator communicates with each service which service should be executed using MQ events/return events.

Please note that the coordinator's requests for each service may be REST communication. The processing result will be awaited at the MQ return event.

Choreography with Saga patterns

In Saga pattern choreography, each service generates or listens to events from other services and decides whether to process them. The choreography results in each service being a single local transaction. The first transaction is initiated by an external request corresponding to a system operation, and each subsequent step is triggered by the completion of the previous transaction.

Please note that if you need to track the status of your order, the order fulfillment service may monitor all events and update its status.

Rollback of Saga pattern

For example, if a payment fails during a transaction, an event is issued to cancel the process and a rollback is attempted.

It is important to define a common identifier for each transaction in the event that occurs so that all consumers receiving the event immediately know which transaction it refers to.

Points to note when implementing Saga pattern

Processing using the Saga pattern starts with consuming an event, and by issuing another event (Publish), the processing is propagated to the next service. A series of processes chain this, and the last process publishes a completion event to the first process and ends. Events to be published are also retained in the database managed by each service, taking into account retries.

[Exhibitor]

https://www.nginx.com/blog/event-driven-data-management-microservices/

The Saga pattern is superior to the TCC pattern in that services are loosely coupled and flexible, and the impact on business logic is localized. On the other hand, the feasibility of compensating transactions must be considered, and care must be taken to ensure that the chain does not become too long.

summary

This time, I tried to touch on the TCC pattern and Saga pattern, which are architectures that I'm not used to hearing about. What did you think? Although some advanced companies are disseminating information, there are still few examples of implementation of these patterns, and they may not yet be at the widespread stage.

Recently, container orchestration using Kubernetes and service meshes using istio have become popular, making it relatively easy to introduce microservice architectures, but these technologies do not make transaction control easy. Transactions will continue to require careful consideration as microservices architectures become a reality.

We hope that this article will be of some help to you towards the introduction of microservices.

Thank you for reading to the end!