Global Transactions with Narayana JTA and Geode as Last-Resource Commit Optimization.

Note: The Narayana JTA LGPL 2.1 license is incompatible with the ASF policies. Later prohibits  Apache Geode from distributing LGPL components within its releases. However according to ASF policies, the NarayanaJTA/Geode utilities can be considered non-included work (optional features) and Geode can provide instructions on how to obtain and install the non-included work. This Wiki page is meant to provide such instructions. 

The Apache Geode (Gemfire) - Narayana JTA enables global transactions between XA (RDBMS, JMS...) and non-XA (such as Apache Geode and Gemfire) resources. It leverages the embeddable, standalone, open-source Narayana JTA and implements the Last-Resource Commit Optimization (LRCO) for preserving the ACID properties. (related blog)

This integration removes the dependency on an external, obsolete Java EE application servers as well as the restrictive programming/operational models imposed by them. It can be used in cloud-native architectures.

Apache Geode already allows integration [2] with the JTA managers of several legacy Java EE platforms such as JBoss, WebLogic, WebSphere. But the obsoleted Java EE technology stack doesn’t scale with the evolving business needs, programing models and the devops/cloud trends. The recent Gartner report (Market Guide for Application Platforms [3]) highlights that the “digital business initiatives require new features and capabilities in application platforms, and Java EE has failed to keep pace... Application leaders responsible for modernizing application infrastructure should develop a strategy to deal with the obsolescence of Java EE”.

The narayana-jta-geode-support [1] extends existing Geode JTA support by integrating a standalone (open-source) JTA provider and enabling LRCO for transaction-atomicity and data-consistency. The utility includes two sub-projects:

• narayana-geode-core - uses minimal external dependencies. Only the Narayana and the plain Geode/Gemfire APIs. No dependencies on Spring, Spring Data Gemfire and so.

• narayana-geode-springboot extends narayana-geode-core to provide a seamless integration with Spring Boot and Spring Data Gemfire (SDG).

Support is provided for Gemfire 8.2.x, Gemfire 9.0.x and Geode 1.1.x. (See the Appendix B matrix).

Quick Start

• Generate a Spring Boot application, preconfigured with Narayana, JPA, H2 and GemFire starters: http://bit.ly/2ugGK5U

   

• Add narayana-gemfire82-springboot to application's POM. Dependency is resolved from Maven Central.

  <dependency>
    <groupId>io.datalake.geode.jta</groupId>
    <artifactId>narayana-gemfire82-springboot</artifactId>
    <version>0.1.11</version>
  </dependency>

• Enable the SpringBoot Transaction Management (TM) and enable the Geode/Narayana JTA integration. Ensure that the TM is initialized first, by setting the @EnableTransactionManagement#order attribute to a small value.

  @SpringBootApplication
  @EnableGeodeNarayanaJta
  @EnableTransactionManagement(order = 1)
  public class SampleNarayanaApplication implements CommandLineRunner { }

• Use the Spring Data idioms to create and configure JPA and Geode repositories.

  interface MyJpaRepository extends CrudRepository<JpaCustomer, Long> {}
  interface MyGeodeRepository extends CrudRepository<GeodeCustomer, Long> {}
  ...
  @EnableJpaRepositories(basePackageClasses = JpaCustomer.class)
  @EnableGemfireRepositories(basePackageClasses = GeodeCustomer.class)

• Use the @Transactional Spring idioms to participate in a distributed transactions

  @Transactional
  public void addNewCustomer(String firstName, String lastName) {
    jpaRepository.save(new JpaCustomer(firstName, lastName));
    geodeRepository.save(new GeodeCustomer(666L, firstName, lastName));
  }

JTA/JCA Overview

The Java Transaction API (JTA [4]) enables global transactions across multiple two-phase commit  (e.g XA compliant) resources. It allows multiple resources (such as databases, application servers, message queues, transactional caches, etc.) to be accessed within the same transaction, thereby preserving the ACID properties.

In addition the Java EE Connector Architecture (JCA [5]) defines a standard for connecting a Java EE application server to an enterprise information systems (EIS) such as Apache Geode. Among others it defines how the EIS can participate in JTA transactions managed by the Java EE application server.

The JTA specification defines the interface between the global transaction manager and the local resource managers. Specification requires two-phase commit (e.g. XA-compliant) participants to ensure that all resources either commit or rollback, preserving the data consistently.

For non-XA compliant, one-phase commit (1PC) resources (such as Apache Geode) to be enlisted within transactions that manipulate two-phase commit (2PC) participants, the Last Resource Commit Optimization (LRCO) feature needs to be implemented by the JTA provider.

When LRCO is enabled the transaction coordinator asks each 2PC participant to prepare and if they all vote yes only then the 1PC participant is asked to commit. If the 1PC participant commits successfully then the decision to commit is logged and then commit is called on each 2PC participant. If the 1PC participant commits fail, than the global transaction is rolled back as are the transactions of all 2PC participants.

The LRCO is out of the scope of the JTA specification and therefore is an implementation-specific option. Some Java EE application servers (like WebLogic and WebSphere) provide LRCO support for JCA Resource Adapters that implement the LocalTransaction level of transaction support. But note that LRCO is “out of the scope of the connector architecture (JCA) specification” [11] as well.

Historically JTA and JCA were designed as an inseparable component of the Java EE application servers. Today the Java EE AS stack is considered obsoleted [3]. The evolving business needs, the new programing models and the raise of cloud and microservices had lead to the birth of standalone, open-source JTA implementations such as Atomikos, Bitronix and Narayana [6],[7]. Later allow embedding the JTA provider directly within one’s applications (or microservices). Furthermore the Spring Boot standalone JTA starters [8] provide out of the box integration for those standalone JTA providers.

Geode JTA support 

Apache Geode already provides two JTA integration options - both work (only) within Java EE application servers:

• The first option - Coordinating with External JTA Transactions Managers [9] - uses the javax.transaction.Synchronization callback to coordinate with an external JTA transaction manager in the Java EE containers. On each Region operation it checks with the JTA TransactionManager (TM) for running global transaction. If such is found, then Geode starts a local transaction and registers it as Synchronization callback in the global one. The JTA TM uses the synchronization mechanism to notify Geode before completion (prior to the start of the two-phase transaction commit process) and after completion (after the two-phase transaction participants have been committed). 

  This approach is unsafe and can lead to data inconsistency. The global transaction manager doesn’t control the commit order of the transaction participants and can not guarantee the atomicity of the 1PC participants (such as Geode)!

• The second option - Geode as the “Last Resource” in a Container-Managed JTA Transaction [10], sets Geode as the “last resource” while using a container as the JTA transaction manager and one-phase commit Java Connector Architecture (JCA) adapter implementation. This approach, uses the “last resource” feature implemented by some 3rd party AS containers such as WebLogic or WebSphere that allow the use one non-XAResource (such as Geode) in a transaction with multiple XAResources while ensuring the data consistency. “To accomplish this, the application server container must use a JCA Resource Adapter to accommodate Geode as the transaction’s last resource. The transaction manager of the container first issues a prepare message to the participating XAResources. If the XAResources all accept the transaction, then the manager issues a commit instruction to the non-XAResource (e.g. Geode). The non-XAResource participates as a local transaction resource. If the non-XAResource fails, then the transaction manager can rollback the XAResources.” 

  While this integration option provides a reliable JTA integration it is excessively complex to setup and operate. It also requires a commercial Java EE application server (such as WebLogic or WebSphere) and imposes severe constraints on the programing model with all inherited management, scalability and performance implications. 

• Deprecated

  Deprecated third option exists, that allows Geode to act as JTA manager itself. This approach though uses a not XA-compliant implementation that doesn’t persist the transaction state and can lead to inconsistent data on member crash. 

For all integration options, at startup Geode looks for an instance of the javax.transaction.TransactionManager bound to a well-known JNDI context name. That implies that prior starting Geode there should be a running JNDI server, with the Transaction Manager references bound. Appendix A lists the known context names used by Gedoe.

Provided integration utilities can be used either programmatically offering minimal external dependencies, or declaratively (just a single annotation) when the Spring Boot programming model is used.

Geode/Narayana JTA Implementation

Currently three, open-source, standalone JTA projects exist: Atomikos, Bitronix and Narayana. They all provide robust JTA-compliant implementations, and can easily be integrated with Gemfire using the Synchronization callback to coordinate with an external JTA transaction manager (e.g Option 1). See the SpringBoot-JTA-Atomikos-Gemfire for a demo application that does this.

But as explained in the previous chapter this approach could be unsafe if 1PC resource such as Apache Geode participate in the global transaction. The LRCO feature allows a single 1PC participant to join the transaction and still preserve the ACID properties. From the three OSS projects only Narayana provides an explicit LROC support. Therefore Narayana was selected to implement the reliable Goede/JTA integration as explained below.

Narayana is an open-source (LGPL), light-weight (e.g. out-of-container), embeddable global transaction manager. It is JTA compliant and provides an Extended XAResource Control to allows Last-Resource Commit Optimization (LRCO).

To use the Narayana LRCO feature, a resource must implement the com.arjuna.ats.jta.resources.LastResourceCommitOptimisation interface. Then this resource can be enlisted in the global transaction via the standard Transaction.enlistResource(myLRCO) method. Narayana drives this resource as last in the commit protocol, and no invocation of method prepare occurs on it. Also Narayana ensures that only a single instance of this type of participant is used within each transaction.

Narayana-Geode-Core Project

The narayana-geode-core project provides a NarayanaGeodeLastCommitResource implementation of the LastResourceCommitOptimization interface and provides utility to register this LRCO resource implementation in a global transactions via the enlistGeodeAsLastCommitResource utility method.

To use the narayan-geode-core you can add the dependency to your application POM. It is resolved from Maven Central.

<dependency>
   <groupId>io.datalake.geode.jta</groupId>
   <artifactId>narayana-geode-core</artifactId>
   <version>0.1.11</version>
</dependency>

For Geode / Gemfire version support matrix check Appending B. 

Then the following snippet illustrates how use the utility to implement reliable JTA transactions with Geode/Gemfire:

    public static void main(String[] args) throws Exception {
        // Bootstrap a standalone JNDI server
        SingletonNamingServer jndiServer = new SingletonNamingServer();
        // Bind Narayana TM to JNDI context (e.g. "java:/TransactionManager").
        JNDIManager.bindJTAImplementation();
                
        // Start Narayana JTA transaction.
        UserTransaction jta = com.arjuna.ats.jta.UserTransaction.userTransaction();
        jta.begin();
        
        // Enlist Geode as Last Resource Commit Resource. Must be called before 
        // first Geode operations
        NarayanaGeodeSupport.enlistGeodeAsLastCommitResource();
        // Create and use other global transaction resources (JPA, JMS ...).
        
        // Perform Geode operations (PUT, GET …)
        region.put("666", 666);
        // Commit the Narayana JTA transaction.
        jta.commit();
        // Stop the JNDI server.
        jndiServer.destroy();
    }

Note that only the native Geode/Narayana  APIs are used. Find the complete source code here.

The SingletonNamingServcie (from the narayana JNPServer sub-project) is standalone JNDI server and the JNDIManager.bindJTAImplmentation() binds the Narayana TransactionManger in the JNDI context so it can be discovered by Geode on startup.

If you do not call the NarayanaGeodeSupport.enlistGeodeAsLastCommitResource() inside the transaction (before the first Geode operation), then Geod will register itself as javax.transaction.Synchronization resource.

Narayana-Geode-SpringBoot project

The narayana-geode-springboot project extends narayana-geode-core to provide a seamless SpringBoot (and SDG) integration. 

To use it you need to build a SpringBoot app and add the spring-boot-starter-jta-narayana and the narayana-geode-springboot dependencies to application's POM.

  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-jta-narayana</artifactId>
  </dependency>
  <dependency>
     <groupId>io.datalake.geode.jta</groupId>
     <artifactId>narayana-geode-springboot</artifactId>
     <version>0.1.11</version>
  </dependency>

Then add the @EnableGeodeNarayanaJta annotation to enable Narayana JTA with Geode/Gemfire as Last-Resource-Commit

 @SpringBootApplication
 @EnableGeodeNarayanaJta
 @EnableTransactionManagement(order = 1)
 public class SampleNarayanaApplication implements CommandLineRunner { 
  ... 
 }

Internally the @EnableGeodeNarayanaJta starts a standalone JNDI server and binds the Narayana JTA transaction managers. The annotation also activates the NarayanaLrcoAspect that automatically enlist the NarayanaGeodeLastCommitResource implementation in all @Tranasactional transactions.

For a complete Narayana Geode SpringBoot example check the narayana-geode-springboot-example demo project.

Appendices

Appendix A: Geode JNDI Lookup names

Transaction manager JNDI context names used by Geode to lookup JTA transaction managers.

Appendix B: Supported Apache Geode / Gemfire Versions

References:

[1] Narayana JTA, Aapache Geode/Gemfire Integration Github project: https://github.com/tzolov/narayana-jta-geode-support
[2] Coordinating with External JTA Transactions Managers: https://geode.apache.org/docs/guide/11/developing/transactions/JTA_transactions.html#concept_cp1_zx1_wk
[3] Market Guide for Application Platforms: https://content.pivotal.io/analyst-reports/gartner-market-guide-for-application-platforms
[4] Java Transaction API (JTA): http://www.oracle.com/technetwork/java/javaee/tech/jta-138684.html
[5] JavaTM EE Connector Architecture 1.6: https://jcp.org/aboutJava/communityprocess/final/jsr322/index.html
[6] Ataomikos - standalone, open-source JTA server: https://www.atomikos.com/Main/TransactionsEssentials
[7] Narayana - standalone, open-source JTA server: http://narayana.io//docs/product/index.html
[8] Spring Boot embedded JTA support: https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-jta.html
[9] Coordinating with External JTA Transactions Managers
https://geode.apache.org/docs/guide/11/developing/transactions/JTA_transactions.html#concept_cp1_zx1_wk
[10] Using Geode as the “Last Resource” in a Container-Managed JTA Transaction: https://geode.apache.org/docs/guide/11/developing/transactions/JTA_transactions.html#concept_csy_vfb_wk
[11] Java EE Connector Architecture Specification : Scenarios Supported chapter 7.6.1
[-] Logging last resource system (LLR) Patent: http://www.google.nl/patents/US2008025027
[-] Recoverable last resource commit Patent:
https://patents.google.com/patent/US20080250074A1/en?q=Logging&q=last&q=resource&q=commit