Sync Setup Notes and Example

Default example in OFBiz

The example in OFBiz (in specialpurpose/pos/data/PosSyncSettings.xml) is designed for a 3 levels architecture, where Per Store Servers (PSS) are used between POS terminals and the Master Central Server (MCS).
Here is a short explanation on how it works
This setting is intended to synchronise a main database (on the MCS) with the databases of local stores servers (PSS) each of them being synchronised with their local POS terminals. Each of these OFBiz instances use a database. Store here means a physical store where POS terminals are.

There are 4 types of flow :

• MCS -> PSS
• PSS -> POS
• POS -> PSS
• PSS -> MCS

Where

• MCS is a central OFBiz server (with one database, the central database where all products information are maintained and where the consolidation is done)
• PSS is an instance of a server store (with one database, the store database where all store inventory informations are stored)
• POS is an instance of a POS terminal (with one database, the POS database where terminal information are collected)

• The flow MCS -> PSS is used to update catalogs, from the central database where the main catalogs are maintained, to the stores. This include roughly pricing, promotions, basic product info, etc.
• The flow PSS -> POS is used to roughly update catalogs of the POS terminals
• The flow POS -> PSS is used to roughly update orders, payments , etc. of the store
• The flow PSS -> MCS is used to roughly update orders, payments, inventory, etc. in the main database.

This scheme is very powerful and help you to extend your net of stores has needed. Remember that this is only an example and you can extend it from your needs.

Using Entity Synchronization

The general idea behind the Entity Sync stuff is to re-create transactions from a central database into one or more other databases, each represented by a set of record in the EntitySync and related entities.

The four fields (lastUpdatedStamp, lastUpdatedTxStamp, createdStamp, createdTxStamp) are added to each entity and automatically populated so that queries can be done on each entity to get all records created, updated, or removed in a certain transaction, and so that in time order those transactions can be reproduced remotely to get the data into the remote database in the proper order.

The enable-lock attribute on the entity element in the XML entity definitions only loosely relates to entity synchronization. This optimistic locking mechanism uses the lastUpdatedStamp of the entity to be stored and the current entity in the database to see if it was modified by another process before the current thread gets to it. If it was modified between the read and write an exception is thrown. This check is only done if enable-lock="true".

The no-auto-stamp attribute is used to tell the Entity Engine not to automatically add these stamp fields, and not to automatically set the stamps when creating and updating the entities. If it is not set to false, or if it is set to true then these values will be automatically set, and when records are removed the primary key will be saved in the EntitySyncRemove entity's table so that the record can be removed in the remote database.

When runEntitySync is called you pass an entitySyncId that is used to look up an EntitySync record and its corresponding EntitySyncInclude records to see what should be sync'ed. The package on the EntitySyncInclude refers to the package-name attribute on the entity definition XML element, and partial package names can be included with an implied wild-card at the end. The EntitySyncHistory entity is used to store information about what has happened in each entity sync run, which means one runEntitySync call.

For the remote call to save the sync data, which protocol to use depends on your requirements. The most efficient by far is the RMI remote dispatcher. If you only have port 80, or another HTTP port open, then the HTTP remote dispatcher may be your only choice.

When a runEntitySync call fails the runStatusId will stay in ESR_RUNNING. When the server crashes or the process is interrupted in some other way the thread has no real way to reset the status. This needs to be manually monitored and reset when necessary to ESR_NOT_STARTED or some other status that will be picked up for running. It could perhaps be automatically resetted...

Default example in OFBiz

The example in OFBiz (in specialpurpose/pos/data/PosSyncSettings.xml) is designed for a 3 levels architecture, where Per Store Servers (PSS) are used between POS terminals and the Master Central Server (MCS).
Here is a short explanation on how it works
This setting is intended to synchronise a main database (on the MCS) with the databases of local stores servers (PSS) each of them being synchronised with their local POS terminals. Each of these OFBiz instances use a database. Store here means a physical store where POS terminals are.

There are 4 types of flow :

• MCS -> PSS
• PSS -> POS
• POS -> PSS
• PSS -> MCS

Where

• MCS is a central OFBiz server (with one database, the central database where all products information are maintained and where the consolidation is done)
• PSS is an instance of a server store (with one database, the store database where all store inventory informations are stored)
• POS is an instance of a POS terminal (with one database, the POS database where terminal information are collected)

• The flow MCS -> PSS is used to update catalogs, from the central database where the main catalogs are maintained, to the stores. This include roughly pricing, promotions, basic product info, etc.
• The flow PSS -> POS is used to roughly update catalogs of the POS terminals
• The flow POS -> PSS is used to roughly update orders, payments , etc. of the store
• The flow PSS -> MCS is used to roughly update orders, payments, inventory, etc. in the main database.

This scheme is very powerful and help you to extend your net of stores has needed. Remember that this is only an example and you can extend it from your needs.

In the commented examples the MCS update the PSS, hence POS catalogs, every 2 hours. And the POS terminals update MCS database about orders every 5 minutes .

Another example with some questions (extracted from the OFBiz wiki archive)

EntitySync configuration

configuring peers - 'HOW ?'
configuring what entities to sync - 'HOW ?'

Say one wants to set up a few servers in the following way:

HOSM : Head-office of master-data (maintained there)
HOST : Head-office of transactional-data (collected from sites)
BOS1, BOS2, ... BOSx: Back-office (many small sites)

(most important) flows:

HOSM => HOST, BOSx: Product, ProductCategory, Facility, Parties for the facilities (and at least 50 other Entities)
BOSx => HOST: Order, and Party, Person, ContactMech, etc... related to the orders (and some other)

Filtering what to sync

With the above approach it is possible to configure at entity level: fully sync some entities and not sync others. If the HOSM and HOST were merged to a HOS (which I don't like for other reasons), there would be a need to filter (at least) Party,Person records:

HOS => BOSx : 'only send Person, Party, ContactMech records that are related to Facility (not those objects that are related to Order originated at other BOSy)'.
BOSx => HOS : transfer everything, easy

Would this be possible with current implementation of EntitySync? How is this configurable? Note that even if referred objects are synced automagically (to preserve foreign-keys):

syncing Facility (HOS => BOSx direction) could bring the Party objects referred by the Facility objects
but Party does not refer to Person (because of the direction of the relation: Person has the partyId), but the Person, ContactMech objects (that belong to the Facility) would be desirable to sync as well

configuration on Entity level: does not support the above filtering
configuration on Entity and entity-relations level: entity-relations could be switched to 'follow / not follow' (naturally, there is no choice for direct references, those are always "follow" to preserve foreign key integrity). These flags would determine if the related objects are to be synced or not. Note: final decision is for Objects, not for classes (all objects of an Entity)In the commented examples the MCS update the PSS, hence POS catalogs, every 2 hours. And the POS terminals update MCS database about orders every 5 minutes .

A simple but more detailled example

...