Anatomy of Micro services based & distributed Master Data Management Architecture

Please note this is improved version of previously[1] explained architecture.

[1] http://dhanuka84.blogspot.com/2017/06/master-data-management-for-distributed.html

Master Data Change Detection

1. Admin Panel will change or add master data.
2. Changes goes to DB.
3. Update Hibernate First Level Cache.
4. Distribute data via Kafka.
5. Kafka Consumer detect master data change notification.
6. Update Hazelcast Distributed map with which master data has been changed.
7. Scheduler application task read from HZ map and get to know about which master data changed.
8. Get latest data for changed master data from master data micro service.
9. Update LocalEntityMap with business key and Entity.
10. Update Business key to Primary key map.
11. Removed master data notification record from HZ distributed map.

Scheduler Application Load Balancing

1. Get Business key to primary key mapping to all master data.
2. Update Business key to primary key mapping
3. Read Business key to primary key map then load balanced among HZ cluster members.
4. update HZ distributed map.
5. Keep business keys which load balanced to local application node.

Processing using master data within scheduler task

1. Read from HZ distributed map and get master data local to this node for processing and Keep them for processing.
2. Take business keys to process.
3. Get locally cached master data for business keys for processing.
4. If can’t find in the cache get from micro service.

Master Data Management for Distributed Applications

Requirement Criteria

1. Self Load Balancing.

There are 100 products and those products need to be processed by each node without duplicating.

Example:


There are two nodes which need to process all the products as 50 products each at a time. So Node A will process 50 products while Node B will process remains 50 products.


2. High Availability and Fault Tolerant

If one node goes down remains should process all the products.

Example:

Let's say out of two nodes one nodes crashed so remain other node need to process all the 100 products until crashed node get recover.

3. Local (L1 cache) cache should be maintained by each node.

By doing this we can save unwanted round trips to Database for master data reading. Same time each node only need few Database connections to get master data from centralized DB. This is a micro services friendly approach when you use centralized DB for master data management.

4. Once master data got changed that should be reflected (synched) in each Local Cache.

Each node should process with latest data.

5. Distributed Map should maintain bare minimum information within its cache.

This will help to maintain performance and stability of distributed Map.


Architecture Explained

1. We used Hazelcast as a distributed in memory cache.

Using hazelcast distributed map we share all the products among the nodes.

2. Again we use Hazelcast for application clustering so each node will act as hazelcast cluster node. All the cluster communication and cluster management will done by hazelcast itself.

3. We used Hibernate as an ORM tool.

Hibernate Session act as a Local Cache so basically we just have to use it's API for L1 cache management.

4. In here Kafka act as data pipeline for the whole architecture.

So once master data got changed by Node.JS admin panel it will be sent to Kafka as a JSON.


Node which enabled DB write permission will update DB with necessary changes and same time update Distributed map with business key and primary key.


5. Each hazelcast cluster node acting as a consumer for Kafka under different consumer group.

Hazelcast Cluster == Kafka Consumer Group

So one of cluster member which disabled DB write permission will consume same JSON and updated relevant distributed map with master data record business key and primary key. By doing that rest of the members in the same consumer group (cluster) will get to know the changes and updated their L1 cache after reading changes from DB.

Master Data Management & Application Self Load Balancing

GREEN ARROWS :  From the Admin panel, user performs Create, Update, and Delete, or CRUD operations using Rest service provided. In here we are using JSON as payload format.

RED ARROWS : Once the application receives an admin request, the application makes requests to do write operations.Those changes are first applied to L1 cache, then L2 cache, and finally into MySQL DB.

BLUE ARROWS:  Once the application receives an admin request, the application makes requests to do read operations. The application will search the L1 cache first and then move on to L2 and finally mySQL directly if it cannot be found in each subsequent location..

YELLOW ARROWS: Application even can ignore L1 and L2 cache and directly call MySQL DB.

BLACK ARROWS : Cluster communication.

• Please note that applications can work without L2 cache as well.

Application Load Balancing

Hazelcast is used for application clustering and we are using a distributed map to share productId information among the applications which are run inside the cluster. That way, each application knows what product Ids that need to be processed.

Ex: 100 Products process within two nodes application cluster.

Each node processes 50 products and those 50 products also can be processed in parallel within each node.

Source Code:
https://github.com/dhanuka84/distributed-applications