The purpose of this document is to explain how microservices based architecture can help to build APIs for financial and banking sector. This document will discuss all the main layers, components and patterns related to banking, financial services and insurance (BFSI) in technical perspective.
Key components of the microservices-based architecture
API Gateway
A single-entry point with standard APIs that are exposed to external & internal partners and developers to enable access to financial data, products, and services.
Sitting in front of APIs, the gateway acts as a protector, enforcing security and ensuring scalability and high availability. To put it simply, the API Gateway takes all API requests from a client, determines which services are needed, and combines them into a unified, seamless experience for the user.
Service Mesh
The layer that service orchestration and composition happens.
These are the features/characteristics within the service mesh.
Multiple microservices can be composed into a single API.
Workflow orchestration between microservices - Example: SAGA/BPMN
Integration with internal systems, services and products. Now a day’s event driven approach has become popular for internal service communication.
Business rules can be executed as a service for relevant microservices. This segregation helps to isolate business logic from rule execution and same rules can be used by multiple microservices.
The communication between services can be made secure with various security measures such as encryption, mTLS, certificate management and access controls.
Service registry and discovery capability will help to remove that responsibility from microservice and which helps to wire services dynamically.
Also, to achieve above functionalities microservices should be integrated/communicated through various channels within service mesh. To do that we need different kinds of connectors and adaptors.
As an example, we can take the Kafka connector, Zeebe connector, Mule adapter and other integration tools.
Microservices
The key idea of the microservice architecture is functional decomposition. Instead of developing one large application, you structure the application as a set of services.
Communication Flow
External API consumers always access microservices, as APIs all the way from API gateway and service mesh. They don’t have direct access to services or service mesh but APIs defined at API gateway.
Internally there can be both API consumers and publishers. So microservices can be exposed via API gateway and via service mesh.
Scenario 01: via API gateway
Some live information needs to be exposed only to employees’ office mobile apps.
To access this data, the office mobile app will access this API through API gateway as a rest call.
Scenario 02: via service mesh
Trading platforms want to access core banking platforms and vice versa.
This can be done effectively with event sourcing communication channels via service mesh.
Good technological example for this is Kafka and Solace.
Microservices Architecture
Microservice architecture - is an architectural style that structures an application as a collection of services that are
Highly maintainable and testable
Loosely coupled
Independently deployable
Organized around business capabilities
Owned by a small team
The microservice architecture structures an application as a set of loosely coupled services. The goal of the microservice architecture is to accelerate software development by enabling continuous delivery/deployment.
DESIGN ISN’T ABOUT LOOKS; IT’S ABOUT RELATIONSHIPS
Domain-Driven Design (DDD)
Domain-Driven Design is a concept introduced by a programmer Eric Evans in 2004 in his book Domain-Driven Design: Tackling Complexity in Heart of Software.
DDD has two concepts that are incredibly useful when applying the microservice architecture: subdomains and bounded contexts.
What is domain-driven design?
Domain-driven design (DDD) is a conceptual software development approach where the business domain is the focus of development.
Domain-driven design:
places a project’s focus on domain logic
bases complex designs on the domain model
encourages collaboration between developers and domain experts to address domain problems and create high-quality software
Bounded Context
A Bounded Context defines the boundaries of services that won’t have any conflicting models inside of them.
Bounded context tries to define boundaries of our complex domain into business context. Bounded contexts are important because they allow us to define a ubiquitous language that is shared and valid within a boundary.
Ubiquitous language is a model that acts as a universal language to help communication between software developers and domain experts.
Example: Online Purchase System
The meaning of a product for the shipping bounded context is not the same as for the product bounded context.
Maintain Consistency
Shipping service and pricing service will subscribe to product service. So, when a product change happens, those two will be notified and they will update their respective product details within their bounded context.
This way each service can maintain their own product entity/table.
Pattern: Decompose by subdomain
DDD refers to the application’s problem space - the business - as the domain.
A domain is consisting of multiple subdomains. Each subdomain corresponds to a different part of the business.
Examples
The subdomains of an online store include:
Product catalogue
Inventory management
Order management
Delivery management
The corresponding microservice architecture would have services corresponding to each of these subdomains
Microservices Design Patterns
When it comes to transform from monolith to microservices, there are few patterns that we can follows. Now these patterns can be categorized into three areas.
Application Patterns
Infrastructure Patterns
Application Infrastructure Patterns
A pattern language for microservices
“A Pattern Language” is not about architecture, but about how specific design choices can help us build better relationships.
As described in above diagram, you can see the relationships between design patterns and its components.
Application Patterns
Decomposition
How to decompose an application into services?
Decompose by business capability - define services corresponding to business capabilities
Decompose by subdomain - define services corresponding to DDD subdomains
Self-contained Service - design services to handle synchronous requests without waiting for other services to respond
Data management
How to maintain data consistency and implement queries?
Database per Service - each service has its own private database
Shared database - services share a database
Saga - use sagas, which are a sequence of local transactions, to maintain data consistency across services. In a way to manage distributed transactions with microservices.
API Composition - implement queries by invoking the services that own the data and performing an in-memory join
CQRS - implement queries by maintaining one or more materialized views that can be efficiently queried
Domain event - publish an event whenever data changes
Event sourcing - persist aggregates as a sequence of events
Testing
How to make testing easier?
Consumer-driven contract test - a test suite for a service that is written by the developers of another service that consumes it.
This can be achieved with proxy services.
Consumer-side contract test - a test suite for a service client (e.g. another service) that verifies that it can communicate with the service.
In here basically testing communication channels is working from client to service and vice versa.
Service component test - a test suite that tests a service in isolation using test doubles for any services that it invokes.
This is about functional testing of service itself.
Application Infrastructure Patterns
Transactional messaging
Coordination of distributed transactions
There are two ways of coordination sagas
Orchestration - an orchestrator (object) tells the participants what local transactions to execute.
Choreography - each local transaction publishes domain events that trigger local transactions in other services
Cross cutting concerns
This is mainly about configuration management
Externalized configuration - externalize all configuration such as database location and credentials
Service Template - a template that implements standard cross cutting concerns and is intended to be copied by a developer in order to quickly start developing a new service
Observability
How to understand the behaviour of an application and troubleshoot problems?
Application metrics - instrument a service’s code to gather statistics about operations
Audit logging - record user activity in a database
Distributed tracing - instrument services with code that assigns each external request a unique identifier that is passed between services. Record information (e.g. start time, end time) about the work (e.g. service requests) performed when handling the external request in a centralized service
Exception tracking - report all exceptions to a centralized exception tracking service that aggregates and tracks exceptions and notifies developers.
Health check - service API (e.g. HTTP endpoint) that returns the health of the service and is intended to be pinged, for example, by a monitoring service
Centralized Logging – This will easily manage, store and analyse application logs to get high value from it. Elastic tech stack would be a good example for this. Troubleshooting and doing business analysis are two major benefits of this.
Communication patterns - Style
Which communication mechanisms do services use to communicate with each other and their external clients (even outside the same business domain)?
Remote Procedure Invocation - use an RPI-based protocol for inter-service communication (example: gRPC )
Messaging - use asynchronous messaging for inter-service communication
(example: pub-sub)
Domain-specific protocol - use a domain-specific protocol
Reliability
How to prevent a network or service failure from cascading to other services?
Circuit Breaker - invoke a remote service via a proxy that fails immediately when the failure rate of the remote call exceeds a threshold
https://martinfowler.com/bliki/CircuitBreaker.html
Security
As explained in the below diagram, we have used Kubernetes and Istio service mesh.
Istio’s security features involve multiple components:
Citadel for key and certificate management.
Sidecar and perimeter proxies to implement secure communication between clients and servers and to enforce policies.
Pilot to distribute the authentication policies and secure naming information to the proxies.
Mixer to manage authorization and auditing.
Infrastructure Patterns
Principles-of-Container-App-Design
Cloud-native applications anticipate failure; they run and scale reliably even when their infrastructure experiences outages.
These contracts ensure that applications they run conform to certain constraints and allow the platform to automate application management.
Build time
Single Concern: Each container addresses a single concern and does it well.
Self-Containment: A container relies only on the presence of the Linux kernel. Additional libraries are added when the container is built.
Image Immutability: Containerized applications are meant to be immutable, and once built are not expected to change between different environments.
Runtime
High Observability: Every container must implement all necessary APIs to help the platform observe and manage the application in the best way possible.
Lifecycle Conformance: A container must have a way to read events coming from the platform and conform by reacting to those events.
Process Disposability: Containerized applications must be as ephemeral as possible and ready to be replaced by another container instance at any point in time.
Runtime Confinement: Every container must declare its resource requirements and restrict resource use to the requirements indicated. The build time principles ensure that containers have the right granularity, consistency, and structure in place. The runtime principles dictate what functionalities must be implemented in order for containerized applications to possess cloud-native function.
Deployment patterns
How to deploy an application’s services?
Multiple service instances per host - deploy multiple service instances on a single host
Service instance per Container - deploy each service instance in its container
Service deployment platform - deploy services using a highly automated deployment platform that provides a service abstraction (example: Docker & Kubernetes)
External API
How do external clients communicate with the services?
API gateway - a service that provides each client with unified interface to services
High Level Conceptual Solution Architecture for Personal Banking
Technology Stack
Cloud Native Platform: Kubernetes
Service Mesh Framework: Istio
API Gateway: WSO2, APIGEE, Kong, Nginx
Event Streaming: Kafka or Solace ( Solace is ideal for department level integration)
Workflow Engine: Camunda
The below solution architecture based on above banking use case.
There is a separate workflow within each system, between different systems and products.
Products will expose their services/APIs via API gateway to banking customers.
