It is very important to have an architecture-&-design blueprint for your application or platform. One can understand the importance of architecture-&-design from builders-&-developers in the building construction area, who developed the Burj-khalifa (830 mts height & 163 floors). It is impossible to develop such a massive building structure without having the architecture-&-design blueprint in advance, if one tries the same without the blueprint he may succeed initially, may-be initial floors, but scaling up-to 163 floors, providing stability against desert stormy wind & earth-quake etc, will add the risk & life-threat to people of surroundings as well.
Considering the architecture benefits and developer’s struggle in preparing a good architecture, I thought to share a standard-architecture-template, which can be reused in any domain which has covered the aspects of Scalability, High-availability, Stability, Resiliency inside.
#architecturebuildingblock, #ABB, #architecturaldesign #architects #startup #softwarearchitecture #softwareengineer
System architecture : standard architecture building blocks (ABBs)
Why this article?
I have been engaged in architecture-design of several large-&-complex projects, these projects have been in diversified areas including telecom, retail, education, health, manufacturing segments including design of aggregator-platforms with multi-tenancy capabilities.
During this architecture development & transformation journey, I interacted with many individuals, groups & startups and learned how-much they struggle to build a good application architecture for their project. I noticed that they just focus on requirements & coding, on the architecture-&-design front either they don’t focus or they struggle to achieve. It is important to define the architecture evolution/roadmap journey at organization level or lower level which could be at project/application/organization-function level.
Let’s talk about a few important architecture-&-design artifacts as follows, which organization having long-term vision should have or get it done from their teams.
- Requirement specification document has business use-case, process-flows, actors in the process, UX prototypes, business rules & validation, etc.
- Overall architecture-&-design document has functional-architecture, system-architecture, integration-architecture, deployment-architecture, database architecture, various components hosting approach, security guideline, performance KPIs & engineering, NFR criterias & strategy, CI/CD strategy, QA strategy, architecture style & design patterns, etc.
- Consolidated-design-document consists of solution approaches for all customer journeys, impacted components, integration API specification, data models, test-cases, issues & assumptions.
It is very important to have an architecture-&-design blueprint for your application or platform. One can understand the importance of architecture-&-design from builders-&-developers in the building construction area, who developed the Burj-khalifa (830 mts height & 163 floors). It is impossible to develop such a massive building structure without having the architecture-&-design blueprint in advance, if one tries the same without the blueprint he may succeed initially, may-be initial floors, but scaling up-to 163 floors, providing stability against desert stormy wind & earth-quake etc, will add the risk & life-threat to people of surroundings as well.
Being an architect, we understand the architecture-&-design blueprint will vary from large-&-complex project to simple project. Scalability, High-availability, Stability, Resiliency are key design principles while designing the architecture-&-design blueprint for any application or platform. Be it simple or complex, small or large, architecture-&-design principles will remain the same. A large-&-complex application-or-platform can consists of several small & simple applications e.g. Telecom Billing & CRM platform can have small applications like customer-information-management, partner-or-dealer-management, order-management, inventory-management etc. if one watches closely will realize that each small application follows the
Considering the architecture benefits and developer’s struggle in preparing a good architecture, I thought to share a standard-architecture-template, which can be reused in any domain which has covered the aspects of Scalability, High-availability, Stability, Resiliency inside.
System architecture reference diagram
Purpose of each architecture building block (ABB)
| 1 | Channel-layer | Channel layer will act as the first touchpoint towards the application or platform. This layer is also called omni-channel layer, usually companies want to have unified customer experience across all omni-channels. Mobile app, web-app, SMS, USSD, Social sites, partner’s application and so on, there could be several channels for end-customers to interact with the application or platform. |
| 2 | Load-balancer | Load-balancer will act as a router & equal distributor of requests from the channel towards the next level architecture building blocks (i.e. web-server, service layer & queue layer). |
| 3 | Web-server | Web-server caters the web-page request i.e. based on web-page required content, web-server will pull data from backend, render the page with data and forward the rendered page to the user’s machine browser. Example of web-servers are Apache HTTP server, NGINX Server, IIS server, etc. |
| 4 | Queue-layer (asynchronous request) | To process customer requests in asynchronous manner to optimize spike during peak hour and to also avoid long block threads e.g. JMS Queue, Apache Active MQ, Rabbit MQ, etc. Queues can be planned for Incoming traffic, Outgoing traffic and internal traffic itself. If the particular request processing requires complex computation at backend (e.g. multiple application integration, database tables fetch, business validation, transformation, request processing at 3rd party systems, publishing messages on various downstream systems etc.), it is recommended to opt for an asynchronous option e.g. spinning a virtual machine on cloud platform, buy a new mobile-connection with entertainment services like netflix, amazon-prime & Disney+ hotstart and so on. |
| 5 | Direct from Load balancer to Application API layer (synchronous request) | To process customer requests in a synchronous manner. If the request processing does not create a long-blocking thread (i.e. the particular request processing does not require multiple application integration, database tables fetch, business validation, transformation, etc.), it is recommended to opt for a synchronous option for faster response e.g. get my outstanding balance, get my current plan. |
| 6 | Cache-layer | Cache-layer is to keep frequently fetched information and will be used to enhance the overall application performance. Cache-layer is the optional component and should be considered if there is a need for faster response from application or platform. |
| 7 | Application API layer | Application API layer is the actual processing engine for any kind of request coming from the channel-layer. This works as a hosting layer for APIs (which are developed on the ecosystem) which will be consumed internally/externally and synchronously/asynchronously. API complexity ∝ Number of computation tasks in the API APIs can be simple or complex in term computation i.e. complexity from simple-to-complex directly proportional to number of computation tasks in the API.To process customer requests in a synchronous manner to respond faster. APIs can be hosted in Apache Tomcat, RedHat JBoss server, Oracle WebLogic, Kubernetes Docker Cluster dependending on the need of a virtualized or containerized hosting. |
| 8 | Database-layer | Database-layer is persistence layer for the reference data & transaction data. Database-layer will be used for read & write operations of relevant data. Database-layer is one of most critical ABBs, depending upon the load of read & write operation and performance expectation, segregation of read & write operation should be done. Architecture of the database-layer itself plays an important role in the efficient-&-performant system design. |
| 9 | ODS (MIS Reporting) layer | Now-a-days none of the businesses can survive without a good MIS-reporting layer. Usually companies implement different kinds of DWH solutions depending on their budget and criticality. ODS is called an operational data store which can be considered as a basic alternate of complex DWH solutions. ODS is similar to database-layer only but this is dedicated for MIS reporting purposes. If there are a huge number of requests from channel-layer, it is not recommended to have a common database for your end-customer’s request processing and internal MIS reports generation. ODS-layer is replicated from main-database-layer, there are various approaches of data-replication from main-database-layer to ODS-layer. |
| 10 | Audit History Log Layer | Audit-history-log-layer is a very important ABB in the ecosystem which enables the companies to visualize the historical changes on their business data. There could be various approaches to implement audit-history e.g. write a custom-audit-layer which will hold the min information about data-changes and consolidate the sequence of changes whenever required. The simplest approach could be to have a history database-layer for each or relevant database entities with the same entity structure including history date-time stamp and consolidate data to show the historical change history whenever required. |
| 11 | Observability layer | Observability-layer is to keep centralized logging from every ABB and establish log-correlation for debugging purposes. Observability-layer will hold the logs in unified formation from each layer. Let’s understand this with one example, suppose one request is sent from the end-user’s browser which may pass-through from load-balancer, web-server, application-API-layer, queue-layer, cache-layer and others. Considering this example every ABB’s log will be captured in the unified-log-format on the observability-layer (unique-correlation-id will be assigned for each request from channel), in case of issue with any request, observability-layer will leverage the companies to provide consolidated logs by correlation-id. |
| 12 | Backup & recovery | Backup-&-recovery-layer to restore the database snapshot in case of main-database crash due to any reason. Backup-&-recovery-layer will also define a policy to restore application from archive in case of any deployment issue. |
| 13 | DevOps-layer | DevOps is not the ABB which will participate in the request processing of requests coming from channel-layer, but this has become an important ABB which will automate the source-code version management, apply static code analysis, perform code-coverage & test-coverage checks and deployment on relevant environment depending upon %-coverage-pass policy. |
Understanding the end-to-end flow in system architecture
In order to understand system-architecture, let’s take a few examples like below to correlate with architecture diagrams.
- Get new mobile connection,
- Buy OTT subscription (individual or bundled)
- Change mobile plan (from individual plan to Family pack of 1100 subscription fee monthly)
- Recharge your subscription (e.g. Prepaid mobile, Satellite TV, etc.)
- View outstanding bill or prepaid balance or wallet balance etc.
Considering all above examples, let’s understand how requests will flow from NORTH to SOUTH. For the sake of simplicity, complex or exceptional scenarios are not being considered in the explanation.
- End-users will initiate requests from channel-layer i.e. respective applications (mobile-app or channel-app) will send relevant requests to API-Gateway.
- API-Gateway will apply throttling rule or transformation rule depending on business needs, and will forward requests to load-balancer.
- Load-balancer distributes the traffic evenly among the API hosting layer e.g. if you have cluster-setup with managed-servers of JBoss AS, Weblogic, Tomcat WS etc. for API hosting and cluster-setup with managed-servers of Apache HTTP servers, NGINX servers etc. for web-page hosting, load-balancer will configure the route from front-public-ip to backend-managed-server-ip and request will be routed via front-public-ip to any one backend-managed-server-ip. Load-balancer is configured with a relevant distribution-algorithm, so that all managed-servers will even request-distribution.
- If this is web-page request, request will flow from load-balancer to web-server-cluster
- If this is API request, request will flow from load-balancer to application-API-cluster
- Since web-page request will make various API calls, so those call can be direct to application-API-cluster-layer i.e. bypass from load-balancer-layer or API-gateway-layer
- Now the request from the end-user has arrived at application-API-cluster-layer which is an actual application-processing-engine, based on use-case implementation needs, the particular API will interact with queue-layer, cache-layer, database-layer, downstream-layer, observability-layer.
System architecture for relevant environments
This is the second area where customers don’t focus much on having a cleaner environment strategy and most of the time they go with just two environments: development & production environments. With two environments strategy, one could survive in initial days of business, with business growth, frequent changes in requirements, product configuration, offer-strategy, etc. it is important to have dedicated environments for each software delivery stage. DevOps will enable the companies to have CI/CD pipeline to automate their deployment process including code-coverage and test-coverage reports.
Now again the question comes to what should be system-architecture for each environment, well this will again depend on company budget and business-criticality. Each environment should have the same ABBs with different NFR options. Most likely non-production will not require scalable & highly-available infrastructure, in that case configuration size can be reduced to fulfill the basic needs of business.
About author
Profile : Rajesh Verma – Brief profile
Source : link for this article here
Series : S1 (Architecture & design)
Episode : S1E1 (System architecture : standard architecture building blocks (ABBs))
Author’s approach : Rajesh wants to share his learning & experience gained throughout his career from various sources. Author started the series on architectural topics and this article is the first episode in that attempt. Author feels that lots of information is available on various forums, but scattered here & there. Episodes in this series will be designed for most of the relevant topics in architecture-&-design, published gradually and organized in logical sequence. Principally episodes will have linkage with other episodes, so that readers can have proper connection among the topics and would be able to correlate with ongoing activities in their software life. Topics for example will be related to functional architecture, integration architecture, deployment architecture, microscopic view of mostly architecture-building-blocks (ABBs), security guidelines & approach to comply, performance KPIs & engineering, git branch & DevOps enabled automation strategy, NFR aspects (e.g. scalability, high-availability, stability, resiliency, etc.), commonly used architecture styles & design patterns, cloudification approaches, multi-tenancy approach, data migration, channel-cutover & rollout strategy, process standardization & simplification, greenfield rollout & brownfield transformation journeys, etc.
Thank you for reading the post, please stay connected.
No responses yet