Beside all the features and functionalities, having a backend “stable and scalable” is extremely important. A typical backend requires a set of software modules (applications), platform (operating system), and infrastructure (hardware and networking).
An optimal backend in terms of price to performance built on an optimal composition of these above-mentioned elements. The composition of elements is basically an architecture of any system. When architecting backends we need to keep in mind that software modules, platform and infrastructure may change over time, and backends should be able to adopt the changes and we have minimum disruption in providing services. The changes are generally very high on the software side and, more likely, lowest on the platform and infrastructure level.
But let’s first start from the early beginning of the evolution of backend architectures.
Before the emergence of personal computers, workstations or servers, we used to have only mainframe computers — we are considering here the second generation of mainframe computers with a capability of being connected remotely over the Internet. The mainframe computers basically acted as remote servers where users could login and perform needed computation tasks. The mainframe computers could provide services like FTP, remote terminals: Telnet, and mail servers. Mainframe computers were intended to be quite powerful with plenty of compute power, memory and storage. This obviously came with a lot of expenses and also resource management such as power consumption and cooling solutions. The hardware was intended to run by a single operating system (OS) without any virtualization.
One of the drawbacks of the mainframe computers was that they couldn’t scale up much due to the inefficient use of capacity and very high cost tag. Beside that it was also relatively hard to develop software optimized for the mainframe computers.
Personal computers, workstations and servers changed the paradigm of mainframe computers. Not only computers become cheaper but easier to manage as well. Moore’s law keeps predicting rightly the number of transistors doubling roughly by 18 months, which generally increases the performance at the factor of 2. Since CPU and memory upgrade was quite often, mainframe computers couldn’t keep up the price to performance ratio.
The story however does not end here, Early servers were still very much like mainframe computers, having dedicated OS, and no OS level of virtualization.
The use of virtualization or virtual machines (VMs) was another step forward in terms of better use of capacity, and system management. VMs are basically simulated computer systems that run on the top of computer hardware. Though a VM can also run on the top of another VM, in other words simulation within a simulation, but the main idea is to utilize and distribute resources from the underlying system, whether it’s hardware or another VM to the upper virtual environment.
VMs became more popular as the number of cores and hardware threads increased in CPUs.This lead to efficiently running multiple OSs using VMs on the single CPUs. This gave immense benefit in better use of resources as well as insolation of application running, and fault tolerance. VMs such as vmware, KVM etc. have been the most common use of virtualization. VMs use hypervisors which make virtualization possible. There are two types of hypervisors: 1) Bare metal hypervisor, 2) OS hosted hypervisor.
In early time, many big companies like Amazon, Google, Microsoft have had a large number of their dedicated servers, and heavy use of virtualization. Since the load on their servers was not constant, the server capacity was under utilized most of the time. Companies started realizing to rent out the extra capacity of their servers to other companies. This led to the era of commercial cloud computing. Amazon was the very first big cloud provider, and then followed by google, Microsoft and many others.
In the following sections first I will try to explain different type of virtualization.
Bare metal hypervisor (process virtual machine)
A bare metal hypervisor runs directly on hardware, which means that there is no middle OS between hypervisor and hardware. The hypervisor can directly control, distribute or manage hardware resources. It is also called type 1 hypervisor.
The benefit of type 1 hypervisor is a more optimized distribution of hardware resources. Type 1 hypervisor is basically an extremely stripped down version of OS with the sole purpose to run VMs.
Advantages of Type-1 hypervisor
- A far less overhead than full OSs, and therefore can provide greater performance
- Since they are a very striped down version of OS, it’s less likely things will go wrong compared to full blown OS
- Consumes less resources in comparison
- Good level of security as less vulnerable for security attacks
- Simplicity and stability
Disadvantage of Type-1 hypervisor
- Limited features and limited control in comparison
- Overhead still exists
- Potential security issues
OS level hypervisor – Type 2 hypervisor
Type 2 hypervisor runs on OSs instead of running directly on the hardware. Type 2 hypervisor shares most of the resources from the host OS. This is probably the most common hypervisor used, when developing and testing software. It’s also commonly used on PCs where people want to run multiple OSs.
Advantages of Type-2 hypervisor
- Easy to create and manage VMs
- Host OS and VMs can switch in between seamlessly
- Faster server setup and experimentations
- Easy to create replicas of OSs
Disadvantages of Type 2 hypervisor
- Significant overhead compare to Type-1 hypervisor
- Not suitable for large scale deployments
- Less control of hardware
- Potential security risks
These hypervisors require a separate management console to perform activities like instance creation, migration, etc. The management console can be just web-based or a separate package that you can install on another machine. Using this console, one can connect to the hypervisor to manage virtual environment operations.
VMs are continued to be used at large scale. However, in recent years, another type of virtualization became very popular, namely containers. The idea of containers, such as docker containers, is considered a big leap in virtualizations. Containers emerged as mini virtual machines, though not exactly virtual machines.
The difference is that containers share host OS kernels, binaries and libraries, but do not have their own kernel. The shared components with containers are read only. Containers are very light weight, usually in megabytes, which make them quick to start, in a few seconds, whereas VMs can take several seconds to minutes. Other than that, containers reduce management overhead as they share common OS. Shared common OS has both drawbacks and advantages though, a single OS needs bug fixes and security patches, however a single point of vulnerability and failure can also occur. The hypervisor type 1 has similar characteristics in terms of single point of failure but as hypervisors are way less complex than OS.
There are several differences between VMs and containers. The one big difference is that containers provide a way to virtualize an OS without needing to fully virtualize the hardware. This enables containers portability, speed and agility.
Many applications require a backend serving multi geographic locations, tens of thousands of users, high responsiveness, security and with having all this very cost effective. Single point of failure is not desirable and therefore redundancy is needed.
Backend architectures
The focus of our discussion is designing a backend architecture, which is stable and scalable. In this section I will try to examine what backend architecture choices are available. I will also try to correlate how different architectures evolved over time with respect to virtualization technologies.
We may categorized backend architecture into three:
1 – Monolithic architecture
2 – Microservice architecture
3 – Serverless architecture
Monolithic architecture
Monolithic architecture can be seen as a single unit software application where software modules are stacked/layered together. Basically in this architecture modules and services are tightly interrelated. This may make the element of modules work together fast and more efficiently.
The efficiency only lasts until the number of modules in the blacked grows large and eventually difficult to manage. The capacity in terms of computation and memory is reserved in advance. The capacity of the resources are normally dimensioned considering the worst case scenarios, such as heavy load conditions, and therefore mostly under utilised.
Taking a typical web application example, all the modules, user interface, business logic, data layer and database are stacked together.
There is generally a misconception that monolithic architecture does not scale. This is not entirely true. Monolithic architecture can scale horizontally, which implies essentially replication of the system. Horizontal scalability also increases the overhead as well. Also scalability is generally expensive and still the overhead increases.
Monolithic architectures were closely part of traditional server setup and virtual machines.
Advantages
- Low latency among modules
- Tight integration among modules
- Fast deployment [initially]
- Easy to manage when services are in low number
- Uses one core programming language and single framework
Disadvantages
- Hard to scale in terms of individual services
- Over capacity utilization
- Single point of failure
- Difficult to maintain with large number of services
- Increasingly harder to deploy new services
- Different compute requirements for different modules
Choosing a monolithic architecture
- Small team – If you are a startup and you have a small team, you may not need to deal with the complexity of the microservices architecture. A monolith can meet all your business needs so there is no emergency to follow the hype and start with microservices.
- A simple application – Small applications which do not demand much business logic, superior scalability, and flexibility work better with monolithic architectures.
- No microservices expertise – Microservices require profound expertise to work well and bring business value. If you want to start a microservices application from scratch with no technical expertise in it, most probably, it will not pay off.
- Quick launch – If you want to develop your application and launch it as soon as possible, a monolithic model is the best choice. It works well when you aim to spend less initially and validate your business idea.
Microservice architecture
The idea behind a microservice architecture is to separate the services from one big stack of services put together such as in monolithic architecture.
In the microservice architecture the services can be run on separate containers like docker containers as explained before.
Instead of one big monolithic backend architecture where services are put together in layers, services in microservice architecture can be treated as a singular entity, and have its own infrastructure. This enables scaling much easier, as new containers can spin up in a few seconds and are ready to serve, as well as spin down.
The service management becomes easier as containers are normally responsible for few services. The separation of services also makes it easy to independently develop the services.
New services can be added in a very rapid manner, without needing to affect the whole system.
In case of crashes, containers can spin up fast. An example of this is kubernetes which orchestrates containers to facilitate the microservices. Hundreds and thousands of containers can run on a single OS.
Today several cloud providers offer fully managed container services. However, containers still require some level of management, but also provide some level of flexibility.
A drawback is that if the number of services increases, the complexity also increases. In particular when services needed to depend on each other.
Advantages
- No single point of failure
- Ease in managing services
- Application can scale easily
- Better maintainability
- Services can be developed independently
- A single application can be developed in several programming language
- Less disruption while adding and deploying new services
Disadvantages
- Complexity in terms of communication between microservices
- Latency inter function communication
- No statefulness
- Cold starts
- Running time limit
Choosing a Microservice architecture
Serverless architecture
Serverless architecture, or in short “serverless”, is the further evolution of microservice architecture. Serverless pushes the hardware and software abstraction further away. The name is though little misleading – serverless does not mean that there is no server behind. There are of course servers running behind, however they are completely managed by the cloud provider. The core idea of serverless is to focus on developing, testing and deploying applications without thinking much of servers. It’s much like pay and go services, that means you pay what you use for – don’t pay for the idle time of the service. Your virtual machines, or containers don’t need to be available all the time.
The convenience of serverless is that it does not require infrastructure management work such as cluster provisioning, patching or capacity provisioning. Everything is managed by the cloud providers.
APIs have become the most important form of value delivery:
- Serverless computing is where cloud services are fully managed. It allows you to write code to build applications without the need to manage or maintain the underlying platform and infrastructures.
- Serverless computing is universal. It integrates the capabilities of BaaS and supports all the major applications on the cloud.
- Serverless computing supports the pay-as-you-go billing method, which allows users to only pay for the resources consumed. It also supports the subscription billing method. Compared with traditional computing platforms, serverless computing allows users to select billing methods based on specific scenarios, which helps them reduce costs.
- Serverless computing is application-oriented, making it different from resource-oriented computing platforms, such as virtual machines and containers. Serverless computing integrates the cloud service system and ecosystem to help users achieve disruptive innovation in value delivery methods
Serverless is also known as FasS (Function as a Service). Seeing from the name function, serverless can let functions or methods to call and get returned the results.
Advantages
- Lower the cost of ownership – (pay & go)
- Function level managed solution
- Function as a Service (FaaS)
- Highly scalable (Known as unlimited scalability)
- Extremely low management
- Faster development
- Easy to deploy updates and upgrades
- Multiprogramming and framework support
Disadvantages
- Limited computer and memory resources compared to VMs
- Time restricted execution of the function; typically seconds to few minutes
- Cold starts; especially C# and Java, need to optimize, and keep into account cold starts
- Limited programming languages support
- Potential security issues
- Data model needs
- Infrastructure-as-code
- Fully managed
Choosing a Serverless architecture
- Both microservice and serverless architecture are in nature the same. The applications modules are not put together in layers but spread out in a distributed manner. It can be a nature fit to move from containers to serverless.
- Serverless could be very cost efficient as you pay as you use. If you don’t have traffic all the time then serverless is a good choice. It can cut the cost significantly.
- If the application’s memory and computation requirement is not extremely high then serverless can be a good fit.
- Faster development, test and getting in production.
- When load is unpredictable, and requires better use of resources.
- Serverless can suffer from cold starts. If an instance of function is accessed the first time then it may take up to a second to be in a warm state. It’s important to have right choice of programming language as some worse than others.
- Communication between functions can use message queues, which makes sure the message is delivered.
- As functions usually perform single service, it is often easier to update or make changes.
- What are the choices we have in terms of programming languages? A limitation maybe is that there are only a handful of programming language support. However, with time the more and more programming languages are being added.
- Functions have also a limit in the size of codes, including libraries. In some cases functions may not be a good choice for the large code base.
- You only pay what you use. The functions which are not in use don’t require to be paid. This can reduce the cost drastically if service is not accessed all the time.
What is the future of serverless?
Serverless is essentially a big leap for interpreted, and single threaded languages. Serverless allows us to write scalable and fast architecture without going too deep into built threaded problems. The initialization of interpreted languages, such Python or JS, are fast. Serverless functions can run in many instances in parallel and can scale based on predefined traffic, processing or other needs. It allows to control concurrency, and also allows to optimize the use of the functions.
Serverless will continue to be matured. It is expected the performance will be improved, especially cold start, the size, stability and further cost optimization. The security will be further enhanced.
There may come better support for GPU. Machine learning applications are thus severely limited by not supported GPUs.
Adoption of serverless architecture is gaining a lot of momentum. More and more companies are moving towards serverless.
Some trends regarding serverless:
Serverless will be widely adopted
When looking at the adoption rate, and improvements in serverless computing, we can predict that it will be pervasive in cloud solutions. Serverless will continue to have more programming language support, as well as more options for compute and memory resources. The trend looks very clear that serverless is going to be used widely.
Serverless and containers will be closely integrated
Serverless is considered to be the next evolution of containers, but the importance of containers will continue to exist. In many cases both serverless and containers will be complementing and improving overall performance of the system. Containers brought a lot of innovation to application development and services in respect to agile delivery, as well as brought a lot of convenience in building and delivering applications.
If containers are organized as microservice then they basically are serverless. In case a microservice is compute or GPU heavy then containers can be integrated to the whole serverless philosophy.
One should also keep in consideration that containers are also the building blocks for serverless. The improvements in containers will also benefit serverless framework.
Serverless will continue to improve price:performance
Serverless is basically pay and go service. The amount of performance per event will further increase.
Final words
Building a stable and scalable backend requires several considerations. It requires a great deal of understanding of hardware, operating system environment, cloud services and cloud computing architectures, and most importantly present and future requirements for the application.
In this article I tried to walk through the historical background of the hardware, operating system environment, virtual machines, containers and lastly the serverless computing. Hopefully, this will give an overall good understanding of how backend design evolved over the time.
Cloud computing made a huge leap in terms of performance, productivity and service delivery. This trend will continue further. In future, we will see more cloud providers coming and offering different services. Many companies already rely on multi-cloud providers. Companies tend to choose which fits their cost performance matrix.