Microservices Architectures - Introduction to Auto Scaling


NEW COURSE RELEASED! - 21 HOURS, 200+ VIDEOS - Master DevOps with Docker, Kubernetes and Azure DevOps


In this article, we focus our attention on the Dynamic Scaling or Auto Scaling. Why do we need applications that can auto scale?

You will learn

  • What is Auto or Dynamic Scaling?
  • Why is Dynamic Scaling important in a microservices context?
  • How can you implement Dynamic Scaling in Cloud?

Free Courses - Learn in 10 Steps

Cloud and Microservices Terminology

This is the fifth article in a series of six articles on terminology used with cloud and microservices:

Load On Applications Varies

Load on your applications vary depending on time of the day, the day of the month or the month of the year.

Take for instance, amazon.com. It has very high loads during Thanks Giving Period, up to 20 times the normal load. However, during the major sports events such as Superbowl or a Football World Cup, the traffic could be considerably lesser - because every body is busy watching the event.

How can you setup infrastructure for applications to manage varying loads?

It is quite possible that the infrastructure needed to handle 10x the load, is ten times that needed to handle 1x the load.

If you have on premise infrastructure, you need a large infrastructure in place to handle peak load.

During periods with lesser load, a lot of infrastructure would be sitting idle.

Cloud To The Rescue

That’s where cloud comes into picture. With cloud, you can request more resources when the load is high and give them back to the cloud when you have less load.

This is called Scale Out (create more instances as load increases) and Scale In (reduces instances as load goes down)

How do you build applications that are cloud enabled - applications that work well in the cloud?

That’s where a microservices architecture comes into picture.

Introducing Auto Scaling Example

Building your application using microservices enables you to increase the number of microservice instances during high load, and reduce them during less load.

Consider the following example of a CurrencyConversionService:

image info

image info

The CurrencyConversionService talks to the ForexService. The ForexService is concerned with calculating how many INR can result from 1 USD, or how many INR can result from 1 EUR.

The CurrencyConversionService takes a bag of currencies and amounts, and produce the total amount in a currency of your choice. For example, it will tell the total worth in INR of 10 EUR and 25 USD.

The ForexService might also be consumed from a number of other microservices.

Scaling Infrastructure To Match Load

The load on ForexService might be different from the load on CurrencyConversionService. You might need to have different number of instances of CurrencyConversionService and ForexService. For example, there may be two instances of the CurrencyConversionService, and five instances of the ForexService:

image info

image info

At a later point in time, the load on CurrencyConversionService could be low, needing just two instances. On the other hand, a much higher load on ForexService could need 50 instances. The requests coming in from the 2 instances of CurrencyConversionService are distributed across the 50 instances of the ForexService.

That in essence, is the requirement for auto scaling - dynamically changing number of microservice instances, and evenly distributing the load across them.

Implementing Auto Scaling

There are a few important concepts involved in implementing auto scaling. The following sections discuss them in some detail.

Naming Server

Naming servers enables something called location transparency. Every microservice registers with the naming service. Any microservice that needs to talk to another microservice, will ask the naming server for its location.

Whenever a new instance of CurrencyConversionService or ForexService comes up, it registers with the naming server. image info

When CurrencyConversionService wants to talk to ForexService, it asks the naming server for available instances.

Implementing Location Transparency

CurrencyConversionService knows that there are five instances of the ForexService.

How does it distribute the load among all these instances?

That’s where a load balancer comes into the picture.

A popular client side load balancing framework is Ribbon. image info

Let’s look at a diagram to understand whats happening:

image info

As soon as any instance of CurrencyConversionService or ForexService comes up, it registers itself with the naming server. If the CCSInstance2 wants to know the URL of ForexService instances, it again talks to the naming server. The naming server responds with a list of all instances of the ForexService - FSInstance1 and FSinstance2, and their corresponding URLs.

The Ribbon load balancer does a round-robin among the ForexService instances to balance out the load among the instances.

Ribbon offers wide variety of load balancing algorithms to choose from.

When to increase and decrease instances?

There is one question we did not really talk about.

How do we know when to increase or decrease the number of instances of a microservices?

That is where application monitoring and container (Docker) management (using Kubernetes) comes into the picture.

image info

An application needs to be monitored to find out how much load it has. For this, the application has to expose metrics for us to track the load.

You can containerize each microservice using Docker and create an image.

Kubernetes has the capability to manage containers. Kubernetes can configured to auto scale based on the load. Kubernetes can identify the application instances, monitor their loads, and automatically scale up and down.

Do check out our video on this:

image info

Summary

In this article, we talked about auto scaling. We look at important parts of implementing auto scaling - naming server, load balancer, containers (Docker) and container orchestration (Kubernetes).

What to Learn Next?

NEW COURSE RELEASED! - 21 HOURS, 200+ VIDEOS - Master DevOps with Docker, Kubernetes and Azure DevOps


350,000 Learners are learning everyday with our Best Selling Courses : Spring Boot Microservices, Spring, Spring Boot, Web Services, Hibernate, Full Stack React, Full Stack Angular, Python, Spring Interview Guide, Java Interview, Java Functional Programming, AWS, Docker, Kubernetes, PCF, AWS Fargate and Azure


85,000 Subscribers are learning from our Free Videos on YouTube : JSP Servlets, Spring, Spring Boot, Spring MVC, Hibernate, Eclipse, Maven, JUnit, Mockito, Full Stack - React, Full Stack - Angular, Docker, Kubernetes, AWS, AWS Fargate, PCF and Azure


Here are the recommend articles to read next : Spring Interview Questions, Spring Boot Interview Questions, Microservices, Hibernate, Spring Security, REST API with Spring Boot, Full Stack with React, SOAP Web Services, Exception Handling, Embedded Servers, Spring Data Rest, Spring vs Spring MVC vs Spring Boot, Building Web Application and Spring Data JPA. You can checkout all our 100+ articles here - All Articles.


Do not know where to start your learning journey? Check out our amazing learning paths: Learning Path 01 - Spring and Spring Boot Web Applications and API Developer, Learning Path 02 - Full Stack Developer with Spring Boot, React & Angular, Learning Path 03 - Cloud Microservices Developer with Docker and Kubernetes, Learning Path 04 - Learn Cloud with Spring Boot, AWS, Azure and PCF and Learning Path 05 - Learn AWS with Microservices, Docker and Kubernetes


Related Posts

Docker Tutorial for Beginners - with Java and Spring Boot

Learn about Docker - What is Docker? Why Is Docker Popular? How to create Docker Image for a Java Spring Boot App?

Devops Tutorial | DevOps with Docker, Kubernetes and Azure DevOps

What is DevOps? How is it different from Agile? What are the popular DevOps Tools? What is the role of Docker, Kubernetes and Azure DevOps in DevOps. Let's get started with a simple usecase.

Deploy Java Spring Boot Applications to AWS, Azure, GCP with Docker and Kubernetes

In this article, we focus our attention on the cloud. How to learn the cloud and deploy Java Spring Boot Applications to AWS, Azure, GCP with Docker and Kubernetes?

Microservices Architectures - Non Functional Requirements - Performance

In this article, we explore an important non functional requirement called performance.

Spring Boot Tutorials for Beginners

At in28Minutes, we are creating a number of tutorials with videos, articles & courses on Spring Boot for Beginners and Experienced Developers. This resources will help you learn and gain expertise at Spring Boot.

Microservices with Spring Boot and Java - Part 1 - Getting Started

Let's learn the basics of microservices and microservices architectures. We will also start looking at a basic implementation of a microservice with Spring Boot. We will create a couple of microservices and get them to talk to each other using Eureka Naming Server and Ribbon for Client Side Load Balancing. In part 1 of this series, lets get introduced to the concept of microservices and understand how to create great microservices with Spring Boot and Spring Cloud.

20+ Spring Boot Projects with Code Examples

At in28Minutes, we have created more than 20 projects with code examples on Github. We have 50+ articles explaining these projects. These code examples will you learn and gain expertise at Spring Boot.

REST API Best Practices - With Design Examples from Java and Spring Web Services

Designing Great REST API is important to have great microservices. How do you design your REST API? What are the best practices?

Index - 500+ Videos

At in28Minutes, we are creating a number of tutorials with videos, articles & courses on Spring Boot for Beginners and Experienced Developers. Here's a list of video tutorials and courses for you