In an always-evolving world of technology, if applications are not constantly evolving, they get outdated. Monolithic applications need to be refactored to a microservices model to reap the benefits of the cloud and enable the kind of business functionality that modern businesses demand. The Shaadi.com DevOps team has accomplished one such successful transition in recent times, which is the focus of this post.
The goal: Convert their photo-review app to SaaS
When users sign-up to a platform to find potential life partners, they probably expect verified profiles with legitimate photos and genuine people. But, manually reviewing every picture uploaded by every user to pick out the ones that are not authentic, have no person in them, are inappropriate or simply violate Shaadi.com’s policies is a mammoth task. The team at Shaadi had already built a machine-learning-based app to review these photos automatically. Now, they decided it was time to give this internal application a SaaS upgrade.
The application: What does it do?
Typically, companies have people sitting and reviewing each picture uploaded to their website. It is, obviously, a time-consuming and inefficient process. So, Shaadi.com took a technology-based solution to solve this problem. Now, they decided to go a step further and leverage not just technology, but specifically the cloud to make the application more powerful and usable. The engineers came up with what they called ‘Photo SaaS’, a SaaS version of their photo review application.
Photo SaaS is a serverless cloud-based system that has its entire infrastructure on AWS. All the components of this system are serverless and are run by AWS. Photo SaaS is aimed at providing Image Verification and Image Cropping services. It automatically reviews pictures through machine learning and picks out genuine profiles.
The transition: Monolith to microservices
Shaadi.com’s photo review app started as a monolith
Photo SaaS started as a tightly coupled unit that contained all the functional components, background processing, database operations, and business logic. This worked initially as it is very easy to build and test a basic application on a monolith infrastructure. But, as technology progresses, having a completely monolithic architecture is not sustainable.
The outdated application architecture affected the management, scalability, and the freedom to choose the right technology for the job. Adding new logic to the app required in-depth testing of the complete flow – In other words, high-risk developments.
Disadvantages of monolith
- Small changes are never just small changes. You will need to create and deploy a new build for the entire application to fix the smallest of bugs. It is a highly complicated and cumbersome process.
- Scaling is a mammoth task. Scaling a singular part of the system is impossible as you will need to scale the entire system instead. This leads to symmetric scalability instead of asymmetric scalability which means system resources end up being shared amongst services that do and don’t have scalability requirements.
- They are too rigid, thus preventing agility leading to every action taking too much effort and time.
Advantages of microservices
- The architecture is such that every service is an individual application with its programming language, database, and framework. This means that changes don’t affect the entire system and are restricted to specific components. This makes scaling a highly simple and uncomplicated process.
- They are easy to develop, modify, and be released to the market. This is because there is an opportunity for various teams to work on different components of the same system at the same time.
- With agility and ease of flexibility, microservices make it easy to adapt to new processes.
The architecture
The complete system – based entirely on AWS – resides behind the API gateway which routes the API calls to the correct workflow. AWS Cognito does the job of authentication, which means that just like a username and password authenticates someone and allows them to use a system, AWS Cognito holds user details and authenticates users. The S3 buckets in the system store the incoming images, generated crops, a database of celebrity images, etc.
The “webp converter”, “Image Upload Handler”, “event router”, “Verification workflow”, “crop workflow” and “status tracker workflows” are the microservices that use AWS Step Functions to form a neat interlacing between them. They have a small functional code written in serverless patterns in Lambda.
The “EventBridge” acts as a centralized system that helps communicate between various microservices. It does the job of routing requests to the right workflows. The EventBridge supports a lot more targets, which removes the need for unwanted glue code to connect the services. It has a cross-account delivery capability which amplifies its reach, and further, it supports content-based filtering and input transformation.
SQS can act as an alternative to the Event Bridge with the limitation being that it can merely be a queuing system that holds messages instead of redirecting them.
List of AWS services used & how they work together
(a) Amazon Rekognititon
(b) AWS Lambda
(c) AWS Cognito
(d) AWS S3
(e) AWS Step functions
(f) Amazon EventBridge
When an image is uploaded in the S3 bucket, a Lambda gets triggered to read the metadata of the image, the config file of the client, and find out what all services they have enrolled for. Say, they have enrolled for a verification service, the Lambda then needs to communicate to the verification workflow that verification is required. So, Lambda generates an event that is read by the EventBridge and gets redirected to the appropriate workflow. Amazon Rekognition helps verify images.
The method: machine learning models
AWS Rekognition
Shaadi.com uses Amazon Rekognition as well as in-house data science models for image verification. Amazon Rekognition has APIs which enable verification of images based on simple rule checks that are keyed in programmatically. Although this concept has existed for a while, it used to be a monolithic code and has now been updated to a microservice code.
In-house ML models
Image Cropping uses in-house logic and helps generate images to fit into the screen of your choice. Amazon Rekognition and its ML services are used to detect a face in the image and its dimensions. Then, the in-house Image Cropping logic can figure out how to capture a bigger area around that image. It can generate images in JPEG, webp, and other dimensions. The in-house Cropping logic also has a photo blurring capability.
Photo verification, image cropping, etc. have different microservices and are highly decoupled so they can run individually, without interfering with each other.
The reward: A potential new product
The idea is to release Photo SaaS externally and make it available in the cloud so that others can consume it as well. The engineering team not only built a good service to handle their photo reviews for Shaadi.com but they also have a potential product they can offer as a paid SaaS application for other companies that do similar things. By commodifying something built for an internal purpose and selling it as a product itself, the engineers at Shaadi.com have essentially created a possible small revenue stream for the organization.
A recap
The journey of the Shaadi.com Engineering & DevOps teams going from monolith to microservices is an interesting case study that highlights the advantages of the microservices style of infrastructure. One of the most significant things to note is that Monolith is tightly coupled and microservice is loosely coupled. The transition from monolith to microservices needs to be a well-planned and strategic one and the Shaadi.com Engineering & DevOps team has nailed it. This post serves as a great example of how an engineering team not only solved internal problems but potentially created a new product. And if this journey of the engineering team at Shaadi.com regaled you, and you would like to do work like this, come join us. We’re hiring!
