Data is the lifeblood of a business and your database is the heart that pumps it to the entire body of the organization. As everything expands, how do I efficiently reorganize the structure of my systems to make sure everything is running with maximum efficiency and minimum complexity?
Oren Eini, CEO of RavenDB, will talk about the evolution of a monolith architecture to microservices, and the benefits of localizing roles in your IT systems to microsystems to redistribute complexity while boosting productivity, efficiency, and capacity.
Microservices Databases and Monolithic Architecture
Monolithic Architecture vs Microservices: Advantages and Challenges
For the most part of the last half century, the monolith has been the dominant architecture for computer systems. Technology has advanced to allow for the complexity of microservices to become manageable so the advantages of the newer architecture now clearly outweigh the challenges.
A monolith system is one that does everything. This makes it a Jack of All Trades, but all too often a Master of None. While you only have one place to go for your answers, there are a number of vulnerabilities.
Uneven load distribution: One process in your system can be overloaded with work while other processes can lie fallow. Capacity may be overprovisioned in one place and underprovisioned in another. This costs time, money, and upkeep.
Updates cannot be on the fly: Updating one part of the system means the entire system might have to go down during the update.
Systemwide Issues: If there is a failure in one part of the system, the entire system can go down until that issue is resolved.
Entanglement: In a monolithic database, every component in the monolith touches the same set of tables. There will always be union between multiple tables in a monolith so to read and write to one means to read and write to many. If you need to make a change in the system, you have to scan the entire system to make a change. That gets really time expensive.
Microservices as the Solution to the Monolith Challenges
The answer to all of these challenges is to simply break up your monolith into independent pieces.
You can fix load issues by provisioning more capacity to specific microservices and less capacity to less used services. You can also decide where you want to scale out and scale in.
In a monolith, you cannot replace the system and updates can stall everything. Using microservices, you can update, shut down, and replace any of the pieces and the rest of your system will keep running.
Core Principles of Microservices Databases
The challenge for microservices is to make your database system as flexible and scalable as possible, while minimizing the highly-flammable levels of complexity built into the system.
Meeting the core standards of a microservices database enables you to accomplish just that.
Data Independence: A Microservices Database gives you islands of data. The first challenge is to maintain data independence by effective integration of these data islands.
The worst way to integrate is for one service to take data directly from another service's database. At any given point, the people in charge of that database can change label names, column parameters, just about anything that can break your integration systems. It also puts pressure on other teams to restrain their database from any enhancements so as not to break the bridge you made from their systems to yours.
One solution is to create a shared database with an agreeable schema where you can take data from that database without limiting the evolution of the other team's database. Another is to create effective ETL processes to move data back and forth without touching the actual database.
Don't keep data that you don't need. For core functionality, you want to limit the scope of the service. Your service should handle queries a lot like a government bureaucrat, either service them right away or say, "We don't handle this, go down the hall and take a left at room 303, they can help you." Make sure the role of each service is defined, and you only use resources that fit that services' parameters.
Limited amount of state. Too much state increases complexity. It's easier to have a bunch of simple services that together produce a complex system than to have that complexity inherent inside your system.
Availability and capacity. What happens when one service is offline? In a monolith, if one part goes down, the entire system can be frozen. With microservices, if one service is down, you can have replications of that service on other nodes that can take on the load and keep the service running.
The challenge for this architecture is to make sure that your integration points can handle services they integrate to going offline. Will they failover to duplicate nodes? Will they alert users? What are the options?
Single source of truth. In a distributed system, the notion of data ownership is important. Multiple instances mean information gleaned from a replicated node may not be current, even if it is just nanoseconds old. You need to make sure information going to clients and users is in its most updated form. This makes an ACID database all the more important.
Data ownership between services. Data concurrency is an issue. You need to resolve occurrences where two users update the same piece of data at the same time. Your database must be able to handle that effectively and optimistically.