Did they tell you how to split the Database?

There are many posts about splitting your monolithic source code, but none of them explain how to split your Database. This is my humble intent.

Sep 18, 2024

Thank you to our sponsors who keep this newsletter free:
Multiplayer auto-documents your system, from the high-level logical architecture down to the individual components, APIs, dependencies, and environments. Perfect for teams looking to streamline system design and documentation management without the manual overhead.

Many people focus on dividing the codebase but overlook the importance of your Database. If you stop at the source code level and keep pointing your newly distributed services to a monolithic Database, you will create bottlenecks and eventually negate the benefits of microservices.

Splitting the Database is critical for achieving true scalability and independence between services.

There are two main approaches to this task: Backup And Restore and Replication. However, each method has its own trade-offs.

Like in every single migration, you don't wake up one day ready to start splitting your DB. Many articles talk about this topic, but they miss what I think is a critical step.

Preparation.

I recommend isolating your Database objects —such as tables, stored procedures, etc.— using User-Defined Schemas based on your business domains.

This separation lays the groundwork for migrating each domain to its own Database server, and more importantly, it will show you which domains are coupled.

From here, you can decide which path you want to follow.

Approach 1: Backup And Restore

In simple terms, the idea is to back up domain-specific data and restore it to new servers.

Steps:

Backup Schemas: While most relational Databases don't support backing up individual schemas directly, you can back up the entire Database and keep the schemas you need.
Set Up New Servers: Provision new Database servers for each domain.
Restore Data: Restore the extracted schemas onto their respective servers.
Update Connections: Point your services to the new Database servers.
Clean Up: Remove unnecessary schemas from the original Database.

Trade-Offs:

Pros: Simpler process, easier to manage.
Cons: Typically requires downtime during migration.

Approach 2: Replication

Replication copies data from one Database to another in real-time, allowing for continuous operation during the transition.

Steps:

Set Up New Servers: Prepare new Database servers for each domain.
Configure Replication: Use Database replication tools to sync schemas to the new servers.
Switch Connections Gradually: Slowly update services to point to the new Databases without halting operations.
Remove Old Schemas: Delete unnecessary schemas from the original Database after confirming successful migration.

Trade-Offs:

Pros: Minimizes downtime and allows for a seamless transition.
Cons: More complex setup, requires careful coordination and monitoring.

Splitting your monolithic Database may seem challenging, but it's a critical step toward improving your system.

Organizing and separating your data based on your business domains makes each service more independent and efficient. This leads to faster performance, easier scalability as your user base grows, and simpler maintenance over time.

You'll reduce bottlenecks and minimize the risk of system-wide failures, making your application more reliable.

In the end, splitting your Database not only solves current issues but also sets your system up for long-term success and flexibility.

One final piece of advice:

If you can afford the downtime, go for it. It is way simpler.

Now that you know this, what's holding you back from splitting that huge Database?

System Design Classroom is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

Articles I enjoyed this week

Thank you for reading System Design Classroom. If you like this post, share it with your friends!

System Design Classroom

Discussion about this post

Ready for more?