How many times have you seen analytics on an operational database create issues? I’ve seen it far too often.

The story usually goes like this:

You've got a service and database that handle real-time requests, often mission-critical.

But now, you must create reports, data feeds, or some other batch process.

The most straightforward approach is to run those batch jobs against the same production database cluster.

It works fine, until it doesn't...

As the database grows and the batch jobs change, what once worked fine slowly (sometimes abruptly) increases the load on your production database.

Eventually, the load hits your mission-critical system, often taking it down completely.

So, how do you avoid this scenario?

💡 The Ideal Solution is to separate real-time and analytics systems. The real-time system can publish events consumed by a platform purpose-built for analytics and batch processing.

👨‍🔧 The Pragmatic Solution is to run the batch jobs on a read-only replica database instance/cluster dedicated for this purpose. Only costs a few extra database instances.

Both options will require more infrastructure, but it's a small price to pay to avoid a call at 3 a.m.