Scaling relational databases usually starts innocently. You add a few indexes, bump the instance size, maybe stand up a read replica, and call it a day.
Then the product hits a growth spurt. You ship a new feature, traffic doubles, and suddenly “the database is slow” becomes your team’s most recurring Slack message. Latency goes up, deploys get scary, and everyone learns, again, that the database is not just another stateless service you can autoscale away.
Scaling relational databases is the art of keeping correctness, performance, and operability intact while the workload grows. That sounds obvious until you realize you are juggling connection limits, lock contention, replication lag, hot partitions, and queries that quietly went from “fine” to “catastrophic” as data volume changed. Let’s talk about the mistakes teams repeat, why they happen, and what you can do before you end up rate limiting your own app to save the database.
What experienced operators keep warning about
If you listen to people who have been on call for large production databases, a pattern shows up fast: most scaling incidents are not about raw CPU. They are about coordination.
Distributed systems researchers and long time database engineers keep pointing to the same pressure points. Sharding and read replicas do not magically fix the problem of too many application instances opening too many database connections. If you do not control access, you simply move the bottleneck around. Likewise, once your system crosses service boundaries, pretending you can keep doing large, cross-cutting transactions with the same semantics becomes a reliability tax. Teams that scale well design for clear ownership boundaries and accept that “inside the database” and “outside the database” behave differently.
Cloud provider guidance reflects this hard earned reality. Connection churn and uncontrolled connection counts are one of the fastest ways to take down an otherwise healthy database. Pooling is not a nice to have optimization, it is basic infrastructure.
The takeaway is simple and uncomfortable: the biggest failures come from coordination overhead you did not model, especially connections, locks, and cross-node consistency.
Mistake 1: Confusing “more app pods” with “more capacity”
This one shows up everywhere. You autoscale the API tier because it is stateless, and the database becomes the stateful punching bag.
Here is a worked example you can sanity check in under a minute:
- Your database can handle about 500 concurrent connections before it starts thrashing.
- You run 50 application instances.
- Each instance uses a client side pool of 20 connections “just to be safe.”
That is 50 × 20 = 1,000 possible database connections. You are already 2x over the line before traffic spikes. Add a deployment, where old and new versions overlap, and you can briefly double again.
At that point the database spends its time context switching, allocating memory per connection, and fighting for CPU instead of executing queries. This is why “just raise max connections” so often makes things worse.
What to do instead: treat connections like a hard budget. Put pooling in the right place, often a server side pooler, and size it intentionally. The goal is to reuse a small, controlled number of server connections, not let every app instance talk directly to the database whenever it feels like it.
Mistake 2: Ignoring lock and contention behavior until it is production shaped
A query that works fine at 10,000 rows can turn into a lock nightmare at 10 million.
Common ways teams get burned:
- Long transactions that hold locks while doing application work, such as network calls or retries.
- Mixed read and write patterns that create hot rows, like counters, “last seen” fields, or queue like tables.
- Schema migrations that take far longer than expected because the table is much larger than it used to be.
The dangerous part is that contention is nonlinear. A little extra latency increases concurrency. Higher concurrency increases lock waits. Lock waits increase latency. Suddenly you have a feedback loop that looks like a random outage.
What to do instead: treat lock waits and transaction duration as first class metrics. Keep transactions short, and never hold locks while doing work you do not control.
Mistake 3: Treating read replicas as a free performance upgrade
Read replicas are powerful, but they are not free.
Teams often route reads to replicas and writes to the primary, then discover the sharp edges:
- Replica lag turns “just written” data into “not found.”
- Write bursts increase lag exactly when you need reads most.
- Some queries still must hit the primary because they need up to date state.
Read scaling does nothing for write bottlenecks, and it does not fix connection storms if every app instance still opens too many connections.
What to do instead: use replicas deliberately. Route workloads that tolerate staleness, such as analytics, feeds, or search, to replicas. Keep correctness critical reads on the primary.
Mistake 4: Sharding too late, or sharding without designing the seams
Sharding is not just a database operation. It is a product decision disguised as infrastructure.
The usual traps look like this:
- Picking a shard key that creates hot partitions, such as a tenant where one customer dominates traffic, or a time-based key where “today” is always hottest.
- Keeping cross-shard joins and transactions in the product, then being surprised by the complexity.
- Trying to preserve identical semantics across shards and paying for it forever.
Once you cross shard boundaries, you must be explicit about what you guarantee and what becomes asynchronous or eventually consistent.
What to do instead: shard while you can still change APIs and data contracts. Aim for clear ownership, ideally a single writer per entity. Push cross-entity workflows into events or sagas instead of multi-way database transactions.
Mistake 5: Not planning for thundering herds and cache stampedes
Relational databases often fail under synchronized spikes, not steady load.
A common pattern is the cache stampede. A popular key expires, many requests miss at once, and they all hit the database to recompute the same expensive result. The database absorbs the entire burst.
What to do instead: add request coalescing, use jittered expirations, and rate limit expensive recomputations. These small guardrails prevent large, sudden spikes from overwhelming the system.
How to scale without stepping on rakes
A pragmatic path that works across most Postgres and MySQL setups looks like this.
Step 1: Put hard limits on connections and make them visible.
Decide how many server connections are safe, then back into per-service budgets. Pool early and size deliberately.
Step 2: Make “slow” and “blocked” different alerts.
Slow queries usually need indexing or query planning. Blocked queries usually need transaction or lock redesign. Treating them as the same problem leads to the wrong fixes.
Step 3: Separate read scaling, write scaling, and coordination scaling.
Replicas help reads. Sharding helps writes. Pooling helps coordination overhead. Each tool solves a different class of problem.
Step 4: Rehearse the scary operations.
Practice migrations on production sized data, including index builds and backfills. The first time you discover a migration takes hours should not be during a weekday deploy.
What each scaling lever actually buys you
| Lever | Helps most with | Common gotcha |
|---|---|---|
| Bigger instance | CPU, memory, IOPS headroom | Buys time, not correctness |
| Connection pooling | Connection storms, churn | Mis-sized pools still overload |
| Read replicas | Read throughput | Lag breaks read after write |
| Sharding | Write throughput, data size | Cross-shard complexity explodes |
FAQ
Is “just increase max connections” ever the right move?
Sometimes, but rarely as the first fix. If application behavior is the problem, raising the limit can increase memory pressure and make performance worse.
When should you introduce a pooler?
When application instances scale horizontally, or when you see connection churn correlated with incidents. Pooling attacks a clear mechanical limit.
Are read replicas worth it if correctness matters?
Yes, if you route only tolerant workloads to replicas and keep critical reads on the primary.
When do you stop scaling a single relational database and change approach?
When sharding complexity or multi-region semantics become the real work. Some teams move to distributed SQL, others redesign workflows to reduce coordination.
Honest Takeaway
Most teams do not hit the hard limits of relational databases. They trip over coordination problems they did not budget for: connections, locks, and cross-boundary consistency.
The fixes are not glamorous, but they work. Connection budgets, short transactions, disciplined use of replicas, and explicit boundaries turn database scaling from a guessing game into an engineering exercise. If you do one thing this week, turn your scaling plan into numbers. Once you can point at a budget and say “we are about to blow this,” scaling becomes far less mysterious.
Kirstie a technology news reporter at DevX. She reports on emerging technologies and startups waiting to skyrocket.
