Scalability for Dummies - Part 2: Database

By Oleksandr Andrushchenko — Published on — Modified on

In Scalability for Dummies - Part 1: Clones, we scaled the application by adding more servers.

Everything looked great. CPU usage dropped, requests became faster, and users were happy.

Then traffic kept growing.

Suddenly the database became slow, connection pools filled up, queries started timing out, and adding more API servers stopped helping.

Welcome to the next bottleneck.

Table of Contents

The Problem

Imagine you have:

10 API servers
       │
       ▼
  PostgreSQL

Adding API servers is easy.

The problem is that every server sends queries to the same database.

If you double the number of API servers, you often double the database workload.

Eventually the database becomes the bottleneck.

Why Clones Stop Helping

Many teams make the same mistake:

System is slow
       │
       ▼
Add more API servers
       │
       ▼
System is still slow

The bottleneck moved.

The application is no longer the limiting factor. The database is.

Common Symptoms

  • Slow queries
  • High database CPU
  • Connection pool exhaustion
  • Lock contention
  • Replication lag
  • Request timeouts

If adding more API servers no longer improves performance, the database is usually the next thing to investigate.

The Usual Optimization Order

Most systems follow roughly the same scaling path.

Do not start with sharding.

Sharding is usually one of the last steps.

1. Optimize Queries

The cheapest database optimization is fixing bad queries.

For example:

SELECT *
FROM orders
WHERE user_id = 123;

Without an index, the database may scan millions of rows.

One optimized query can outperform dozens of additional servers.

2. Add Indexes

Indexes allow the database to find rows faster.

A properly chosen index can reduce query latency from seconds to milliseconds.

Always measure before and after.

Too many indexes make writes slower.

3. Connection Pooling

Creating database connections is expensive.

Connection pools reuse existing connections instead of opening new ones for every request.

API Clones
      │
      ▼
Connection Pool
      │
      ▼
Database

This is one of the easiest wins for busy applications.

4. Read Replicas

Many systems are read-heavy.

For example:

  • Viewing products
  • Viewing profiles
  • Viewing articles

Reads may represent 90% or more of all traffic.

Read replicas allow those reads to be distributed across multiple database servers.

Writes
  │
  ▼
Primary Database
  │
  ├── Replica 1
  ├── Replica 2
  └── Replica 3

The primary handles writes.

Replicas handle reads.

5. Caching

The fastest database query is the one you never execute.

A cache stores frequently requested data in memory.

For example:

  • Product pages
  • User profiles
  • Article metadata

Many applications reduce database traffic by 80-95% using effective caching.

6. Sharding

Eventually one database server becomes too large.

At that point data is split across multiple databases.

Users 1-10M  -> Shard 1
Users 10M-20M -> Shard 2
Users 20M-30M -> Shard 3

Sharding works, but it adds significant complexity.

Most companies should exhaust simpler options first.

Architecture Example

A typical evolution looks like this:

Phase 1

API
 │
 ▼
Database


Phase 2

API Clones
    │
    ▼
Database


Phase 3

API Clones
    │
    ▼
Redis Cache
    │
    ▼
Primary Database
    │
    ├── Read Replica 1
    ├── Read Replica 2
    └── Read Replica N


Phase 4

API Clones
    │
    ▼
Redis Cache
    │
    ▼
Database Shards

Most successful systems follow a path similar to this.

Common Mistakes

Sharding Too Early

Many databases can comfortably handle millions of rows and thousands of queries per second.

Premature sharding creates complexity without solving real problems.

Ignoring Slow Queries

A single inefficient query can consume more resources than thousands of normal requests.

Always check query plans before adding infrastructure.

Using the Database as a Cache

Repeatedly reading the same data from the database wastes resources.

If data rarely changes, caching is often a better solution.

Conclusion

Cloning application servers is usually the first scalability step.

The database is usually the second bottleneck.

Before considering sharding, optimize queries, add indexes, use connection pools, introduce read replicas, and implement caching.

The goal is not to scale the database immediately. The goal is to avoid needing to scale it for as long as possible.

In Scalability for Dummies - Part 3: Cache, we will look at one of the most powerful performance tools in system design: caching.

Comments (0)