Reducing Write Amplification in High Throughput Databases

At high write rates, write amplification stops being an academic metric and starts acting like a silent tax on everything you care about: tail latency, SSD endurance, replication lag, and your ability to do routine maintenance without watching dashboards like a hawk.

Plain definition: write amplification is how many bytes your system writes to storage for every byte of user data you think you wrote. If you insert 1 GB and the storage stack ends up writing 10 GB because of compaction, page rewrites, indexes, journaling, and SSD garbage collection, your write amplification is 10x.

The uncomfortable truth is that the biggest drivers of write amplification are usually “good” features: durability logs, compaction, MVCC, secondary indexes, and background cleanup. Reducing write amplification is less about finding a magic knob and more about deciding which cost you want to pay, and when.

What engine builders keep warning about

Researchers and database engineers who live inside storage engines tend to converge on the same message: write amplification is rarely caused by one bad setting.

People working on LSM engines point out that compaction often rewrites data that does not strictly need rewriting. Partial overlaps, poor file boundaries, and mixed data lifetimes can turn background cleanup into a constant rewrite machine.

Engineers working on MVCC systems like Postgres emphasize a different pain point: every update can silently fan out into multiple physical writes, especially when indexes are involved. If an update cannot stay on the same page, or touches indexed columns, the database ends up rewriting heap pages, index pages, and WAL records even though the logical change was tiny.

The shared lesson is simple: write amplification compounds across layers. Storage engine policy, schema design, workload shape, and SSD behavior all multiply each other.

Where write amplification actually comes from

If you only remember one mental model, use this: every mechanism that rewrites the same logical record again and again stacks its cost on top of everything else.

• LSM-based systems rewrite data as it moves through compaction levels. Leveled compaction favors read efficiency but rewrites data more often. Tiered approaches rewrite less but keep more overlapping data around.
• Page-based systems with MVCC rewrite pages during updates, log changes to WAL, and later rewrite again during vacuum or cleanup.
• Secondary indexes multiply work. One row update can become N index updates.
• SSDs add their own internal write amplification through garbage collection, especially when the device is full or data with different lifetimes is mixed.

This is why “minimize writes” is the wrong goal. The real goal is avoid rewriting cold data, avoid multiplying writes through indexes, and avoid forcing the device to relocate data unnecessarily.

How to measure write amplification (with a concrete example)

Pick a fixed time window and measure two things:

• Logical writes: what the database believes it wrote, often reported as bytes ingested, WAL volume, or mutation size.
• Physical writes: what the operating system reports as bytes written to the block device.

Then compute:

Write Amplification = physical bytes written / logical bytes written

Example:

• Your system ingests 200 MB per second of new user data.
• The storage device reports 4,000 MB per second of sustained writes.

4,000 divided by 200 gives 20x write amplification.

Now translate that into endurance. At 4 GB per second, you are writing over 300 TB per day to the device. Even if your application traffic looks modest, your hardware experiences something very different.

The levers that actually move the number

There is no free lunch. Every lever that reduces write amplification usually increases something else.

The winning move is not maximizing one metric, but picking the cost your workload can tolerate.

A practical four step playbook

Step 1: Identify who is doing the rewriting

Before tuning anything, figure out where the extra writes come from.

Look at device level write rates, database level write counters, and background activity like compaction or vacuum. If physical writes spike even when ingest slows down, background maintenance is likely rewriting large amounts of data.

Also watch how write behavior changes as disks fill. Rising utilization often triggers more internal SSD garbage collection, which can dwarf application level writes.

Step 2: Reduce logical churn first

This is where the biggest wins usually live, and where teams often resist because it feels like product work.

Common churn patterns include:

• Repeated updates to hot rows like counters or status fields
• Wide rows where a tiny change rewrites a large record
• Indexes on frequently changing columns

High impact fixes:

• Move hot, frequently updated fields out of the main record.
• Replace in-place updates with append only event logs where possible.
• Ruthlessly prune secondary indexes, especially on mutable fields.

Every update you eliminate upstream removes multiple downstream writes.

Step 3: Tune the storage engine to protect cold data

For LSM-based systems:

• Consider compaction strategies that favor fewer rewrites when write throughput matters more than read latency.
• Watch for compactions that move a lot of data with minimal key overlap. That is often wasted work.
• Aim to group data with similar lifetimes so cold data stops getting dragged through compaction cycles.

For MVCC, page-based systems:

• Design schemas and updates to stay on-page when possible.
• Leave free space on pages so updates do not force page splits.
• Avoid updating indexed columns unless absolutely necessary.

In both cases, the principle is the same: background cleanup should surgically remove garbage, not reshuffle the entire dataset.

Step 4: Make life easy for the SSD

Even a perfectly tuned database can suffer if the storage device is constantly fighting itself.

Two practical rules:

• Maintain headroom. Running disks near capacity dramatically increases internal copying.
• Avoid mixing short lived and long lived data on the same volume when possible.

When the device has to copy long lived data just to reclaim space from short lived churn, write amplification skyrockets.

FAQ

What is a “good” write amplification number?

There is no universal target. Some workloads accept high amplification in exchange for fast reads. Others prioritize endurance and throughput. What matters is whether the number aligns with your hardware budget and latency goals.

Will reducing write amplification hurt read performance?

Often, yes. Many techniques that reduce rewriting trade write efficiency for more reads or more space. The question is which side of the tradeoff your workload can afford.

Why did write amplification spike when traffic increased?

Because background work could not keep up. Once compaction, vacuum, or garbage collection falls behind, the system enters a feedback loop of more rewriting and higher latency.

Does compression always help?

Not always. Compression can reduce bytes written, but it can also increase CPU cost and change write patterns in ways that trigger more rewrites elsewhere.

Honest Takeaway

You do not eliminate write amplification in high throughput databases. You decide where it shows up.

The durable wins come from boring, structural changes: reducing unnecessary updates, keeping indexes lean, designing schemas that minimize churn, and choosing maintenance strategies that leave cold data alone. Engine level tuning only amplifies the quality of those decisions.

If you treat write amplification as a system wide property instead of a single metric, you can keep throughput high without quietly burning your storage to the ground.

Kirstie Sands

Journalist at DevX

Kirstie a technology news reporter at DevX. She reports on emerging technologies and startups waiting to skyrocket.