Skip to content

C09 — WAL & Replication Slot Health

Replication slots are how PostgreSQL guarantees a replica or logical subscriber won't lose data it hasn't consumed yet — but an inactive or lagging slot achieves that guarantee by retaining WAL indefinitely, which can fill the data directory. This category covers slot validity, retention pressure, logical subscription worker/error status, and (PostgreSQL 17+) failover-ready slot synchronization.

16 checks, PGHF09-001 through PGHF09-016. Every check here is PGHF-namespace and built-in — see Metadata Columns Explained for what each field below actually means, and Resetting to Defaults for why these definitions can't be hand-edited in place.

PGHF09-001 — Worst inactive replication slot lag

An inactive slot keeps retaining WAL on the primary for a consumer that isn't reading it — left unaddressed, this is one of the more common causes of a pg_wal directory quietly filling the disk.

How to fix

  1. Identify whether the consumer (a standby, a logical subscriber) is genuinely gone or just temporarily disconnected.
  2. If it's gone for good, drop the slot:
SELECT pg_drop_replication_slot('the_slot_name');
  1. If it should still be active, fix the consumer's connection instead — dropping the slot on a consumer that's coming back forces a full re-sync from scratch.
  2. As a longer-term guard, set max_slot_wal_keep_size (PGHF09-010) so a future forgotten slot can't retain WAL unboundedly.

PGHF09-002 — Worst retained WAL vs max_slot_wal_keep_size

A slot approaching its configured WAL-retention ceiling is close to being forcibly invalidated by PostgreSQL itself — better to catch it heading there than discover the slot already dead in PGHF09-009.

How to fix

Find the specific lagging slot (pg_replication_slots) and either speed up its consumer or raise the ceiling temporarily while the root cause is fixed:

ALTER SYSTEM SET max_slot_wal_keep_size = '50GB';
SELECT pg_reload_conf();

Raising the ceiling only buys time — the underlying consumer lag (network, I/O, or a stalled apply worker — see PGHF09-004/PGHF09-012) is what actually needs fixing.

PGHF09-003 — Replication slot count vs max_replication_slots

Once slot usage nears max_replication_slots, adding a new standby or logical subscriber fails outright until the ceiling is raised — worth knowing before a routine scale-out attempt hits it unexpectedly.

How to fix

ALTER SYSTEM SET max_replication_slots = 20;

Requires a restart. Also clean up any inactive slots first (PGHF09-001/PGHF09-011) — raising the ceiling doesn't help if the existing slots are dead weight rather than genuine capacity need.

PGHF09-004 — Replication lag (pg_stat_replication)

Lagging replicas are usually the first visible symptom of a network or I/O bottleneck between primary and standby — and a lagging replica risks becoming a stale failover target if promoted while behind.

How to fix

  1. Check which lag phase is largest — write_lag/flush_lag point at network or the standby's WAL-write I/O; replay_lag points at the standby struggling to apply changes (often blocked by PGHF13-006 query conflicts, or CPU/I/O contention on the replica itself).
  2. For persistent replay lag specifically, check hot_standby_feedback (PGHF12-010) and max_standby_streaming_delay — an aggressive delay setting cancels standby queries rather than letting replay fall behind, trading lag for query cancellations.
  3. For network/write lag, check bandwidth and latency between primary and standby directly — this is infrastructure, not a PostgreSQL setting.

PGHF09-005 — Unnamed standbys

A standby connecting without a meaningful application_name is hard to identify in pg_stat_replication during an incident — a small operational-visibility gap that costs time exactly when time matters most.

How to fix

Set application_name in the standby's connection string (primary_conninfo in postgresql.auto.conf, or the equivalent in your provisioning tooling), then restart the standby's WAL receiver to pick it up:

primary_conninfo = '...  application_name=standby-us-east-2'

PGHF09-006 — recovery_min_apply_delay

A deliberately delayed replica provides a recovery window against accidental data loss (e.g. a bad DELETE), but it also directly increases RPO if that replica is ever needed for failover — worth confirming the tradeoff is intentional, not forgotten.

How to fix

No universal fix — this is a deliberate-tradeoff confirmation check, not a defect:

  1. If the delay is intentional (a "delayed replica" recovery safeguard), document it and make sure it's excluded from any automated failover target list.
  2. If it was left over from a one-off maintenance task and forgotten, clear it:
recovery_min_apply_delay = 0

(in the standby's config, then reload).

PGHF09-007 — wal_keep_size vs unslotted standbys

A standby that isn't using a replication slot has no guarantee its needed WAL stays around — with wal_keep_size at 0, a brief network blip can mean the standby's WAL position is gone by the time it reconnects, forcing a full re-sync.

How to fix

The robust fix is a replication slot, which retains exactly the WAL each standby still needs, no more:

SELECT pg_create_physical_replication_slot('standby_us_east_2');

then point the standby's primary_slot_name at it. If a slot genuinely isn't wanted (e.g. to bound worst-case WAL retention regardless of standby state), set a nonzero wal_keep_size instead as a fallback safety margin:

ALTER SYSTEM SET wal_keep_size = '2GB';
SELECT pg_reload_conf();

PGHF09-008 — WAL archiving cross-reference

Replication-slot health (this category) and archiving health (PGHF02) are two different WAL-retention pressures that compound — a full picture of "is WAL under control" needs both read together, not just one in isolation.

How to fix

No fix here directly — see PGHF02-001/PGHF02-002 for archiving remediation and the rest of PGHF09 for slot-side remediation. If both are contributing at once, fix archiving first (it's usually the cheaper, more isolated problem) before chasing slot-retention tuning.

PGHF09-009 — Invalidated replication slots

Requires PostgreSQL 17+.

An invalidated slot is effectively dead — it can never catch up regardless of how long it's left in place — and needs to be manually dropped and its consumer re-synced from scratch; leaving it around gives zero benefit.

How to fix

  1. Check invalidation_reason in pg_replication_slots for why (wal_removed, rows_removed, wal_level_insufficient, and similar).
  2. Drop it — an invalidated slot cannot recover:
SELECT pg_drop_replication_slot('the_invalidated_slot');
  1. Re-create it and fully re-sync the consumer from scratch (a fresh base backup for a physical standby; CREATE SUBSCRIPTION with initial sync for logical). Then address the root cause so it doesn't recur — usually max_slot_wal_keep_size too low (PGHF09-010) or the consumer having been disconnected too long.

PGHF09-010 — max_slot_wal_keep_size

Left at its unlimited (-1) default, a single dead or stuck logical slot has nothing stopping it from retaining WAL until the disk fills — this GUC is the safety valve PGHF09-002 measures usage against.

How to fix

ALTER SYSTEM SET max_slot_wal_keep_size = '50GB';  -- size to available pg_wal headroom
SELECT pg_reload_conf();

This trades slot durability for disk safety — a slot that falls behind past this ceiling gets invalidated (PGHF09-009) rather than being allowed to fill the disk. Size it against actual available space in the pg_wal filesystem, not an arbitrary number.

PGHF09-011 — Inactive logical slots

Inactive logical slots are the highest-risk category of the PGHF09-001 problem: they accumulate WAL for every single write on the primary yet have no consumer actively reading any of it, so the retention grows unbounded until someone notices.

How to fix

See PGHF09-001's remediation — same fix, higher urgency: pg_drop_replication_slot() if the consumer is gone, or reconnect/fix the consumer if it should still be active. For logical slots specifically, check whether the owning subscription (PGHF09-012/PGHF09-013) is the one that's actually broken.

PGHF09-012 — Logical replication subscription health

A subscription with no active apply worker (a null pid) means replication has stopped entirely for it — the subscription still exists and looks configured, but nothing is applying incoming changes. A nonzero apply_error_count/sync_error_count (PG15+) is a lesser but still real signal: the worker is running, but individual changes are failing to apply, silently accumulating drift. This check's evaluator treats the two differently — a missing worker is critical, error counts alone are warning — since one means fully stopped and the other means degraded but still making progress.

How to fix

If a subscription has no active apply worker (critical):

  1. Check the server log for why the worker exited — a conflict (duplicate key, missing target row), a permissions issue, or a schema mismatch (PGHF12-012-style parity problem) are the usual causes.
  2. Once the underlying cause is fixed, disable and re-enable the subscription to relaunch the worker:
ALTER SUBSCRIPTION the_sub DISABLE;
ALTER SUBSCRIPTION the_sub ENABLE;
  1. If the conflict left data inconsistent, check whether ALTER SUBSCRIPTION ... SKIP (lsn = ...) (PG13+) is appropriate to skip the specific offending transaction, or whether a full re-sync is safer.

If apply_error_count/sync_error_count is nonzero but the worker is still running (warning):

  1. The worker is retrying the same failing change repeatedly — check the log for the specific error (constraint violation, missing row, type mismatch) rather than waiting for it to self-resolve.
  2. Fix the root cause (correct the conflicting row on this side, or adjust the publisher), then confirm the count stops climbing on the next run — it's cumulative and only clears via pg_stat_reset_subscription_stats(), not automatically once the underlying issue is fixed.

PGHF09-013 — Subscription table sync state

A table stuck in the initialize or data-copy sync state is NOT being replicated until that sync completes — it can look like a healthy subscription overall while specific tables are silently out of date.

How to fix

  1. Check pg_stat_subscription_stats/the server log for why the specific table's initial sync stalled (often a lock conflict, or a constraint violation on the copied data).
  2. Once the blocker is cleared, the sync worker should resume automatically; if it doesn't, refresh the subscription for just that table:
ALTER SUBSCRIPTION the_sub REFRESH PUBLICATION;
  1. For a table stuck for an extended period with no clear cause, drop and re-add it to the publication to force a clean re-copy.

PGHF09-014 — Streaming replication lag (time)

On a synchronous standby, every COMMIT on the primary blocks for however long replay lag takes — a hidden performance killer that byte-based lag metrics alone won't reveal, since a small byte gap can still mean a large time delay under load.

How to fix

Same remediation as PGHF09-004, but check synchronous_commit/synchronous_standby_names specifically — if a synchronous standby is the one lagging, every primary commit pays that cost directly. Consider synchronous_commit = remote_write (relaxes the durability guarantee slightly, cuts the wait) or removing a chronically slow standby from the synchronous set if its durability guarantee isn't actually load-bearing.

PGHF09-015 — logical_decoding_work_mem

At the modest 64MB default with multiple active logical slots, large transactions spill decoded changes to disk in pg_replslot/ — those spill files accumulate and persist until the slot actually consumes them, quietly eating disk space.

How to fix

ALTER SYSTEM SET logical_decoding_work_mem = '256MB';
SELECT pg_reload_conf();

Multiply by the number of concurrently-active logical slots (PGHF09-011) to estimate the real worst-case memory impact before raising this cluster-wide.

PGHF09-016 — Failover-ready logical slot synchronization

Requires PostgreSQL 17+; Standby only.

PG17's failover-slot mechanism is what lets logical replication survive a physical failover — a slot flagged failover=true but not yet synced=true means a promotion happening right now would silently lose that logical subscriber's replay position, defeating the feature entirely. PGHF09-009 covers invalidation_reason but not failover-readiness — a different failure mode on the same view.

How to fix

  1. Confirm standby_slot_names on the primary lists every physical standby that needs to carry failover-ready logical slots — a slot can't sync to a standby that isn't listed there.
  2. Trigger an immediate sync rather than waiting for the next automatic cycle:
SELECT pg_sync_replication_slots();  -- run on the standby
  1. If slots stay unsynced despite both of the above, check the standby's connection to the primary and the server log for sync-worker errors — a persistent gap here means this standby genuinely isn't failover-ready for logical replication yet, not just running slightly behind.

Continue to C10 — pg_upgrade Readiness.