How Threshold Evaluation Works¶

Threshold evaluation is a separate, optional layer that judges already-collected data. Nothing here touches collection — the evaluation engine never re-runs a check, it only reads raw results that a collection pass has already recorded.

The severity scale¶

Four ascending alert levels, matching the same model used by the upstream project this framework's default thresholds are drawn from:

ok  <  info  <  warning  <  critical

Plus two states that aren't really rungs on that ladder at all:

unknown — the check reported itself as skipped, or failed outright. This is a real gap in your data, and is never silently read as ok.
not_evaluable — no threshold is configured for this check. Many jsonb-typed checks (a structured breakdown with no single number to threshold) fall here permanently; others are simply checks nobody has set a threshold for yet.

Setting a threshold¶

One function, pghf.set_threshold(), is the single entry point for every case: seeding a built-in check's default, adding one for your own custom check, or updating any of the above later. It's an upsert — there is no separate "override" concept and no protected, unchangeable state:

-- a check-level default (applies to every suite that runs this check)
SELECT pghf.set_threshold('PGHF01-003', p_warn_value => '75'::jsonb, p_crit_value => '90'::jsonb);

-- a per-suite override (only applies when running THAT suite)
SELECT pghf.set_threshold('PGHF01-003', p_check_run_id => '<uuid>', p_warn_value => '60'::jsonb);

-- update a seeded value later — same function, replaces it in place
SELECT pghf.set_threshold('PGHF01-003', p_warn_value => '70'::jsonb, p_crit_value => '85'::jsonb);

Any combination of the info, warn, and crit tiers may be set — many real checks only use one or two of them. Boolean (true/false) checks use a single "which severity does a failing value map to" parameter instead of numeric tiers, since there's no scale to set tiers along.

Evaluating a run¶

CALL pghf.evaluate_run(p_run_id => '<run_id>'), after collection has already run, judges that run's data. See Running It End to End for a full worked example against the seed catalog, with real captured output.

pghf.evaluate_run() is a PostgreSQL procedure, not a function — procedures can't hand back a result set directly, only individual output values. Two separate, read-only functions cover reading the results back out, for two different audiences. They are also the only supported way to read this data — the underlying tables aren't openly readable (see The Access Model below), so these two functions are the actual interface:

pghf.get_results_json(p_run_id) — one block of structured data, for a machine to consume. This is what you'd call to build the payload for an external notifier, a webhook, or a dashboard backend — anything that wants "the results of this run" as one self-contained unit and will do its own filtering:

SELECT pghf.get_results_json('<run_id>');
-- [{"check_id": "PGHF01-003", "raw_severity": "ok", "confirmed_severity": "ok", "message": null, "detail": {"value": 1.03}}, ...]

A flat list, one entry per evaluated check, ordered by check ID. No severity filtering — it's the complete picture, and it's up to whatever consumes it to decide what matters.

pghf.get_results_sql(p_run_id, p_min_severity => 'ok') — a proper result set, for a database administrator reading output directly in psql:

SELECT * FROM pghf.get_results_sql('<run_id>');
--  severity | category_id |  check_id  |                  check_name                   | value  |                  message
-- ----------+-------------+------------+-----------------------------------------------+--------+--------------------------------------------
--  critical | PGHF11      | PGHF11-009 | Superuser login roles as % of all login roles | 100.00 | value 100.00 exceeds critical threshold 50
--  ok       | PGHF01      | PGHF01-003 | Connection saturation                         | 1.03   |
--  ...

-- narrow to just what needs attention:
SELECT * FROM pghf.get_results_sql('<run_id>', p_min_severity => 'warning');   -- warning + critical only

Worst severity first, already joined so the check's name and its actual collected value are right there — no second query needed. p_min_severity works like a logging level: the tier you name, and everything more severe, is included. The default, 'ok', is the lowest real tier, so it shows literally everything, unknown and not_evaluable rows included (they share ok's rank, since they aren't really rungs on the alert ladder at all). Asking for 'info', 'warning', or 'critical' progressively narrows the result.

Both functions work for any run ID, not only one you just evaluated in the same session, and both raise a clear error for a run ID that doesn't exist at all — rather than silently returning something empty. A run that exists but simply hasn't been evaluated yet is a different, legitimate case: that's an empty answer, not an error.

Knowing whether a run actually finished¶

Neither function above tells you whether a run actually completed. Because the collection engine commits after every single check (so a crash partway through never loses what was already collected), an interrupted run's partial results look entirely plausible on their own — nothing in the results distinguishes "genuinely nothing critical happened" from "the run crashed before the one check that would have found something critical ever ran." pghf.get_run_status() closes that gap:

SELECT * FROM pghf.get_run_status('<run_id>');
--     ...      | finished_at | is_collection_complete | checks_expected | checks_observed | checks_evaluated | is_evaluation_complete
-- --------------+-------------+-------------------------+------------------+------------------+-------------------+-------------------------
--     ...       | 2026-...    | t                       | 150              | 150              | 150               | t

is_collection_complete simply means the run has a finish time recorded; is_evaluation_complete means every collected check has also been evaluated. checks_expected reflects the suite's current membership, not what it was at the time of that run — suite membership isn't tracked historically — so for an older run, treat this as a sanity check rather than something that must match exactly.

Finding a run in the first place¶

Everything above assumes you already have a run ID. Finding one is pghf.list_runs()'s job — your entry point into all of this when you don't have direct access to the underlying tables (see The Access Model below):

-- what result sets are even available? most recent first
SELECT * FROM pghf.list_runs();

-- narrow to one suite, or a time window
SELECT * FROM pghf.list_runs(p_run_key => 'default', p_since => now() - interval '7 days');

This returns one row per run — which suite it was, when, and a count of each result type, both from the collection side (how many collected, skipped, or errored) and the evaluation side (how many at each severity — all zero for a run that hasn't been evaluated yet, which isn't an error). A limit parameter (defaulting to the 50 most recent, unbounded if you pass none) caps how far back it looks.

Once you have a run ID from this, a handful of further read-only functions give you the full detail behind it: the plain run record itself (who or what triggered it, the server's PostgreSQL version, whether it's part of a Spock cluster); every check's full collection detail rather than just its evaluated value (useful for a jsonb-typed check, or for reading a skipped check's explanation); and the extra parameters a run was given (useful for working out why a check that depends on one of them reported itself as skipped). All of these are documented in full, with their exact parameters, in the Reference Guide.

Debounce — requiring more than one bad result in a row¶

A threshold's debounce setting (the default is 1, meaning no debounce at all) requires a chosen number of consecutive breaching runs before a check's confirmed severity actually reflects the breach — its raw severity always shows exactly what that one run's data says on its own, with no smoothing. A single healthy run always clears the breach immediately — recovering isn't held to the same "N in a row" rule as breaching is. This means a check that's flapping right at its threshold will alert, clear, alert, clear on every other run once it's past its first confirmation, rather than needing several consecutive good runs to clear, the same way it needed several consecutive bad ones to confirm.

The lookback for this is scoped to the specific suite being evaluated — a check's streak is tracked per check-and-suite combination, not globally, so moving a check to a different suite (or evaluating it via a suite for the first time) starts its streak over at zero.

Custom evaluators¶

For a condition a flat threshold simply can't express — a rolling average, a rule that depends on more than one field at once, a categorical judgment on text — register a function instead of numeric tiers. This uses the same extensibility idea as checks themselves: write a function with the right shape, and the framework calls it automatically, with no special-casing needed anywhere else. Five such functions ship with the framework already, covering things like a self-adjusting rolling baseline for WAL growth rate, and a rule that treats replication lag differently depending on whether a standby is synchronous. See Writing a Custom Evaluator for the full pattern and worked code.

Continue to The Access Model.