Debounce: Requiring Consecutive Failures Before Reporting¶
A single noisy sample shouldn't page anyone. The consecutive-breaches setting on a threshold requires a chosen number of consecutive breaching runs before a check's confirmed severity actually reflects the breach. Every command and output below was run for real.
1. A controllable demo check¶
Its value comes from a table you flip between healthy and breaching, so the demonstration doesn't depend on real server load or timing:
CREATE TABLE demo_control (v numeric);
INSERT INTO demo_control VALUES (10);
CREATE OR REPLACE FUNCTION pghf.chk_demo_debounce(p_run_id uuid) RETURNS pghf.check_result
LANGUAGE sql AS $$
SELECT ROW('COLLECTED', jsonb_build_object('value', (SELECT v FROM demo_control)), NULL)::pghf.check_result;
$$;
SELECT pghf.upsert_namespace('DEMO', 'Debounce Demo');
SELECT pghf.upsert_category('DEMO', '01', 'Debounce Demo');
SELECT pghf.create_check('DEMO', '01', 1, 'demo debounce check',
'pghf.chk_demo_debounce(uuid)'::regprocedure, 'numeric',
p_rationale => 'demonstrates min_consecutive_breaches',
p_numeric_direction => 'lower_is_better');
-- => 'DEMO01-001'
2. A threshold requiring 3 consecutive breaches, then a suite to run it in¶
SELECT pghf.set_threshold('DEMO01-001', p_crit_value => '50'::jsonb, p_min_consecutive_breaches => 3);
SELECT pghf.create_check_run('demo_suite', 'debounce demo');
SELECT pghf.add_check_to_run('demo_suite', 'DEMO01-001');
3. Run it five times¶
Flip the value between healthy and breaching, using the combined collect-and-evaluate call each round:
-- round 1: healthy
CALL pghf.run_and_evaluate(p_run_key => 'demo_suite');
-- round 2: breach #1
UPDATE demo_control SET v = 100;
CALL pghf.run_and_evaluate(p_run_key => 'demo_suite');
-- round 3: breach #2 (still 100)
CALL pghf.run_and_evaluate(p_run_key => 'demo_suite');
-- round 4: breach #3 (still 100)
CALL pghf.run_and_evaluate(p_run_key => 'demo_suite');
-- round 5: recovered
UPDATE demo_control SET v = 10;
CALL pghf.run_and_evaluate(p_run_key => 'demo_suite');
Pulling the whole sequence for that one check afterwards:
SELECT row_number() OVER (ORDER BY r.started_at) AS round,
(rr.observed->>'value') AS collected_value,
e.raw_severity, e.confirmed_severity
FROM pghf.evaluations e
JOIN pghf.runs r ON r.run_id = e.run_id
JOIN pghf.raw_results rr ON rr.run_id = e.run_id AND rr.check_uid = e.check_uid
WHERE e.check_uid = (SELECT check_uid FROM pghf.checks WHERE check_id = 'DEMO01-001')
ORDER BY r.started_at;
round | collected_value | raw_severity | confirmed_severity
-------+------------------+--------------+--------------------
1 | 10 | ok | ok
2 | 100 | critical | ok
3 | 100 | critical | ok
4 | 100 | critical | critical
5 | 10 | ok | ok
What's happening at each round:
- Round 1 — healthy, nothing to debounce.
- Rounds 2 and 3 — the raw severity is already
criticalimmediately, with no smoothing on the raw signal at all, but the confirmed severity staysokbecause only 1, then 2, consecutive breaching runs have happened — not yet the 3 the threshold requires. - Round 4 — the third consecutive
criticalresult in a row. The confirmed severity flips tocriticalhere, and only here. - Round 5 — one healthy run clears it immediately, straight back to
ok. Recovery is not symmetric with breaching: it doesn't take 3 good runs to clear, just 1 — a deliberate simplification worth knowing about if your own experience with the framework suggests it should behave differently.
A few things worth knowing before relying on this:
- The comparison is by severity rank, not an exact match. A
warning-tier debounce is also satisfied by an interveningcriticalrun — getting worse while waiting to confirm doesn't reset the streak. - Debounce counts evaluated runs, not clock time. "3 consecutive breaches" means 3 runs of whichever suite is being evaluated — 15 minutes if you run every 5 minutes, 3 days if you run once a day. There's no independent time-window option.
- Scoped to the specific check-and-suite combination. The same check running in two different suites debounces independently in each — a streak in one doesn't count towards, or interfere with, the other.
- Works identically for a per-suite override (see Overriding a Threshold Per Suite) — the consecutive-breaches setting is just another field on whichever threshold record applies, default or override included.
Clean up the demo afterwards the same way as any other custom check — inactivate it (see Inactivating a Check), then drop the demo table and function yourself if you don't want them lingering.
Continue to Overriding a Threshold Per Suite.