On-Event Checks

Every check built so far in this book is polled: it joins a suite, and the collection engine invokes it as that suite's ordered walk reaches it, on whatever cadence a scheduler drives. That's the right shape for "count the instances of a condition right now" — connection saturation, table bloat, replication lag.

Some things aren't like that. "A specific job just finished with a bad status" isn't a condition to poll for — it's a discrete event that happens once, and by the time it happens, there's nothing left to measure. The event is the finding. Whatever triggers it already knows how bad this is and which record it's about; there's no threshold to compare anything to, because nobody is proposing a number that might or might not be a problem. That's what this chapter is for: a check invoked standalone, exactly once, by something else — almost always a trigger on the table where the event shows up — told directly what severity to record and, where relevant, which row it's about.

Registering one

An on-event check is an ordinary check in every other respect — same (p_run_id uuid) RETURNS pghf.check_result contract from the Check Contract, same registration functions, same thresholds and debounce once its result lands. What's different is declared once, at registration:

SELECT pghf.create_check(
    p_namespace_code => 'X',
    p_category_code  => '01',
    p_run_number     => 3,
    p_check_name     => 'Widget job failed',
    p_routine        => 'pghf.chk_x01_003(uuid)'::regprocedure,
    p_result_type    => 'jsonb',
    p_rationale      => 'A widget-processing job that finishes with a non-normal status needs attention close to when it happens, not at the next scheduled poll.',
    p_execution_mode => 'on_event',
    p_source_table   => 'public.widget_jobs'
);
-- => 'X01-003'

p_execution_mode defaults to 'polled' — every built-in, and every check built earlier in this book. Set it to 'on_event' and two things change structurally:

• pghf.add_check_to_run() (and the bulk-add functions that delegate to it) refuse to add this check to any suite. That's not a naming convention — an on-event check's body typically depends on context that only a direct invocation provides (see The event supplies its own verdict below), so a suite silently picking it up would run it decontextualized on every scheduled pass.
• p_source_table becomes required. The table this event is about — free text, conventionally schema-qualified ('pgrelay.log', 'public.widget_jobs'), not validated against the catalog (the table may belong to an extension not installed yet). Purely descriptive: nothing in the engine reads it, but it's how a human — or eventually a notification layer — gets from "X01-003 fired" to "here's the table to look at," without already knowing this specific check.

pghf.copy_check() follows the usual inherit-by-blank rule for both fields — leave them out to keep the source's values, or state one to fork a polled built-in into your own on-event variant.

The event supplies its own verdict

The check routine's own signature never changes — one run ID, nothing else. But pghf.execute_check()/pghf.execute_check_event() themselves take three more parameters, and this is the part that actually makes an on-event check what it is:

CALL pghf.execute_check_event(
    p_check_id  => 'X01-003',
    p_run_id    => v_run_id,
    p_severity  => 'critical',                                    -- one of ok / info / warning / critical
    p_record_pk => jsonb_build_object('id', NEW.id),               -- or a composite key, see below
    p_detail    => jsonb_build_object('error_message', NEW.error)  -- optional, anything else worth keeping
);

p_severity is not a measurement — it's the caller's own verdict, asserted directly, because the trigger invoking this already knows it. It's restricted to ok/info/warning/critical; the framework's other four severity states describe the framework's own machinery failing to judge something, not an outcome your event should ever be asserting about itself — and it's validated immediately, before anything else happens, so a bad value raises right there in your trigger's own transaction rather than surfacing later during evaluation.

p_record_pk identifies the offending row. For a composite primary key, every column becomes its own key in one object:

p_record_pk => jsonb_build_object('tenant_id', NEW.tenant_id, 'job_id', NEW.job_id)

It must always be a keyed object — never a bare array or scalar, even for a single-column key ({"id": 123}, not 123) — validated the same way, immediately. That's what makes a composite key and a single-column key the same shape to whatever eventually looks the record up: build a WHERE clause generically from whatever keys are present, using pghf.checks.source_table to know which table to query, with no per-check hardcoding required.

The ready-made routine, for a check that does nothing but package these three values up — which is most of them:

CREATE OR REPLACE FUNCTION pghf.chk_x01_003(p_run_id uuid) RETURNS pghf.check_result
LANGUAGE sql AS $$ SELECT pghf._event_check_result(p_run_id); $$;

pghf._event_check_result() reads back whatever execute_check_event() staged and returns it as observed = {"severity": ..., "record_pk": ..., "detail": ...} — nothing left to compute, so the routine body is one line. Pair it with the shipped pass-through evaluator, which echoes that severity into a real evaluation instead of comparing it to anything:

SELECT pghf.set_threshold('X01-003', p_evaluator_routine => 'pghf.evaluate_passthrough_severity(uuid)'::regprocedure);

pghf.evaluate_passthrough_severity() is the one evaluator in this codebase that isn't check-specific — every other evaluator_routine example is written for and hardcodes one check_id, because it's comparing that check's own data against real judgment logic. This one has no judgment to apply, so it has nothing to hardcode: any number of on-event checks can point their threshold at this exact same function. It reads observed->>'severity' and returns it verbatim, carrying record_pk/detail through into the evaluation's own detail field — so debounce, hysteresis, pghf.alert_status, and notifications all keep working exactly as they do for a judged, polled check. None of that machinery cares where a severity came from.

This is a pattern, not a requirement. p_severity/p_record_pk/p_detail all default to null, and a routine that ignores them, measures something itself, and gets judged by a real threshold instead is an equally legitimate on-event check — see The other shape below.

Two invocation functions, and why

CALL pghf.execute_check('X01-003', p_severity => 'critical', p_record_pk => ...);                  -- top-level only — a human or a script, on demand
CALL pghf.execute_check_event('X01-003', v_run_id, p_severity => 'critical', p_record_pk => ...);  -- safe from inside a trigger

This is a real PostgreSQL rule, not a style choice. The collection engine commits after every check it runs, and PostgreSQL only allows a procedure to COMMIT when it's reachable via a top-level CALL — never from inside a trigger or a function, both of which are already inside someone else's transaction and can't be split across two. pghf.execute_check() is the committing half: top-level only, for a person or orchestrator running one check on demand without building a whole suite for it. pghf.execute_check_event() is its no-commit twin — safe to call from inside a trigger, which is where the real work almost always happens. pghf.execute_check() is nothing more than a CALL to pghf.execute_check_event() followed by a COMMIT, so a given check behaves identically through either door.

A worked trigger

Reacting to a bad row the moment it lands in some other table, handing over the verdict and the row it's about directly:

CREATE OR REPLACE FUNCTION trg_widget_job_failed() RETURNS trigger
LANGUAGE plpgsql AS $$
DECLARE
    v_run_id uuid;  -- unused, but must still be declared — see below
BEGIN
    IF NEW.status <> 'ok' THEN
        CALL pghf.execute_check_event(
            p_check_id  => 'X01-003',
            p_run_id    => v_run_id,
            p_severity  => 'critical',
            p_record_pk => jsonb_build_object('id', NEW.id),
            p_detail    => jsonb_build_object('error_message', NEW.error_message));
    END IF;
    RETURN NEW;
END;
$$;

CREATE TRIGGER trg_widget_job_failed
AFTER INSERT ON widget_jobs
FOR EACH ROW EXECUTE FUNCTION trg_widget_job_failed();

One sharp edge, worth knowing before your first trigger raises a confusing error. p_run_id is an input/output parameter — fine to omit entirely from a bare top-level client call (the default applies, and a fresh run ID just comes back as an ordinary result column), but PostgreSQL requires a writable variable for an input/output parameter whenever CALL is issued from inside PL/pgSQL, trigger bodies included. Omit it there and you get procedure parameter "p_run_id" is an output parameter but corresponding argument is not writable. The fix is always the one line above: declare v_run_id uuid;, and pass it explicitly even if you never read it back.

If the trigger's own surroundings might raise for reasons that have nothing to do with the check, wrap the call and swallow it — a monitoring check misbehaving should never be able to break the write it's watching:

BEGIN
    CALL pghf.execute_check_event(p_check_id => 'X01-003', p_run_id => v_run_id, p_severity => 'critical');
EXCEPTION WHEN OTHERS THEN
    NULL; -- never let a monitoring check break the write it's watching
END;

(pghf.execute_check_event() already catches anything the check routine itself raises and logs it as an error result, same as the collection engine — this second layer is only insurance against something going wrong in the surrounding plumbing, such as calling it against a database where pghf.seed_data() was never run, or an invalid p_severity/p_record_pk you don't want to crash the write over.)

Where its runs are grouped

Every run needs somewhere to point — the framework's run table always references a suite. Rather than creating one container per check (a lookup-or-create on every single trigger firing, with its own race), pghf.seed_data() seeds exactly one reserved suite, event_alerts, shared by every on-event check that ever runs. It's never purged — retention only ever deletes run history, never suite/catalog definitions — so there's no re-creation cost to design around. It also never has members: neither invocation function walks suite membership the way the collection engine does; each call names its one check directly. To find which checks are on-event, query the catalog (pghf.list_checks() filtered on execution_mode = 'on_event'), not suite membership — membership would imply something that's never true here.

Collecting is not evaluating

pghf.execute_check_event() only collects — even with a severity already supplied, it never writes a pghf.evaluations row, because pghf.evaluate_run() commits internally, the identical rule that makes execute_check_event() need to exist as a separate no-commit function in the first place. A trigger-collected run therefore sits in pghf.raw_results, verdict and all, with no debounce progress and no fired notifications, until something else, running in a real top-level context, evaluates it — even for a pass-through evaluator that's just going to echo the severity straight through. Two answers, for the two situations that come up:

• A manual, top-level invocation — a person or script running one check on demand: pghf.run_and_evaluate_check('X01-003') collects and evaluates in one CALL, the on-event counterpart to pghf.run_and_evaluate().
• Everything a trigger has collected — pghf.evaluate_pending_event_alerts() finds every finished, not-yet-evaluated run under event_alerts and evaluates each, oldest first. Schedule this on a short interval (a cron/pg_cron entry, the same way you'd schedule any suite — see Scheduling Health Checks) if you want an on-event check's threshold, debounce, and notifications to actually fire. This is a real requirement, not an optional nicety: skip it, and the check keeps collecting real data forever while nothing ever judges it.

-- somewhere on a short cron interval, e.g. every 10-30 seconds
CALL pghf.evaluate_pending_event_alerts();

Safe to schedule tightly enough that two calls overlap — an internal advisory lock makes an overlapping call notice another is already sweeping and return immediately rather than racing it.

A note on concurrent triggers. Two events for the same check firing close together create two runs whose started_at values could theoretically commit out of order under load. pghf.evaluate_pending_event_alerts() evaluates in started_at order, and pghf.alert_status's own out-of-order guard — the same one already protecting concurrent suite runs of a check — simply won't let a late-arriving older run regress the current-state projection. Every run still gets a correctly-computed evaluation row either way; the guard only affects which one wins the "what's true right now" slot.

The other shape: measuring it yourself

Not every on-event check has to supply a severity — that's the common case, not a requirement. A check can instead ignore p_severity/p_record_pk entirely, read its own data straight from the source table when it's invoked, and be judged by a real threshold exactly like a polled check would be — useful when the trigger firing it doesn't itself know enough to state a verdict, or when you'd rather centralize that judgment in the threshold instead of the trigger:

CREATE OR REPLACE FUNCTION pghf.chk_x01_003(p_run_id uuid) RETURNS pghf.check_result
LANGUAGE plpgsql AS $$
DECLARE
    v_job RECORD;
BEGIN
    SELECT * INTO v_job FROM widget_jobs
     WHERE status <> 'ok'
     ORDER BY finished_at DESC LIMIT 1;
    RETURN ROW('COLLECTED', to_jsonb(v_job), NULL)::pghf.check_result;
END;
$$;

Give a check like this an ordinary numeric/boolean/text threshold with pghf.set_threshold(), the same way as any polled check — no evaluator_routine needed. This also keeps the check testable the ordinary way (see Testing a Check in Isolation) — it reads real data on its own, with nothing to set up first.

Notifying on each occurrence

An on-event check's whole point is that each occurrence is a distinct event — often about a different record_pk. The notifications layer has a trigger mode built for exactly this: on_state, which fires on every in-set evaluation, not just on a transition.

SELECT pghf.create_notification('widget job failed', 'function', p_check_id => 'X01-003',
    p_trigger_mode => 'on_state', p_severities => ARRAY['critical'],
    p_routine => 'my_schema.alert(bigint, bigint, jsonb)'::regprocedure);

Register that, and three failed widget jobs produce three notifications — each firing's outbox payload (and pghf.notification_context(evaluation_id)) carrying that occurrence's own record_pk and this check's source_table, so the routine or an outbox consumer knows exactly which row to look at. This is why p_source_table is required at registration: it's the "here's the table" half of that answer, with no per-check hardcoding.

The default trigger mode, on_change, is deliberately not the right fit here: it fires only on a transition across the severity set, so a run of same-severity occurrences would notify only on the first one. Use on_change for polled conditions, on_state for discrete on-event occurrences. (And remember the pending sweep still gates all of this — nothing fires until pghf.evaluate_pending_event_alerts() judges what the trigger collected.)

---

Continue to Inactivating a Check, or to the Reference Guide's Collection Engine / Evaluation Engine chapters for every function's exact parameters.