Skip to content

How Firing Actually Works

The severity set a notification watches

A notification fires on confirmed severity — the post-debounce value, the same one the reporting functions show, not the raw one. So it automatically respects whatever consecutive-failure requirement a threshold configures: if a check needs 3 consecutive breaching runs before its confirmed severity moves off ok (see Debounce), a notification watching it doesn't fire on run 1 or 2 either.

Which severities qualify is a set. Given as p_min_severity, it's a rank floor: severities rank skipped < not_evaluable < ok < info < warning < invalid < critical < error_in_performing_check (see How Threshold Evaluation Works), and p_min_severity => 'warning' means at or above — so warning, critical, and invalid/error_in_performing_check (a check whose evaluator started raising, or that started erroring outright, ranks above the real tiers). This is deliberate: a broken check is treated as at least as urgent as a confirmed breach, not silently invisible — worth knowing if a notification fires and the collected value looks unremarkable. Given as p_severities, the set is exactly those values, nothing implied.

on_change: transition-only

The rule for on_change: fire once when severity crosses into the set, then stay silent while it stays in. Fire again only after it drops out and later re-enters. For a set of {warning and worse}:

Run Confirmed severity Fires? Why
1 ok No Not in the set.
2 warning Yes (breach) Crossed in — a transition.
3 warning No Still in; no transition.
4 critical No Still in (critical is in a {warning and worse} set).
5 ok No* Left the set. *With p_fire_on_clear => true, fires here as a 'clear'.
6 warning Yes (breach) A new transition — left at run 5, re-entered at run 6.

No "last fired" flag: each evaluation, the engine compares the check's immediately preceding surfaced evaluation against the current one — current in-set and previous not in-set is a 'breach'; with fire_on_clear, the mirror image is a 'clear'. There's no "resolved" judgment — critical easing to warning clears a {critical} notification while a {warning and worse} one stays breached; each set answers for itself. A check's very first evaluation counts as "not previously in the set," so a brand-new check already in the set fires immediately.

on_state: every in-set evaluation

The rule for on_state: fire on every evaluation whose severity is in the set — no transition comparison at all — throttled only by cooldown_minutes (measured from the last firing; an intervening severity never resets it). This is the vehicle for two things on_change deliberately won't do:

  • On-event checks. Each execute_check_event() is a distinct real-world occurrence (often a different record_pk). They share one severity stream, so an on_change notification would fire only the first of a run of same-severity occurrences — an on_state one fires for every occurrence (500 → 500), which is usually what you want for discrete events. Set a cooldown if you'd rather throttle.
  • Nag reminders. "Still critical — remind me every 4 hours" is on_state with cooldown_minutes => 240.

Every firing lands in the outbox

Regardless of trigger mode or delivery kind, one pghf.notification_log row per firing — the notification, the transition ('breach' / 'clear' / 'state'), old/new status, the matched set, and a self-contained payload that includes record_pk and source_table. outbox waits 'pending'; notify also rings the doorbell; function runs inline and records 'delivered'/'failed'. The outbox is never consulted for the firing decision — it's what makes "did we notify on this, and was it delivered?" answerable and lets a consumer retry without ever double-deciding.

Multiple notifications, and scope

A check can have any number of notifications, each computed independently. A notification targets one check, one category, or — with no scope — globally (every check): one global {critical} row is the whole "anything goes critical, anywhere" configuration. Firing is always computed against the check's own one evaluation stream (the scope only decides which notifications hear about it); like thresholds, there is no suite scoping — the baseline is the check's most recent prior surfaced evaluation, whichever suite produced it.

Continue to Managing Notifications.