Alerts

Alerts turn governance and scheduler events into email notifications: a policy blocks a query, a scheduled job fails or runs slow, and the right people hear about it — through your own SMTP server or a transactional provider.

Alerts are a Pro feature. Manage them under Governance → Alerts; the API lives at /api/governance/alerts/* (returns 402 on the free edition, and the background dispatcher only runs with a Pro license).

The pipeline is: events (emitted by CH-UI) → rules (which events matter, at what severity) → channels (how email leaves the building).

What can trigger an alert

Three event types, emitted automatically:

So yes — schedules alert out of the box: every failed run emits a schedule.failed event with the schedule name, run ID, elapsed time, and error in the payload. You only need a rule and a channel to receive it.

Severities rank info < warn < error < critical.

Channels

A channel is one way to deliver email. Three provider types:

Channel configs are encrypted at rest with the app secret key. Secrets are never echoed back: the API returns the password / API key blanked, with a has_secret flag so the UI knows one is stored — updating a channel without re-entering the secret keeps the existing one.

SMTP options

Common setups:

• Port 465 (implicit TLS) — port: 465, use_tls: true. Gmail, most managed SMTP.
• Port 587 (STARTTLS) — port: 587, leave the TLS flags alone: starttls defaults to on whenever use_tls is off. The most common submission setup.
• Port 25, no TLS (internal relay) — port: 25, use_tls: false, starttls: false.

One caveat worth knowing: STARTTLS is opportunistic — if the server doesn't advertise the STARTTLS extension, the mail is sent in plaintext rather than failing. For a guaranteed-encrypted connection, use implicit TLS (use_tls: true).

Testing a channel

Every channel has a Test button (POST /api/governance/alerts/channels/{id}/test with recipients and optional subject / message). The test sends immediately, bypassing the rule engine and the dispatch queue — so it verifies exactly one thing: can this config deliver mail. Test before you depend on it.

Rules

Rules decide which events become email, and where it goes:

Available template variables: {{event_type}}, {{severity}}, {{title}}, {{message}}, {{payload_json}}, {{channel_name}}, {{channel_type}}, {{created_at}}, {{event_id}}, {{rule_name}}. Leave the templates empty for a sensible default ([CH-UI][ERROR][schedule.failed] … subject, plain-text body with the event details and payload).

A good starter rule: event_type: *, severity_min: error, one channel, your on-call address. Widen from there.

Dispatch

A background dispatcher ticks every 8 seconds:

1. Materialize — new events are matched against enabled rules (event type + severity floor); each matching active channel binding becomes a dispatch job (up to 100 events per tick).
2. Deliver — due jobs are sent (up to 30 per tick).

Reliability behavior:

• Cooldown dedupe — events carry a fingerprint (e.g. schedule:{id}:error, or policy + user + query hash). Within a rule's cooldown window, the same fingerprint is not re-dispatched to the same channel — a job failing every minute produces one email per cooldown, not sixty.
• Retries — failed deliveries retry with exponential backoff: 10s × 2^(attempt−1), capped at 30 minutes, until max_attempts is reached, then the job is marked failed with its last error.

Events history

Governance → Alerts → Events shows the raw event stream — every alert-worthy thing that happened, whether or not a rule matched it. Each event records its type, severity, title, message, JSON payload, fingerprint, and processing status.

GET /api/governance/alerts/events supports limit (default 100), event_type, and status filters. If an email never arrived, check here first: an event with no matching rule means a routing gap, no event at all means the trigger never fired.

API

Channel and rule changes are recorded in the Audit Log (alerts.channel.*, alerts.rule.* actions).

Alerts are the notification layer for Governance policies and Schedules: those pages define what gets watched, this one defines who finds out.