Coverage plan — making these failures impossible to miss¶
Companion to BRIDGE_BASELINE.md. The baseline says what a bridge
must do; this says how we find out when it stops doing it. Requirement numbers below
are the baseline's BR-n.
The problem this has to solve¶
Every failure found on 2026-10-09 had already happened, silently, for weeks:
- 21 orphaned Home Assistant discovery configs — a duplicate device for the real gateway,
published under the
agatefallback node. Nothing raised, nothing logged, nobody knew. sw_versionreporting0.1.0while the package was0.2.0, so every HA device showed the wrong version. No event, no error.- A scheduled occurrence burned by a transient failure writes one log line and moves on.
smokehad been failing in CI since the repo went public.
So notifications alone cannot cover this. Notifications are event-driven, and these
are standing conditions — nothing happens, and that is the bug. Worse, every trigger the
bridge has today (offline, off_grid, dispatch, soc_low, soc_full) is about the
battery. Not one is about the bridge's own health.
Coverage therefore needs three layers, not one:
| Layer | Answers | Good for |
|---|---|---|
| Outcome reporting | "this call just went wrong" | events, at the moment they happen |
| Periodic self-check | "something is wrong and has been for a while" | standing conditions that raise nothing |
| Surfacing | "a human, or an automation, actually saw it" | both |
Layer 1 · Outcome reporting (events)¶
Already built in resilience.py; wiring in progress. Every boundary call yields an
Outcome, and _report() is the single place severity is decided:
| Verdict | Log level | Audit | Notify |
|---|---|---|---|
OK, first attempt |
debug | writes only | no |
OK, after retries |
info — "recovered after retry" | writes only | no |
FAILED |
warning | writes only | writes only |
UNKNOWN |
error — "UNKNOWN OUTCOME" | always | always, own title |
SKIPPED |
debug | no | no |
UNKNOWN is deliberately the loudest: it means a write may have landed and nobody knows
(BR-8). It is the one state a human must resolve, so it bypasses every throttle below.
Layer 2 · Periodic self-check (standing conditions)¶
New. A scheduled audit — hourly, plus on startup — that actively looks for the
conditions nothing else reports. Each check returns findings of
{id, severity, br, summary, detail, count, first_seen}.
| Check | BR | Severity | What it catches |
|---|---|---|---|
orphaned_entities |
BR-23, BR-26 | warning | retained discovery configs for nodes we no longer publish. Reuses the existing dry-run scan; reports, never purges |
gateway_identity_deferred |
BR-5 | warning after 15 min | a gateway whose serial never read, so it publishes nothing |
mock_in_production_roster |
BR-27 | info | mock gateways registered on a non-demo instance |
version_drift |
BR-28 | warning | __version__ vs pyproject vs config.yaml disagree |
occurrence_burned |
BR-16 | warning | a schedule window that closed without its action succeeding |
schedule_stalled |
BR-18 | info | a schedule stopped mid-window and never resumed |
poll_fail_streak |
BR-14 | warning at 3, error at 10 | consecutive failed polls per gateway |
mqtt_publisher_down |
BR-25 | warning after 5 min | MQTT enabled but no publisher for a live gateway |
foreign_producer_collision |
BR-4 | warning | another integration publishing under our device identifier |
writes_unavailable |
BR-33 | info | a control is exposed but its transport is unreachable |
dispatch_unknown_outstanding |
BR-9 | error | an UNKNOWN dispatch nobody has acknowledged |
Checks are pure functions over current state, so each is unit-testable without a broker, a gateway or a clock.
Layer 3 · Surfacing¶
A finding is worthless if it only exists in a JSON endpoint.
- App Logs — every finding writes to the bridge's own log stream at its severity,
tagged
source=selfcheckand attributed to a gateway where it has one, so the Logs tab can filter to exactly these (BR-29). - Health card —
GET /api/health/findingsdrives a card that is always visible, showing "no issues" when clean. A surface that only appears when broken is a surface nobody learns to look at. - Notifications — a new
bridge_healthtrigger family alongside the existing battery ones, so this is user-configurable like everything else. - Home Assistant entities — a
problembinary_sensor plus an issue-count sensor, so HA automations can act on bridge health rather than a human remembering to look. This is the one that makes coverage real for an unattended install.
Notification policy — the part that decides whether this works¶
Over-notifying is how alerts get ignored, which fails the goal as surely as silence:
- Notify on transition, when a finding appears — never per tick while it persists.
- Re-notify after 6 hours if still outstanding, so a real problem does not fade away.
- One daily digest listing everything still open, or nothing if clean.
- Clearing notifies too — "resolved" closes the loop, so a silent recovery is not mistaken for an unresolved problem.
UNKNOWNoutcomes ignore all of the above and notify immediately, every time, until acknowledged. Safety outranks tidiness (BR-9).- Honour the existing master switch and per-trigger enables.
Test coverage — the plan is not done until each is proven¶
Detection that nobody tests is the same as no detection. Each check needs three tests:
- fires — given the bad state, the finding is produced with the right severity and BR;
- stays quiet — given a healthy state, nothing is produced (the false-positive guard,
which matters most for
orphaned_entities, where a wrong answer deletes another integration's entities); - surfaces — the finding reaches the log, the endpoint, and the notifier, with throttling applied.
Plus two regression tests for issues that already shipped silently: version_drift (now
covered) and orphaned_entities (validated against a real 467-config broker capture).
Phasing¶
| Phase | Delivers | Closes |
|---|---|---|
| 1 | wire resilience.call into the scheduler and the bridge's writes |
BR-6–13, BR-16 |
| 2 | self-check engine + GET /api/health/findings + app-log emission |
BR-14, BR-26, BR-28 |
| 3 | Health card + bridge_health notification family + throttling |
BR-29, BR-30 |
| 4 | HA problem binary_sensor + issue-count sensor |
unattended coverage |
| 5 | remaining checks, each with its three tests | the rest of the table |
Phase 2 is the one that matters most, and it is the one that would have caught every silent failure in this list. Phase 1 only covers things that actively go wrong while someone is watching.
Explicitly out of scope¶
Not metrics-grade monitoring, not Prometheus, not an SLA. The bar is: a condition this bridge can detect about itself should be visible without reading source or querying SQLite, and should reach a human or an automation once — not zero times, and not hourly.