Turn Webhook Logs into a State View That Can Go Stale
Sources and verification
- Source dates
- Oldest source check: 12 September 2026.
- Technical verification
- Separate technical verification has not been recorded.
Publication approval and technical verification are recorded separately. Automated link checks establish reachability, not accuracy. A checked example verifies only its stated test cases, not the whole article or your production setup.
In short
Store a last observation and a freshness deadline for each status key. Compute age when the status is read, rather than trusting a producer-supplied “OK two hours ago” string. Separate failed, stale, and unknown states, reject obsolete updates, and monitor the receiver itself. Otherwise a stopped script can leave a permanently green tile backed by yesterday’s message.
Key takeaways
- Freshness belongs to each monitored key, not to one global dashboard timestamp.
- A delayed old success must not overwrite a newer failure.
- The status receiver and renderer need their own independent checks.
Define a status record instead of a display string
A webhook channel records that something happened. A current-state view must also decide whether the evidence is still recent enough to trust. Start with one stable key per job or service and a declared expectation for how often it should report.
{
"key": "nightly-catalog-backup",
"state": "ok",
"run_id": "catalog-2026-09-12-01",
"observed_at": "2026-09-12T02:14:00Z",
"received_at": "2026-09-12T02:14:03Z",
"max_age_seconds": 97200,
"evidence_ref": "backup-catalog-2026-09-12-01"
}
This is an example application contract, not an existing provider API. The receiver should set received_at itself. Keep the expected maximum age in trusted configuration so a faulty producer cannot silently increase its own deadline. Restrict each producer credential to the keys it is allowed to update.
Compute state from evidence and time
Define unknown as no accepted observation, failed as a recent explicit failure, and stale as evidence older than the allowed age. Preserve the last reported state alongside freshness: “last report failed; now stale” conveys more than overwriting everything with a grey tile.
Calculate elapsed age from timestamps when rendering and refresh it while the page stays open. A static HTML page with a green word and no age calculation can freeze just as easily as the original webhook log. If using client-side age calculation, show when the data was fetched and account for an obviously incorrect client clock. A server-computed expiry time plus periodic refresh is often easier to inspect.
Use a scheduler outside the viewing browser for alerts. Nobody should have to keep a tab open for a missed report to be detected. Heartbeat deadline checks (opens in a new tab) illustrate the missing-arrival part of this design; arbitrary numeric values still require a suitable store and renderer.
Handle retries and out-of-order messages
Authenticate updates and deduplicate repeated run IDs. Preserve observation time separately from receipt time so a message delayed in transit does not look newly measured. Choose an ordering rule appropriate to the producer: a monotonic sequence within an identified producer epoch is stronger than loosely synchronized wall clocks.
If an older success arrives after a newer failure, retain it in history without replacing current state. Define what happens after a producer restarts and its sequence resets. Reject timestamps far in the future, since they can otherwise keep a record fresh indefinitely.
Test the silent failure modes
| Injection | Expected view and alert |
|---|---|
| Stop one producer | Only its key expires |
| Replay an old success | Current failure remains current |
| Stop the ingestion worker | External receiver watchdog alarms |
| Keep the page open without new data | Age advances and freshness expires |
| Return a cached old status document | Data age is visibly old |
Keep a compact audit history and link detailed logs separately. Back up configuration and the current state where useful, but never restore an old green snapshot as fresh evidence. After recovery, show its original timestamps and wait for new observations. That keeps a useful overview from becoming another place where silence looks healthy.
Did this help?
Your answer helps us improve this guide. We save only the page and your choice for 30 days.
No name, email, or incident details are requested.
Sources
Vendor facts change. Each source below shows the date this page last checked it.
- Configuring deadline and heartbeat checks — Healthchecks. Checked 12 September 2026.
Related
One practical idea, occasionally
The On-Call Brief: short field notes, templates, and operational lessons.
Follow the field guide
Email subscriptions are not open yet. Read new guides in your feed reader — no email address needed.
Subscribe with RSS