Correlate Uptime Checks Across Two Locations
Sources and verification
- Source dates
- Oldest source check: 12 September 2026.
- Technical verification
- Separate technical verification has not been recorded.
Publication approval and technical verification are recorded separately. Automated link checks establish reachability, not accuracy. A checked example verifies only its stated test cases, not the whole article or your production setup.
In short
Collect the same application check from independent locations and correlate only fresh observations from the same time window. Keep location and address-family labels. Two failures strengthen evidence of broad impact, but one regional failure still matters. A missing probe is unknown, not a healthy vote. Define and test the incident rule rather than assuming two monitoring instances coordinate automatically.
Key takeaways
- Independent locations provide context, not automatic proof of a global cause.
- A majority rule can hide a real regional outage.
- Stale or missing observations must not count as healthy votes.
Make the observations comparable
Use the same hostname, method, authentication contract, expected content, timeout, and address family from each location. Record what differs intentionally, such as regional CDN resolution. A check of the login page in one region and a private API in another is not a useful quorum.
Choose locations with different relevant failure domains. Two virtual machines in one data center may share transit, power, or a provider control plane. Independence is a property of the network path and operation, not just having two process names.
The multi-target exporter pattern (opens in a new tab) is one way to collect protocol observations separately from their presentation. Whatever implementation you choose, preserve location identity through storage and alert evaluation.
Correlate a bounded time window
Store target, location, observation time, receipt time, protocol, address family, and result. Define a maximum observation age and tolerated clock skew. If locations run once per minute, a short comparison window can be useful, but choose its width using the actual schedule and delay. Never compare today's failed observation with yesterday's successful one.
Check the probes themselves against a small set of reference endpoints. If a location cannot reach any reference, classify that location's target observations as suspect and report a monitoring-path problem. This should not erase independently confirmed target failures elsewhere.
Use more than a binary global vote
| Location A | Location B | Useful interpretation |
|---|---|---|
| Fresh success | Fresh success | Tested paths currently healthy |
| Fresh failure | Fresh failure | Broad failure evidence; investigate shared dependencies |
| Fresh failure | Fresh success | Regional or path-specific failure |
| Fresh success | Missing or stale | One healthy path; coverage incomplete |
| Fresh failure | Missing or stale | Confirmed path failure plus uncertainty |
These are proposed decision states, not built-in Uptime Kuma coordination features. Two independent instances do not necessarily share observations or incident keys. Use a supported aggregator or implement a small explicit rule layer with a clear owner.
A two-of-three rule may reduce noisy global paging, but it must not hide the customers served by the failing region. Route regional degradation to an appropriate incident or investigation policy even when the global condition does not fire.
Preserve diagnostic detail
Attach resolved address and failure stage to the incident. Different CDN edges, WAF policy, regional DNS, and IPv6 availability can produce different outcomes without the application origin being completely down. Keep expected protocol settings (opens in a new tab) aligned when upgrading probes.
Test the voting rule with controlled failures
Use fixtures that fail everywhere, fail only from one permitted test network, and stop reporting from one probe. Also replay a delayed healthy observation after a fresh failure. Verify that the aggregate view represents each scenario accurately and that the delayed sample cannot clear a current incident.
Record which observations led to the decision and when they were accepted. After a probe recovers, wait for a new target observation before restoring its vote. This makes regional monitoring useful evidence instead of a mechanism that turns incomplete coverage into an unjustified green status.
Did this help?
Your answer helps us improve this guide. We save only the page and your choice for 30 days.
No name, email, or incident details are requested.
Sources
Vendor facts change. Each source below shows the date this page last checked it.
- Prometheus multi-target exporter pattern — Prometheus. Checked 12 September 2026.
- Blackbox exporter probe configuration — Prometheus. Checked 12 September 2026.
Related
One practical idea, occasionally
The On-Call Brief: short field notes, templates, and operational lessons.
Follow the field guide
Email subscriptions are not open yet. Read new guides in your feed reader — no email address needed.
Subscribe with RSS