How to Choose Direct SQL Alerts or a Prometheus Exporter

Define the condition in business terms

Start with one question such as: how old is the oldest unprocessed import? Record the required observation delay and first response. A periodic capacity trend and an imminent processing deadline need different evaluation intervals.

Then identify the cheapest trustworthy data source. A full scan of a busy transaction table may be unnecessary when a maintained summary or queue-state table already answers the question. Review the query with the database owner before scheduling it repeatedly.

Compare the two collection paths

An exporter is useful when a small stable set of numeric measurements serves many consumers. It can be unnecessary machinery for one inexpensive, well-bounded query. Conversely, dozens of duplicated direct queries may create avoidable database work.

Make the direct query compatible with alert evaluation

Grafana's SQL Server alerting documentation (opens in a new tab) currently specifies numeric time-series queries with a time column and sorted results. Do not assume a table that renders in a dashboard is an alert-compatible query.

Use a read-only monitoring identity with only the required permissions. Set a timeout and an interval that the database can sustain. A SELECT can still cause expensive reads or contention; read-only access is not a performance guarantee.

Avoid dashboard variables that depend on an interactive user's selection. A background rule needs explicit scope. State how an empty result differs from a legitimate zero, and verify the actual evaluator's behavior for both.

Make exported measurements trustworthy

Choose metric names, units and labels deliberately. Export a count or age, not a separate time series per order or customer. Track the last successful database collection separately from the exporter's ability to answer an HTTP request.

If the exporter caches results, expose their age or an explicit freshness condition. Otherwise a responsive exporter can continue serving yesterday's healthy value. Scrape health and business-condition health need different runbook steps.

Document the interval across database collection, scrape and evaluation. The total detection delay includes all of them, plus any condition duration and notification grouping delay.

Pilot against known fixtures

In a test database, prepare a healthy queue, an overdue item, no rows, a delayed query and a denied permission. For each path, measure query duration, database work, returned value and time to a review notification.

Compare the results over the same time window and scope. Verify that zero, missing data and collection failure remain distinguishable. If both architectures pass, prefer the one the owning team can diagnose and maintain with less operational work.

Keep the chosen query, schema assumptions and test data definition with the runbook. Re-run the pilot after schema, exporter or evaluation changes; successful dashboard rendering alone is insufficient validation for a production alert.