
Slow queries rarely fail loudly. They usually degrade: p95 latency creeps up, one endpoint starts timing out, queue depth rises, and suddenly you’re debugging a “random” incident. The fix is to set alerts for slow SQL query times that trigger early, with low noise, and without adding load or risky write operations to production.
This guide focuses on practical, production-safe patterns you can implement with read-only queries against PostgreSQL or MySQL. You’ll learn what to measure, how to pick thresholds, and how to write alert queries that won’t page you every time traffic spikes.
What “slow” really means: pick the right signal
Before writing any alert, decide what you’re protecting. “Slow query time” can mean at least four different things:
- Per-query latency (e.g., a specific query fingerprint exceeds 300ms p95)
- Database-wide latency (overall p95 increases because of contention)
- Wait time (locks, I/O, CPU saturation)
- Throughput collapse (TPS drops while latency rises)
For alerting, you typically want a combination of:
- Top slow query fingerprints (the “who”)
- Trend vs baseline (the “is it abnormal”)
- Impact (calls per minute × latency, or % of total time)
Alert threshold strategies that reduce noise
Static thresholds (“alert if avg > 500ms”) are simple but noisy. Better options:
| Strategy | Example rule | Best for | Risk |
|---|---|---|---|
| Static threshold | p95 > 300ms for 10 min | Known SLOs | Noise during traffic shifts |
| Baseline multiplier | p95 > 2× 7-day median | Seasonal systems | Needs enough history |
| Error budget coupling | Latency high and timeouts rising | User-impacting issues | May miss early warnings |
| Top-N contribution | One query > 20% of total DB time | Regression detection | Requires per-fingerprint stats |
Recommended starting rule (works in many teams)
- Alert when a query fingerprint exceeds a latency threshold (e.g., p95 > 300–800ms) and
- It has meaningful volume (e.g., > 30 calls in the last interval) and
- It’s abnormal vs baseline (e.g., > 2× its 7-day median)
This avoids paging on rare ad-hoc queries or low-traffic endpoints.
PostgreSQL: alerting using pg_stat_statements
If you can enable it, pg_stat_statements is the most practical read-only source for query performance by fingerprint. It aggregates stats such as total time, mean time, calls, and more (exact columns vary by Postgres version).
Tip: Keep alert queries lightweight. Don’t run heavy text processing or full scans of
pg_stat_activityat high frequency. Prefer aggregated views likepg_stat_statements.
1) Find the worst offenders by mean time
Use this to build a “top slow queries” panel or a scheduled check. It’s also a good first alert.
-- Top slow query fingerprints (Postgres)
-- Requires pg_stat_statements
SELECT
queryid,
calls,
mean_time AS mean_ms,
total_time AS total_ms,
rows,
LEFT(query, 120) AS sample
FROM pg_stat_statements
WHERE calls >= 50
ORDER BY mean_time DESC
LIMIT 10;
Alert idea: trigger if any row returns with mean_ms over your threshold and calls over a minimum volume.
2) Alert on regression: “this query got slower than its baseline”
To compare against a baseline, you need historical snapshots. A common pattern is to store periodic snapshots of pg_stat_statements in a separate monitoring database (or time-series store). If you can’t store history, you can still alert on absolute thresholds, but regression alerts are more accurate.
Example schema for snapshots:
-- One-time setup (run in a monitoring schema/database)
CREATE TABLE IF NOT EXISTS query_perf_snapshots (
ts timestamptz NOT NULL,
queryid bigint NOT NULL,
calls bigint NOT NULL,
total_time double precision NOT NULL,
mean_time double precision NOT NULL,
PRIMARY KEY (ts, queryid)
);
Then your scheduler inserts snapshots (read-only on production; write only to monitoring storage). Your alert query can compare “last 15 minutes” vs “7-day baseline”:
-- Regression alert example (conceptual)
WITH recent AS (
SELECT queryid,
SUM(calls) AS calls,
SUM(total_time) / NULLIF(SUM(calls),0) AS mean_ms
FROM query_perf_snapshots
WHERE ts > now() - interval '15 minutes'
GROUP BY queryid
), baseline AS (
SELECT queryid,
percentile_cont(0.5) WITHIN GROUP (ORDER BY mean_time) AS baseline_median_ms
FROM query_perf_snapshots
WHERE ts > now() - interval '7 days'
GROUP BY queryid
)
SELECT r.queryid, r.calls, r.mean_ms, b.baseline_median_ms
FROM recent r
JOIN baseline b USING (queryid)
WHERE r.calls >= 30
AND r.mean_ms > 2.0 * b.baseline_median_ms
AND r.mean_ms > 300
ORDER BY r.mean_ms DESC
LIMIT 20;
How to alert: if this query returns any rows, send an alert including queryid, current mean, baseline median, and call count.
MySQL: alerting using Performance Schema and/or slow query log
MySQL environments vary more: some teams rely on the slow query log, others on Performance Schema summary tables. If you want query fingerprint stats without parsing log files, Performance Schema is usually the best starting point.
1) Top statements by average latency (Performance Schema)
-- Top slow statement digests (MySQL)
SELECT
DIGEST AS digest_id,
DIGEST_TEXT AS sample,
COUNT_STAR AS calls,
(AVG_TIMER_WAIT/1000000000000) AS avg_s
FROM performance_schema.events_statements_summary_by_digest
WHERE COUNT_STAR >= 50
ORDER BY AVG_TIMER_WAIT DESC
LIMIT 10;
Alert idea: if any digest exceeds an avg_s threshold and has enough calls, alert.
2) Using the slow query log for “hard” slow events
The slow query log is useful when you specifically want to alert on individual queries exceeding a threshold (e.g., 2s). However, it can be verbose and requires log access and parsing. Prefer it when:
- You need exact examples of slow queries (not just digests)
- Your environment already centralizes MySQL logs
- You want to catch outliers even if averages look fine
How often should you check? (Scheduling patterns)
To set alerts for slow SQL query times effectively, frequency matters. Check too often and you add overhead; check too rarely and the incident is over before you notice.
- Every 1–5 minutes: critical OLTP systems, user-facing APIs
- Every 10–15 minutes: internal tools, analytics backends
- Hourly: batch pipelines where “slow” means “missed SLA”
Also add a simple rule: require the condition to hold for 2 consecutive checks before alerting. This eliminates one-off jitter.
Designing alert output that engineers can act on
A slow-query alert is only useful if it answers “what do I do next?” Include:
- Fingerprint ID (
queryidin Postgres,DIGESTin MySQL) - Current latency metric (mean, p95 if available)
- Baseline comparison (e.g., 2.6× higher than median)
- Volume (calls in the interval)
- Impact (share of total DB time, if you can compute it)
- A sample query snippet (truncated) to help recognition
Example “impact” calculation (Postgres)
-- Estimate impact share: what % of total statement time is this query?
WITH totals AS (
SELECT SUM(total_time) AS all_total_ms
FROM pg_stat_statements
)
SELECT
queryid,
calls,
total_time AS total_ms,
ROUND(100 * total_time / NULLIF(t.all_total_ms,0), 2) AS pct_of_total_time,
LEFT(query, 120) AS sample
FROM pg_stat_statements, totals t
WHERE calls >= 50
ORDER BY pct_of_total_time DESC
LIMIT 10;
Alert idea: notify when a single query exceeds, for example, 20% of total statement time (especially after a deploy).
Security and safety: read-only, least privilege, low load
Monitoring is part of your attack surface. Keep it boring and safe:
- Use read-only database users with permission only to required views/tables.
- Encrypt connections (TLS) and restrict source IPs where possible.
- Avoid running EXPLAIN on production in alerts; it can be expensive and may lock resources depending on the query.
- Cap result sizes with
LIMITand avoid returning full query texts unless necessary. - Monitor the monitors: alert checks should be cheap (milliseconds), not seconds.
Common pitfalls when you set alerts for slow SQL query times
- Alerting on averages only: Averages hide tail latency. If you can, incorporate percentiles from your app/APM, or use outlier-focused logs.
- No volume filter: One slow query run once should not page anyone.
- Ignoring concurrency: Rising latency may be caused by lock contention. Pair slow-query alerts with lock/wait indicators.
- Not resetting/segmenting stats: Aggregated tables can blur “before vs after deploy.” Snapshots help.
- Too many fingerprints: Normalize alerting to top-N by impact, not everything.
A simple runbook checklist (copy/paste)
- Identify the fingerprint from the alert (
queryid/digest). - Confirm scope: is latency up for one query or many?
- Check volume: did call rate spike (traffic change) or did performance regress?
- Look for contention: locks, CPU, I/O saturation, connection pool exhaustion.
- Compare to last deploy: schema/index changes? query plan changes?
- Mitigate: add/adjust indexes, rewrite query, reduce result set, add caching, or roll back.
- Prevent: add a regression alert with baseline, and a top-N impact alert.
Putting it together
To reliably set alerts for slow SQL query times, aim for alerts that are (1) low-impact on production, (2) based on meaningful volume, and (3) tied to baseline or impact rather than a single noisy threshold. Start with a top-N slow fingerprint check (Postgres pg_stat_statements or MySQL Performance Schema), then add regression logic once you have snapshots.
If you prefer running these checks on a schedule and receiving push/email notifications on iOS, an app like Query Alert can be used to turn read-only performance queries into smart alerts without building a custom service.