How to Monitor MongoDB Collection Count with Alerts: A Practical Guide

Need to monitor MongoDB collection count with alerts so you can react the moment a collection gets too big, too small, or grows at an unusual rate? Whether you’re tracking failed jobs, unprocessed orders, or the size of a queue, reliable count-based alerts are one of the simplest ways to surface data issues before they become incidents.

This guide walks through three proven approaches—scheduled queries (polling), change streams, and Atlas Scheduled Triggers—plus code examples, trade-offs, and best practices to keep your alerting fast, accurate, and low-noise.

What “collection count” really means (and common pitfalls)

Before wiring alerts, choose the right counting method and understand its implications:

• countDocuments(filter): Accurate count that respects a query filter and can use indexes. Best for threshold checks on a subset of documents (e.g., status: failed).
• estimatedDocumentCount(): Very fast estimate of total documents using collection metadata. Not filterable and not exact. Good for coarse capacity alerts when exactness isn’t critical.
• Read concern and timing: Counts reflect the state at the read concern you choose. For operational alerts, majority read concern is often appropriate to avoid flapping on unreplicated writes.
• Indexing matters: If you count with a filter, ensure the predicate is indexed. Otherwise, the count can be slow and expensive on large collections.
• Sharded clusters: Counts fan out across shards; accurate but potentially more costly. Use targeted filters (with shard keys where possible) to keep it efficient.

Strategy 1: Scheduled queries (polling) with thresholds and hysteresis

When to use: You want a simple, reliable method that works on any deployment (replica set or sharded, Atlas or self-managed) without enabling special features.

How it works: Run a periodic job that executes countDocuments (or estimatedDocumentCount), compares the result to thresholds, and sends an alert when the condition is met.

Pattern tips

• Hysteresis: Use separate high and low thresholds (e.g., alert when > 10,000; clear when back < 9,500) to prevent alert flapping.
• Grace period: Require N consecutive breaches before alerting, to smooth transient spikes.
• Dedupe: Emit the same alert at most once per state change.
• Security: Use a read-only user, TLS, and an IP allowlist. Never embed credentials directly in code.

Example: Python polling with a webhook alert

import os, json, time
from datetime import datetime
from pymongo import MongoClient
import requests

MONGO_URI = os.getenv("MONGODB_URI")
DB_NAME = os.getenv("DB_NAME", "app")
COLL_NAME = os.getenv("COLL_NAME", "jobs")
WEBHOOK_URL = os.getenv("WEBHOOK_URL")  # Slack/Teams/Custom
FILTER = {"status": "failed"}          # index this field!

HIGH = int(os.getenv("THRESH_HIGH", "5000"))
LOW  = int(os.getenv("THRESH_LOW",  "4500"))  # hysteresis threshold
CONSECUTIVE = int(os.getenv("CONSECUTIVE", "2"))  # grace periods

STATE_FILE = os.getenv("STATE_FILE", "/tmp/mongo_count_state.json")

client = MongoClient(MONGO_URI, readConcernLevel="majority")
coll = client[DB_NAME][COLL_NAME]

def load_state():
    try:
        with open(STATE_FILE, "r") as f:
            return json.load(f)
    except:
        return {"alerting": False, "breaches": 0}

def save_state(s):
    with open(STATE_FILE, "w") as f:
        json.dump(s, f)

def send_alert(message):
    if not WEBHOOK_URL:
        print("ALERT:", message)
        return
    requests.post(WEBHOOK_URL, json={"text": message}, timeout=5)

state = load_state()

count = coll.count_documents(FILTER)  # uses index if available
now = datetime.utcnow().isoformat() + "Z"
print(f"[{now}] failed_count={count}")

if not state["alerting"]:
    if count > HIGH:
        state["breaches"] += 1
        if state["breaches"] >= CONSECUTIVE:
            send_alert(f"MongoDB alert: failed jobs count {count} > {HIGH}")
            state["alerting"] = True
            state["breaches"] = 0
    else:
        state["breaches"] = 0
else:
    # currently alerting; wait to recover below LOW
    if count < LOW:
        send_alert(f"Recovered: failed jobs count {count} < {LOW}")
        state["alerting"] = False
        state["breaches"] = 0

save_state(state)

Scheduling: Run every 1–5 minutes via cron, a containerized job (Kubernetes CronJob), or a serverless scheduler (GitHub Actions, Cloud Scheduler, Lambda + EventBridge).

When to prefer: You need simplicity, read-only access, and low operational overhead. This is the most universal approach to monitor MongoDB collection count with alerts.

Strategy 2: Event-driven counts with MongoDB change streams

When to use: You want near-real-time notifications without frequent polling, and your deployment supports change streams (replica set or sharded cluster).

How it works: Watch for insert/delete/replace events and maintain a running count. Emit alerts when thresholds are crossed. This minimizes query load and reduces latency.

Important considerations

• Unfiltered vs filtered counts: Unfiltered total counts are easy to maintain by adjusting on inserts/deletes. Filtered counts are trickier on deletes because you don’t get the deleted document by default. To support filtered deletes, enable pre-image capture (record pre-images) or periodically recompute the count to correct drift.
• Resume tokens: Store the latest resume token so the stream can continue after restarts without losing events.
• Backstops: Periodically re-check with countDocuments (e.g., hourly) to reconcile any missed events and ensure accuracy.

Example: Node.js change stream for unfiltered count

const { MongoClient } = require('mongodb');
const fetch = require('node-fetch');

const uri = process.env.MONGODB_URI;
const dbName = process.env.DB_NAME || 'app';
const collName = process.env.COLL_NAME || 'jobs';
const webhook = process.env.WEBHOOK_URL;

const HIGH = parseInt(process.env.THRESH_HIGH || '100000', 10);
const LOW  = parseInt(process.env.THRESH_LOW  || '95000', 10);

let alerting = false;

async function sendAlert(text) {
  if (!webhook) { console.log('ALERT:', text); return; }
  await fetch(webhook, { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ text }) });
}

(async () => {
  const client = new MongoClient(uri, { readConcern: { level: 'majority' } });
  await client.connect();
  const coll = client.db(dbName).collection(collName);

  let count = await coll.estimatedDocumentCount(); // total only

  const pipeline = [
    { $match: { operationType: { $in: ['insert', 'delete', 'replace'] } } }
  ];

  const changeStream = coll.watch(pipeline, { fullDocument: 'default' });

  const reconcile = async () => {
    const c = await coll.estimatedDocumentCount();
    count = c; // correct drift
  };

  setInterval(reconcile, 60 * 60 * 1000); // hourly backstop

  changeStream.on('change', async (change) => {
    if (change.operationType === 'insert') count += 1;
    else if (change.operationType === 'delete') count -= 1;
    // 'replace' keeps count the same; included for completeness

    if (!alerting && count > HIGH) {
      await sendAlert(`MongoDB alert: collection size ${count} > ${HIGH}`);
      alerting = true;
    } else if (alerting && count < LOW) {
      await sendAlert(`Recovered: collection size ${count} < ${LOW}`);
      alerting = false;
    }
  });

  console.log('Watching for changes...');
})();

When to prefer: You need near-real-time total count alerts without frequent polling, and you can run a small, always-on service.

Strategy 3: MongoDB Atlas Scheduled Triggers

When to use: You’re on Atlas and want a managed, serverless scheduler that runs JavaScript against your cluster, optionally calling webhooks or emails when counts cross thresholds.

How it works: Create a Scheduled Trigger (e.g., every minute). The trigger function runs countDocuments and posts to a webhook or a notification service when conditions are met.

Example: Atlas scheduled trigger function

exports = async function() {
  const coll = context.services.get("mongodb-atlas").db("app").collection("jobs");
  const count = await coll.countDocuments({ status: "failed" });
  const HIGH = 5000, LOW = 4500;

  // Persist alert state in Secrets or a small collection if desired.
  const kv = context.services.get("mongodb-atlas").db("ops").collection("kv"); // { _id: 'failed_alerting', on: true/false }
  const st = await kv.findOne({ _id: 'failed_alerting' }) || { on: false };

  if (!st.on && count > HIGH) {
    await context.http.post({ url: context.values.get("WEBHOOK_URL"), body: { text: `Failed jobs ${count} > ${HIGH}` }, encodeBodyAsJSON: true });
    await kv.updateOne({ _id: 'failed_alerting' }, { $set: { on: true } }, { upsert: true });
  } else if (st.on && count < LOW) {
    await context.http.post({ url: context.values.get("WEBHOOK_URL"), body: { text: `Recovered: failed jobs ${count} < ${LOW}` }, encodeBodyAsJSON: true });
    await kv.updateOne({ _id: 'failed_alerting' }, { $set: { on: false } }, { upsert: true });
  }
};

When to prefer: You want fully managed scheduling without running your own infrastructure, and you’re comfortable with Atlas’s serverless functions.

Choosing the right approach

Best practices for low-noise, secure alerts

• Index your filter: If you count { status: 'failed' }, add db.collection.createIndex({ status: 1 }).
• Hysteresis and grace periods: Use separate fire/recovery thresholds and require consecutive breaches.
• Rate-limit notifications: Cap alert frequency or implement cool‑downs.
• Read concern and read preference: Use majority for consistent operational decisions; prefer primary or a stable secondary based on your SLA.
• Secure connectivity: TLS, read-only users, IP allowlists, Secrets/Vault for credentials.
• Observability: Log the raw count and decision state; export as metrics to Prometheus/Grafana for trend analysis and capacity planning.
• Roll-up windows: Sometimes absolute count isn’t enough; consider counts over a time window (e.g., last 15 minutes) to detect spikes.

Common count scenarios (queries you can reuse)

1) Total documents

// Fast estimate
db.orders.estimatedDocumentCount();

// Accurate count
db.orders.countDocuments({});

2) Unprocessed jobs

// Index: { status: 1 }
db.jobs.countDocuments({ status: 'queued' });

3) Failed jobs in the last 15 minutes

// Index: { status: 1, failedAt: 1 }
const fifteenMinutesAgo = new Date(Date.now() - 15 * 60 * 1000);
db.jobs.countDocuments({
  status: 'failed',
  failedAt: { $gte: fifteenMinutesAgo }
});

4) Per-tenant counts (multi-tenant apps)

// Index: { tenantId: 1, status: 1 }
db.jobs.aggregate([
  { $match: { status: 'failed' } },
  { $group: { _id: '$tenantId', count: { $sum: 1 } } },
  { $match: { count: { $gt: 100 } } } // tenants over threshold
]);

5) Growth rate (day-over-day)

// Index: { createdAt: 1 }
const today = new Date(); today.setUTCHours(0,0,0,0);
const yesterday = new Date(today.getTime() - 24*60*60*1000);

const todayCount = db.orders.countDocuments({ createdAt: { $gte: today } });
const yestCount  = db.orders.countDocuments({ createdAt: { $gte: yesterday, $lt: today } });
// Alert if todayCount > yestCount * 1.5 (spike) or < 0.5 (drop)

Filtered counts with change streams: two reliable patterns

If you need filtered counts in near real time, use one of these patterns to avoid inaccuracies:

1. Enable pre-images: Turn on pre- and post-image capture for the collection and request fullDocumentBeforeChange: 'required' in your change stream. This lets you evaluate whether a deleted or updated document matched your filter. Be aware of storage overhead and operational considerations.
2. Hybrid approach: Use change streams to react quickly to inserts/updates, but run a periodic countDocuments reconciliation (e.g., every 5–10 minutes). This ensures correctness without pre-images, at the cost of an occasional count scan.

Troubleshooting and performance tips

• Slow counts: Add or refine indexes. Verify the filter is selective; consider a precomputed field to make filtering cheaper.
• Sharded cluster hot spots: If counting a filtered subset, align your filter with the shard key where possible.
• Flapping alerts: Increase hysteresis gap, add grace periods, and confirm read concern consistency.
• Missing alerts with change streams: Ensure your application stores and resumes from the last resume token; add a periodic reconciliation job.
• Security errors: Verify read-only roles include find on the target collection and that TLS and IP rules allow your runner or function environment.

Putting it together

If you just need a dependable solution this week, start with scheduled polling and hysteresis. As your scale or latency needs grow, add a change stream listener for low-latency events and keep a periodic reconciliation to guarantee correctness. Atlas users can offload scheduling to Scheduled Triggers for a fully managed experience.

Prefer a mobile-first workflow? Tools like Query Alert let you schedule read‑only checks against MongoDB and get push or email notifications when thresholds are crossed, without building a backend from scratch.

With the right mix of indexing, thresholds, and delivery channels, you’ll have actionable, low-noise alerts on collection counts—and the confidence to catch issues before customers do.