High-Performance Data Architecture with MongoDB

MongoDB is the database of choice for modern scalable applications due to its flexible JSON-native document model, horizontal sharding capabilities, and rich aggregation framework.

However, poor schema design and missing indexes often lead to slow table-scan queries (COLLSCAN), high CPU utilization, and application timeouts.

In this guide, we break down the engineering principles required to maintain sub-10ms query execution in production.


1. Document Modeling: Embedding vs Referencing

A common mistake in NoSQL is designing schemas either as completely normalized relational tables (too many references) or storing infinitely growing arrays inside a single document (violating the 16MB document limit).

Apply these rules of thumb:

  • •Embed when:
  • • - Data is 1:1 or 1:Few (e.g. an order with up to 10 line items, or a user with 2 address records).
  • • - Data is consistently queried and displayed together.
  • • - The embedded array does not grow unboundedly.
  • •Reference when:
  • • - Data is 1:Many (e.g. a course with thousands of enrolled student logs) or Many:Many.
  • • - Documents frequently mutate independently of the parent record.
  • • - You need to avoid the 16MB BSON size ceiling.

2. The ESR Indexing Rule (Equality, Sort, Range)

Compound indexes are crucial for optimizing multi-criteria queries. When structuring the order of keys in a compound index, always follow the ESR Rule:

  1. 1.E — Equality: Fields queried using exact matches (status: 'PUBLISHED', categoryId: 'cat_web') should come FIRST.
  2. 2.S — Sort: Fields used for ordering results (createdAt: -1) should come SECOND.
  3. 3.R — Range: Fields queried using range filters (score: { $gte: 75 }, date: { $lt: now }) should come LAST.
// Query:
// db.submissions.find({ status: 'APPROVED', score: { $gte: 80 } }).sort({ submittedAt: -1 })

// OPTIMAL INDEX applying the ESR rule:
// 1. Equality: status
// 2. Sort: submittedAt
// 3. Range: score
db.submissions.createIndex({ status: 1, submittedAt: -1, score: 1 });

3. Profiling Queries with `explain("executionStats")`

Never guess why a database query is slow. Use MongoDB's explain() method to inspect the query execution plan:

const stats = await db.collection('blog_posts')
  .find({ status: 'PUBLISHED', categoryName: 'Web Development' })
  .sort({ publishedAt: -1 })
  .explain('executionStats');

console.log('Execution Stage:', stats.executionStats.executionStages.stage);
console.log('Total Docs Examined:', stats.executionStats.totalDocsExamined);
console.log('Total Keys Examined:', stats.executionStats.totalKeysExamined);
console.log('Execution Time (ms):', stats.executionStats.executionTimeMillis);
  • •Red Flag: If totalDocsExamined is much higher than the number of returned documents, or if the stage is COLLSCAN, your query is scanning entire collections sequentially.
  • •Target: Aim for IXSCAN (Index Scan) with totalDocsExamined equal to the return count.

4. Advanced Aggregation Pipelines: `$facet` for Search & Faceting

The aggregation pipeline processes data through sequential stages. Using $facet, you can calculate total match counts, paginated documents, and category distributions in a single round-trip query:

const results = await db.collection('courses').aggregate([
  { $match: { published: true } },
  {
    $facet: {
      paginatedResults: [
        { $sort: { enrolledCount: -1 } },
        { $skip: 0 },
        { $limit: 10 },
        { $project: { title: 1, slug: 1, category: 1, enrolledCount: 1 } }
      ],
      totalCount: [
        { $count: 'total' }
      ],
      categories: [
        { $group: { _id: '$category', count: { $sum: 1 } } }
      ]
    }
  }
]).toArray();

5. Serverless Connection Pooling Best Practices

In serverless environments like Next.js API routes or AWS Lambda, creating a new MongoClient on every incoming request quickly exhausts database connection pools. Always cache the client promise globally:

// lib/db/mongodb.ts
import { MongoClient } from 'mongodb';

const uri = process.env.MONGODB_URI!;
let cachedClient: MongoClient | null = null;

export async function getMongoDb() {
  if (!cachedClient) {
    cachedClient = new MongoClient(uri, {
      maxPoolSize: 10,
      serverSelectionTimeoutMS: 5000,
    });
    await cachedClient.connect();
  }
  return cachedClient.db('prayxis_production');
}

Master database engineering and backend design in the PRAYXIS Full Stack Web Development Track.