Custom Middleware Development

Module 22: Web Frameworks I (JavaScript) - Tuesday: Express.js Advanced

Understanding Middleware Architecture

Middleware is at the heart of Express.js, forming a pipeline that processes requests and responses. Before diving into creating custom middleware, let's understand how the middleware architecture works.

flowchart LR A[Client Request] --> B[Middleware 1] B --> C[Middleware 2] C --> D[Middleware 3] D --> E[Route Handler] E --> F[Middleware 3] F --> G[Middleware 2] G --> H[Middleware 1] H --> I[Client Response] style B fill:#a1c2ff,stroke:#333,stroke-width:2px style C fill:#a1c2ff,stroke:#333,stroke-width:2px style D fill:#a1c2ff,stroke:#333,stroke-width:2px style E fill:#f9d71c,stroke:#333,stroke-width:2px style F fill:#a1c2ff,stroke:#333,stroke-width:2px style G fill:#a1c2ff,stroke:#333,stroke-width:2px style H fill:#a1c2ff,stroke:#333,stroke-width:2px

The Assembly Line Analogy

Think of Express middleware as an assembly line in a factory:

  • Raw Materials (client request) enter the factory
  • Workstations (middleware functions) process the materials one by one
  • Each station can either:
    • Modify the materials and pass them to the next station
    • Reject defective materials (sending an error response)
    • Finish the product early and send it out (sending a response)
  • Final Assembly (route handler) completes the main product
  • Quality Control (response middleware) checks the finished product
  • Shipping sends the finished product (response) back to the customer

Just like how different factories can have different assembly lines optimized for different products, Express applications can have different middleware stacks for different routes or endpoints.

Middleware Function Anatomy

An Express middleware function has a specific signature and follows particular patterns:

Basic Middleware Structure


// Basic middleware function
function myMiddleware(req, res, next) {
  // 1. Do something with the request
  console.log(`${req.method} request to ${req.url}`);
  
  // 2. Modify the request or response objects (optional)
  req.customProperty = 'middleware added this';
  
  // 3. End the request-response cycle OR
  // res.send('Response from middleware');
  
  // 4. Call the next middleware in the stack
  next();
}

// Using the middleware
app.use(myMiddleware);
                

This example shows the basic structure of a middleware function and how it's used in an Express application.

Key Components of Middleware

Error-Handling Middleware


// Error-handling middleware has 4 parameters
function errorHandler(err, req, res, next) {
  // Log the error
  console.error(err);
  
  // Send an error response
  res.status(err.status || 500).json({
    error: {
      message: process.env.NODE_ENV === 'production'
        ? 'An unexpected error occurred'
        : err.message
    }
  });
}

// Using error-handling middleware (must be last in the stack)
app.use(errorHandler);
                

Error-handling middleware is identified by having four parameters instead of three. Express treats this as a special case and only calls this middleware when an error is passed to next().

Middleware Execution Flow

Understanding the flow of middleware execution is crucial:

  1. Express executes middleware in the order they're defined with app.use() or router.use()
  2. Execution stops if a middleware doesn't call next() and sends a response
  3. Calling next() with an argument (typically an error) skips to error-handling middleware
  4. Middleware defined after a route handler only runs if the route calls next()

This ordered execution allows you to build processing pipelines for requests, where each step depends on the previous steps.

Creating Purpose-Specific Middleware

Let's explore how to create custom middleware for various common purposes:

Logging Middleware


// Simple request logger
const requestLogger = (req, res, next) => {
  const start = Date.now();
  
  // Log request details
  console.log(`${new Date().toISOString()} - ${req.method} ${req.url}`);
  
  // Add a property to the response object to calculate duration
  res.on('finish', () => {
    const duration = Date.now() - start;
    console.log(`${new Date().toISOString()} - ${req.method} ${req.url} - ${res.statusCode} - ${duration}ms`);
  });
  
  next();
};

// More advanced structured logger with correlation ID
const structuredLogger = (req, res, next) => {
  // Generate a unique correlation ID for request tracing
  req.correlationId = req.headers['x-correlation-id'] || 
                      `${Date.now()}-${Math.random().toString(36).substr(2, 9)}`;
  
  // Add correlation ID to response headers for tracking
  res.setHeader('X-Correlation-ID', req.correlationId);
  
  const start = Date.now();
  const requestInfo = {
    correlationId: req.correlationId,
    method: req.method,
    url: req.url,
    path: req.path,
    query: req.query,
    headers: {
      userAgent: req.headers['user-agent'],
      contentType: req.headers['content-type'],
      authorization: req.headers.authorization ? '[REDACTED]' : undefined
    },
    timestamp: new Date().toISOString()
  };
  
  // Log request
  console.log(JSON.stringify({
    type: 'request',
    ...requestInfo
  }));
  
  // Capture response info when completed
  res.on('finish', () => {
    const duration = Date.now() - start;
    
    console.log(JSON.stringify({
      type: 'response',
      correlationId: req.correlationId,
      statusCode: res.statusCode,
      duration,
      timestamp: new Date().toISOString()
    }));
  });
  
  next();
};

// Using the middleware
app.use(structuredLogger);
                

This example shows two logging middleware implementations: a simple one and a more advanced structured logger with correlation IDs for request tracking.

Authentication Middleware


// JWT Authentication middleware
const jwt = require('jsonwebtoken');

const authMiddleware = (req, res, next) => {
  // Get the authorization header
  const authHeader = req.headers.authorization;
  
  if (!authHeader) {
    return res.status(401).json({ message: 'Authorization header is missing' });
  }
  
  // Check the format (Bearer [token])
  const parts = authHeader.split(' ');
  if (parts.length !== 2 || parts[0] !== 'Bearer') {
    return res.status(401).json({ message: 'Invalid authorization format. Use: Bearer [token]' });
  }
  
  const token = parts[1];
  
  try {
    // Verify the token (in a real app, use environment variable for secret)
    const decoded = jwt.verify(token, process.env.JWT_SECRET || 'your-secret-key');
    
    // Attach user info to the request
    req.user = decoded;
    
    // Continue to the next middleware
    next();
  } catch (error) {
    if (error.name === 'TokenExpiredError') {
      return res.status(401).json({ message: 'Token has expired' });
    }
    
    return res.status(401).json({ message: 'Invalid token' });
  }
};

// Role-based authorization middleware (to be used after authentication)
const authorize = (roles = []) => {
  // Convert string to array if only one role is provided
  if (typeof roles === 'string') {
    roles = [roles];
  }
  
  return (req, res, next) => {
    // Check if user exists (from auth middleware)
    if (!req.user) {
      return res.status(401).json({ message: 'User is not authenticated' });
    }
    
    // Check if user has required role
    if (roles.length && !roles.includes(req.user.role)) {
      return res.status(403).json({ 
        message: 'Insufficient permissions to access this resource' 
      });
    }
    
    // User has required role, continue
    next();
  };
};

// Using these middleware
app.get('/api/profile', authMiddleware, (req, res) => {
  res.json({ user: req.user });
});

app.get('/api/admin/settings', 
  authMiddleware, 
  authorize(['admin']), 
  (req, res) => {
    res.json({ message: 'Admin settings accessed', settings: {} });
});

app.get('/api/reports', 
  authMiddleware, 
  authorize(['admin', 'manager']), 
  (req, res) => {
    res.json({ message: 'Reports accessed', reports: [] });
});
                

This example shows how to create authentication middleware for JWT verification and role-based authorization middleware to restrict access based on user roles.

Request Validation Middleware


// Basic request validation middleware
const validateUserCreation = (req, res, next) => {
  const { name, email, password } = req.body;
  const errors = [];
  
  // Validate name
  if (!name) {
    errors.push({ field: 'name', message: 'Name is required' });
  } else if (typeof name !== 'string' || name.length < 2) {
    errors.push({ field: 'name', message: 'Name must be at least 2 characters' });
  }
  
  // Validate email
  if (!email) {
    errors.push({ field: 'email', message: 'Email is required' });
  } else if (!/^\S+@\S+\.\S+$/.test(email)) {
    errors.push({ field: 'email', message: 'Email format is invalid' });
  }
  
  // Validate password
  if (!password) {
    errors.push({ field: 'password', message: 'Password is required' });
  } else if (password.length < 8) {
    errors.push({ field: 'password', message: 'Password must be at least 8 characters' });
  } else if (!/[A-Z]/.test(password) || !/[0-9]/.test(password)) {
    errors.push({ field: 'password', message: 'Password must contain at least one uppercase letter and one number' });
  }
  
  // If there are validation errors, return them
  if (errors.length > 0) {
    return res.status(400).json({
      success: false,
      error: {
        message: 'Validation failed',
        details: errors
      }
    });
  }
  
  // Validation passed, proceed to the next middleware
  next();
};

// Using the validation middleware
app.post('/api/users', validateUserCreation, (req, res) => {
  // Create user logic here...
  res.status(201).json({ success: true, message: 'User created successfully' });
});

// Generic validator middleware factory
const validate = (schema) => {
  return (req, res, next) => {
    const { error } = schema.validate(req.body, { abortEarly: false });
    
    if (error) {
      const errors = error.details.map(detail => ({
        field: detail.path.join('.'),
        message: detail.message
      }));
      
      return res.status(400).json({
        success: false,
        error: {
          message: 'Validation failed',
          details: errors
        }
      });
    }
    
    next();
  };
};

// Example using with Joi (you'd need to install joi: npm install joi)
const Joi = require('joi');

const userSchema = Joi.object({
  name: Joi.string().min(2).required(),
  email: Joi.string().email().required(),
  password: Joi.string().min(8).pattern(/^(?=.*[A-Z])(?=.*[0-9])/).required(),
  age: Joi.number().integer().min(18).optional()
});

app.post('/api/users', validate(userSchema), (req, res) => {
  // Create user logic with validated data
  res.status(201).json({ success: true, message: 'User created successfully' });
});
                

This example shows two approaches to request validation: a custom validation middleware and a reusable factory function that can validate requests against schemas (using the popular Joi validation library).

Rate Limiting Middleware


// Simple in-memory rate limiter
const rateLimit = (options) => {
  const {
    windowMs = 60 * 1000, // 1 minute default
    max = 100, // 100 requests per windowMs default
    message = 'Too many requests, please try again later.',
    statusCode = 429, // Too Many Requests
    keyGenerator = (req) => req.ip // Use IP as default identifier
  } = options;
  
  const requests = new Map();
  
  // Cleanup old entries every windowMs
  setInterval(() => {
    const now = Date.now();
    requests.forEach((value, key) => {
      if (now - value.timestamp > windowMs) {
        requests.delete(key);
      }
    });
  }, windowMs);
  
  return (req, res, next) => {
    const key = keyGenerator(req);
    const now = Date.now();
    
    // Get or initialize request count
    const requestData = requests.get(key) || { count: 0, timestamp: now };
    
    // Check if window has passed and reset if needed
    if (now - requestData.timestamp > windowMs) {
      requestData.count = 0;
      requestData.timestamp = now;
    }
    
    // Increment request count
    requestData.count++;
    requests.set(key, requestData);
    
    // Set headers to inform client about rate limit
    res.setHeader('X-RateLimit-Limit', max);
    res.setHeader('X-RateLimit-Remaining', Math.max(0, max - requestData.count));
    res.setHeader('X-RateLimit-Reset', Math.ceil((requestData.timestamp + windowMs) / 1000));
    
    // Check if over limit
    if (requestData.count > max) {
      return res.status(statusCode).json({
        success: false,
        error: {
          message
        }
      });
    }
    
    next();
  };
};

// Using the rate limiter
// Global rate limiter
app.use(rateLimit({
  windowMs: 15 * 60 * 1000, // 15 minutes
  max: 100 // 100 requests per 15 minutes
}));

// More strict rate limiter for login attempts
app.post('/api/auth/login', 
  rateLimit({
    windowMs: 60 * 60 * 1000, // 1 hour
    max: 5, // 5 login attempts per hour
    message: 'Too many login attempts. Try again later.',
    keyGenerator: (req) => {
      // Use email as key for rate limiting login attempts
      return req.body.email || req.ip;
    }
  }),
  (req, res) => {
    // Login logic
    res.json({ token: 'sample-token' });
  }
);
                

This example demonstrates a simple in-memory rate limiting middleware that can be configured with different limits and windows for different routes. In production, you would typically use a more robust solution like express-rate-limit with Redis for distributed rate limiting.

Popular Middleware Packages

While building custom middleware is important, many common use cases already have well-tested packages available:

Purpose Popular Package Key Features
Logging morgan, winston Predefined formats, log rotation, multiple transport options
Authentication passport, express-jwt Multiple authentication strategies, OAuth integration
Validation express-validator, joi Schema validation, sanitization, custom validators
Rate Limiting express-rate-limit Configurable limits, distributed rate limiting with Redis
CORS cors Configurable cross-origin resource sharing

Even when using these packages, understanding how to write custom middleware allows you to extend and customize them to fit your specific needs.

Route-Specific Middleware

Middleware can be applied globally, to specific routes, or to groups of routes using routers:

graph TD A[Express App] --> B[Global Middleware] B --> C[API Router] C --> D[API-specific Middleware] C --> E[/users Router] C --> F[/products Router] E --> G[User Middleware] E --> H[GET /users] E --> I[POST /users] F --> J[Product Middleware] F --> K[GET /products] style A fill:#f9d71c,stroke:#333,stroke-width:2px style B fill:#a1c2ff,stroke:#333,stroke-width:2px style D fill:#a1c2ff,stroke:#333,stroke-width:2px style G fill:#a1c2ff,stroke:#333,stroke-width:2px style J fill:#a1c2ff,stroke:#333,stroke-width:2px

Different Ways to Apply Middleware


const express = require('express');
const app = express();

// Global middleware - applies to all routes
app.use(express.json());
app.use(requestLogger);

// Path-specific middleware - applies to all routes starting with /api
app.use('/api', apiKeyValidator);

// Route-specific middleware - applies only to this route
app.get('/users/:id', authMiddleware, (req, res) => {
  // Route handler
});

// Multiple middleware for a route
app.post('/users',
  authMiddleware,
  validateUserCreation,
  (req, res) => {
    // Route handler
  }
);

// Router-level middleware
const router = express.Router();

// Router-specific middleware - applies to all routes in this router
router.use(specificMiddleware);

router.get('/route1', (req, res) => { /* ... */ });
router.get('/route2', (req, res) => { /* ... */ });

// Mount the router
app.use('/feature', router);
                

This example demonstrates the different ways to apply middleware in Express, from global application to specific routes and routers.

Organizing API Routes with Middleware


// File structure
// routes/
//   api.js
//   api/
//     users.js
//     products.js
//     orders.js
// middleware/
//   auth.js
//   validation.js
//   rateLimiting.js

// server.js
const express = require('express');
const apiRoutes = require('./routes/api');

const app = express();

app.use(express.json());
app.use(express.urlencoded({ extended: false }));
app.use(requestLogger);

// Mount API routes
app.use('/api', apiRoutes);

// Global error handler
app.use(errorHandler);

app.listen(3000);

// routes/api.js
const express = require('express');
const router = express.Router();
const { apiKeyValidator } = require('../middleware/auth');
const userRoutes = require('./api/users');
const productRoutes = require('./api/products');
const orderRoutes = require('./api/orders');

// API-level middleware
router.use(apiKeyValidator);

// Mount resource routes
router.use('/users', userRoutes);
router.use('/products', productRoutes);
router.use('/orders', orderRoutes);

module.exports = router;

// routes/api/users.js
const express = require('express');
const router = express.Router();
const { authMiddleware, authorize } = require('../../middleware/auth');
const { validateUser } = require('../../middleware/validation');
const { userRateLimit } = require('../../middleware/rateLimiting');

// Get all users - admin only
router.get('/', 
  authMiddleware, 
  authorize(['admin']), 
  userController.getAllUsers
);

// Get current user profile
router.get('/me', 
  authMiddleware, 
  userController.getCurrentUser
);

// Create new user
router.post('/', 
  validateUser, 
  userRateLimit, 
  userController.createUser
);

// Update user
router.put('/:id', 
  authMiddleware, 
  validateUser, 
  userController.updateUser
);

module.exports = router;
                

This example demonstrates how to organize an Express application with route-specific middleware, using a modular structure with separate router files for different API resources.

Middleware Stacking Patterns

Various design patterns emerge when working with middleware in real-world applications:

Pipeline Pattern

Create reusable middleware pipelines for common tasks:


// Create a middleware pipeline for protected routes
const protectedRoute = [
  authMiddleware,
  checkPermissions,
  logAccess
];

app.get('/admin', ...protectedRoute, adminController.dashboard);
app.get('/settings', ...protectedRoute, settingsController.view);
                

Conditional Middleware

Apply middleware conditionally based on app configuration:


// Only use certain middleware in production
if (process.env.NODE_ENV === 'production') {
  app.use(securityHeaders);
  app.use(rateLimit);
}

// Advanced conditional middleware
app.use((req, res, next) => {
  if (req.query.debug && process.env.NODE_ENV !== 'production') {
    return debugMiddleware(req, res, next);
  }
  next();
});
                

Middleware Factory Pattern

Create functions that generate specialized middleware:


// Middleware factory for feature flags
const featureFlag = (flagName) => {
  return (req, res, next) => {
    if (isFeatureEnabled(flagName, req.user)) {
      next();
    } else {
      res.status(403).json({ 
        message: 'This feature is not available for your account' 
      });
    }
  };
};

app.get('/beta-feature', 
  authMiddleware, 
  featureFlag('betaTester'), 
  betaController.show
);
                

These patterns help you create more maintainable and reusable middleware in complex applications.

Middleware Communication Patterns

Middleware functions often need to communicate with each other. Let's explore some common patterns:

Using Request Properties


// First middleware adds data to request
const addUserData = (req, res, next) => {
  // In a real app, this might come from a database
  req.userData = {
    preferences: {
      theme: 'dark',
      language: 'en'
    },
    permissions: ['read', 'edit']
  };
  next();
};

// Second middleware uses the added data
const checkEditPermission = (req, res, next) => {
  if (!req.userData || !req.userData.permissions.includes('edit')) {
    return res.status(403).json({ message: 'Edit permission required' });
  }
  next();
};

// Using both middleware
app.put('/documents/:id', 
  authMiddleware,
  addUserData,
  checkEditPermission,
  documentController.update
);
                

This example shows how one middleware can add properties to the request object that subsequent middleware can use.

Using Response Locals


// Add data to res.locals for views
const addViewData = (req, res, next) => {
  // res.locals is designed for passing data to views
  res.locals.user = req.user;
  res.locals.site = {
    title: 'My Express Site',
    description: 'An Express.js application'
  };
  res.locals.nav = [
    { title: 'Home', url: '/' },
    { title: 'About', url: '/about' },
    { title: 'Contact', url: '/contact' }
  ];
  next();
};

// Using view locals in a template engine
app.get('/', addViewData, (req, res) => {
  res.render('home', {
    // Additional page-specific data
    title: 'Welcome to our site'
  });
});

// Example EJS template using res.locals
// <!DOCTYPE html>
// <html>
// <head>
//   <title><%= title %> - <%= site.title %></title>
// </head>
// <body>
//   <nav>
//     <ul>
//       <% nav.forEach(function(item) { %>
//         <li><a href="<%= item.url %>"><%= item.title %></a></li>
//       <% }); %>
//     </ul>
//   </nav>
//   
//   <% if (user) { %>
//     <p>Welcome, <%= user.name %>!</p>
//   <% } %>
// </body>
// </html>
                

This example demonstrates using res.locals to pass data from middleware to views. This is particularly useful for data that needs to be available across multiple views, like user information, navigation menus, or site settings.

Using Middleware Context


// Middleware factory with a shared context
const createAuditContext = () => {
  // Shared context (closure)
  const auditLog = [];
  
  // Return multiple middleware functions sharing the same context
  return {
    // Middleware to setup audit logging
    initialize: (req, res, next) => {
      req.audit = {
        log: (action, details) => {
          auditLog.push({
            timestamp: new Date(),
            user: req.user ? req.user.id : 'anonymous',
            action,
            details,
            ip: req.ip,
            url: req.originalUrl
          });
        },
        // Method to view the current audit log
        getEntries: () => [...auditLog]
      };
      next();
    },
    
    // Middleware to automatically log requests
    logRequest: (req, res, next) => {
      if (req.audit) {
        req.audit.log('request', { method: req.method, path: req.path });
      }
      next();
    },
    
    // Middleware to log responses
    logResponse: (req, res, next) => {
      // Store the original end function
      const originalEnd = res.end;
      
      // Override the end function
      res.end = function(chunk, encoding) {
        // Call the original end function
        originalEnd.call(this, chunk, encoding);
        
        // Log the response
        if (req.audit) {
          req.audit.log('response', { statusCode: res.statusCode });
        }
      };
      
      next();
    },
    
    // Admin middleware to view all audit logs
    viewAuditLog: (req, res) => {
      res.json({ auditLog });
    }
  };
};

// Using the audit middleware
const audit = createAuditContext();

app.use(audit.initialize);
app.use(audit.logRequest);
app.use(audit.logResponse);

// Manual audit logging in a route
app.post('/api/users', authMiddleware, (req, res) => {
  // Create user logic...
  const user = { id: 123, name: req.body.name };
  
  // Log the user creation
  req.audit.log('user_created', { userId: user.id });
  
  res.status(201).json(user);
});

// Admin route to view audit logs
app.get('/api/admin/audit-log', 
  authMiddleware, 
  authorize(['admin']), 
  audit.viewAuditLog
);
                

This example shows a more advanced pattern: creating middleware that shares a common context (the audit log). This allows related middleware functions to communicate with each other while maintaining encapsulation.

Best Practices for Middleware Communication

  1. Use Namespaced Properties: When adding properties to req, use namespaces to avoid conflicts (e.g., req.myApp.userData instead of req.userData)
  2. Document Middleware Dependencies: Clearly document when a middleware depends on properties set by another middleware
  3. Check for Property Existence: Always check if a property exists before using it, as middleware execution order can't always be guaranteed
  4. Avoid Modifying Built-in Objects: Be careful when modifying built-in Express objects and properties
  5. Use Middleware Factories: Create middleware factories for configurable, reusable middleware with shared context

Following these practices will help you create maintainable middleware that works well together in complex applications.

Asynchronous Middleware

Handling asynchronous operations in middleware requires special care to ensure errors are properly caught and handled:

Callback-style Asynchronous Middleware


// Traditional callback-style async middleware
const fetchUserData = (req, res, next) => {
  // Simulate database query
  setTimeout(() => {
    try {
      // Simulate successful query
      req.userData = { id: 123, name: 'John Doe' };
      
      // Continue to next middleware
      next();
    } catch (error) {
      // Pass error to Express error handler
      next(error);
    }
  }, 100);
};

// Using the middleware
app.get('/profile', fetchUserData, (req, res) => {
  res.json({ user: req.userData });
});
                

This example shows a traditional callback-style asynchronous middleware that passes errors to the next middleware using next(error).

Promise-based Middleware with Async/Await


// Async/await middleware (modern approach)
const fetchUserFromDatabase = async (req, res, next) => {
  try {
    // In a real app, this would be a database call
    const user = await User.findById(req.params.id);
    
    if (!user) {
      // Create a custom error
      const error = new Error('User not found');
      error.statusCode = 404;
      throw error;
    }
    
    // Add user to request
    req.user = user;
    next();
  } catch (error) {
    // Pass any errors to the error handler
    next(error);
  }
};

// Request-scoped cache middleware using async/await
const cacheMiddleware = (duration) => {
  const cache = new Map();
  
  return async (req, res, next) => {
    const key = req.originalUrl;
    
    // Check if we have a cached response
    const cachedResponse = cache.get(key);
    if (cachedResponse && Date.now() < cachedResponse.expiresAt) {
      // Return cached response
      return res.json(cachedResponse.data);
    }
    
    // Store original res.json method
    const originalJson = res.json;
    
    // Override res.json method to cache the response
    res.json = function(data) {
      // Cache the result
      cache.set(key, {
        data,
        expiresAt: Date.now() + duration
      });
      
      // Call the original method
      return originalJson.call(this, data);
    };
    
    next();
  };
};

// Using the middleware
app.get('/api/products', 
  cacheMiddleware(60 * 1000), // Cache for 1 minute
  async (req, res, next) => {
    try {
      const products = await Product.find();
      res.json(products);
    } catch (err) {
      next(err);
    }
  }
);
                

This example shows modern async/await middleware with proper error handling. It also demonstrates a more complex example: a caching middleware that intercepts and caches responses.

Error Handling in Async Middleware


// Utility to wrap async handlers and catch errors
const asyncHandler = (fn) => (req, res, next) => {
  Promise.resolve(fn(req, res, next)).catch(next);
};

// Using the utility
app.get('/api/users/:id', asyncHandler(async (req, res) => {
  const user = await User.findById(req.params.id);
  
  if (!user) {
    const error = new Error('User not found');
    error.statusCode = 404;
    throw error;
  }
  
  res.json(user);
}));

// Multiple async middleware
app.get('/api/orders/:id', 
  asyncHandler(async (req, res, next) => {
    const order = await Order.findById(req.params.id);
    
    if (!order) {
      const error = new Error('Order not found');
      error.statusCode = 404;
      throw error;
    }
    
    req.order = order;
    next();
  }),
  asyncHandler(async (req, res, next) => {
    // Fetch related data
    const items = await OrderItem.find({ orderId: req.order.id });
    req.orderItems = items;
    next();
  }),
  asyncHandler(async (req, res) => {
    // Combine order with items
    res.json({
      ...req.order.toJSON(),
      items: req.orderItems
    });
  })
);
                

This example shows an asyncHandler utility that simplifies error handling in async middleware by automatically catching and passing errors to the next function. This pattern is common in Express applications and is also implemented by libraries like express-async-handler.

Async Middleware Best Practices

  1. Always Handle Errors: Never leave async operations without proper error handling
  2. Use Try/Catch with Async/Await: Always wrap async code in try/catch blocks and call next(error)
  3. Consider Wrapper Utilities: Use utilities like asyncHandler to reduce boilerplate code
  4. Be Mindful of the Event Loop: Long-running async operations can block the server; consider offloading to worker threads
  5. Avoid Mixing Patterns: Stick with promises and async/await rather than mixing with callbacks

Major companies like Airbnb and Netflix follow these patterns for handling async middleware in their Node.js services to ensure robust error handling and maintainability.

Testing Middleware

Properly testing middleware ensures it behaves as expected in various scenarios:

Unit Testing Middleware with Jest


// middleware/auth.js
const jwt = require('jsonwebtoken');

const authMiddleware = (req, res, next) => {
  const token = req.headers.authorization?.split(' ')[1];
  
  if (!token) {
    return res.status(401).json({ message: 'Authentication required' });
  }
  
  try {
    const decoded = jwt.verify(token, process.env.JWT_SECRET);
    req.user = decoded;
    next();
  } catch (error) {
    return res.status(401).json({ message: 'Invalid token' });
  }
};

module.exports = authMiddleware;

// tests/middleware/auth.test.js
const authMiddleware = require('../../middleware/auth');
const jwt = require('jsonwebtoken');

// Mock jwt.verify
jest.mock('jsonwebtoken');

describe('Auth Middleware', () => {
  let req, res, next;
  
  beforeEach(() => {
    // Create fresh mocks for each test
    req = {
      headers: {}
    };
    
    res = {
      status: jest.fn().mockReturnThis(),
      json: jest.fn()
    };
    
    next = jest.fn();
  });
  
  test('should return 401 if no authorization header is present', () => {
    // Call the middleware
    authMiddleware(req, res, next);
    
    // Check expectations
    expect(res.status).toHaveBeenCalledWith(401);
    expect(res.json).toHaveBeenCalledWith({ message: 'Authentication required' });
    expect(next).not.toHaveBeenCalled();
  });
  
  test('should return 401 if token format is invalid', () => {
    // Set invalid token format
    req.headers.authorization = 'InvalidFormat';
    
    // Call the middleware
    authMiddleware(req, res, next);
    
    // Check expectations
    expect(res.status).toHaveBeenCalledWith(401);
    expect(res.json).toHaveBeenCalledWith({ message: 'Authentication required' });
    expect(next).not.toHaveBeenCalled();
  });
  
  test('should return 401 if token is invalid', () => {
    // Set up a token but make verification fail
    req.headers.authorization = 'Bearer invalidtoken';
    jwt.verify.mockImplementation(() => {
      throw new Error('Invalid token');
    });
    
    // Call the middleware
    authMiddleware(req, res, next);
    
    // Check expectations
    expect(jwt.verify).toHaveBeenCalled();
    expect(res.status).toHaveBeenCalledWith(401);
    expect(res.json).toHaveBeenCalledWith({ message: 'Invalid token' });
    expect(next).not.toHaveBeenCalled();
  });
  
  test('should call next and set req.user if token is valid', () => {
    // Set up a valid token
    const user = { id: '123', role: 'user' };
    req.headers.authorization = 'Bearer validtoken';
    jwt.verify.mockReturnValue(user);
    
    // Call the middleware
    authMiddleware(req, res, next);
    
    // Check expectations
    expect(jwt.verify).toHaveBeenCalled();
    expect(req.user).toEqual(user);
    expect(next).toHaveBeenCalled();
    expect(res.status).not.toHaveBeenCalled();
    expect(res.json).not.toHaveBeenCalled();
  });
});
                

This example demonstrates unit testing middleware with Jest, using mocks to isolate the middleware from its dependencies and testing various scenarios including success and error cases.

Integration Testing Middleware


// Integration tests with supertest
const request = require('supertest');
const express = require('express');
const authMiddleware = require('../../middleware/auth');
const jwt = require('jsonwebtoken');

describe('Auth Middleware Integration', () => {
  let app;
  
  beforeEach(() => {
    // Create a fresh Express app for each test
    app = express();
    
    // Set up a test route that uses the middleware
    app.get('/protected', authMiddleware, (req, res) => {
      res.json({ user: req.user });
    });
    
    // Set up a route to generate test tokens
    app.get('/generate-token', (req, res) => {
      const user = { id: '123', role: 'user' };
      const token = jwt.sign(user, process.env.JWT_SECRET || 'test-secret');
      res.json({ token });
    });
  });
  
  test('should return 401 when no token is provided', async () => {
    const response = await request(app)
      .get('/protected')
      .expect(401);
    
    expect(response.body.message).toBe('Authentication required');
  });
  
  test('should return 401 when invalid token is provided', async () => {
    const response = await request(app)
      .get('/protected')
      .set('Authorization', 'Bearer invalidtoken')
      .expect(401);
    
    expect(response.body.message).toBe('Invalid token');
  });
  
  test('should allow access when valid token is provided', async () => {
    // First get a valid token
    const tokenResponse = await request(app)
      .get('/generate-token')
      .expect(200);
    
    const { token } = tokenResponse.body;
    
    // Then try to access protected route
    const response = await request(app)
      .get('/protected')
      .set('Authorization', `Bearer ${token}`)
      .expect(200);
    
    expect(response.body.user).toHaveProperty('id', '123');
    expect(response.body.user).toHaveProperty('role', 'user');
  });
});
                

This example shows integration testing middleware using supertest, which allows testing the middleware in the context of actual HTTP requests to an Express application.

Middleware Testing Strategies

Different types of middleware may require different testing approaches:

Middleware Type Testing Approach Key Aspects to Test
Authentication/Authorization Unit + Integration Token validation, permissions checking, error handling
Validation Unit Various input scenarios, error messages, edge cases
Logging Unit with mocks Log content, formatting, correct level of detail
Error handling Integration Error transformation, proper status codes, consistent format
Rate limiting Integration with time mocks Correct counting, timeout behavior, headers

Companies with high-quality Express applications, like Shopify and Airbnb, typically maintain test coverage of 80% or higher for their middleware, as it forms the critical infrastructure of their APIs.

Practical Exercise: Custom Middleware Suite

Let's apply what we've learned by creating a suite of custom middleware for a real-world application:

Custom Middleware Exercise

Objective: Create a set of custom middleware functions to enhance an Express API with advanced features.

Requirements:

  1. Create the following middleware:
    • A structured logger that logs requests and responses with correlation IDs
    • A request validator middleware factory that accepts Joi schemas
    • A role-based authorization middleware
    • A simple cache middleware using in-memory storage
    • An error handler that provides consistent error responses
  2. Organize the middleware into separate files in a middleware directory
  3. Write unit tests for each middleware
  4. Create a sample API that demonstrates the middleware in action

Project Structure:


middleware-exercise/
├── package.json
├── server.js
├── middleware/
│   ├── logger.js
│   ├── validator.js
│   ├── authorization.js
│   ├── cache.js
│   └── errorHandler.js
├── routes/
│   └── api.js
└── tests/
    └── middleware/
        ├── logger.test.js
        ├── validator.test.js
        ├── authorization.test.js
        ├── cache.test.js
        └── errorHandler.test.js
                

Bonus Challenge: Add a feature toggle middleware that enables/disables specific API features based on configuration, and middleware to collect performance metrics for each route.

Further Resources