Module 16: JavaScript Backend

Express.js Built-in Middleware

Express.js comes with several built-in middleware functions that provide essential functionality for web applications. These middleware functions are part of the Express framework and don't require any additional installations.

Analogy: Think of built-in middleware as the standard equipment that comes with a new car—basic but essential components like headlights, windshield wipers, and a steering wheel. They provide core functionality that almost every application needs, without requiring you to shop for additional parts.

express.json

The express.json() middleware parses incoming requests with JSON payloads. It populates the req.body property with the parsed data.

// Setting up express.json middleware
const express = require('express');
const app = express();

// Parse JSON bodies
app.use(express.json());

// Route handler that uses parsed JSON data
app.post('/api/users', (req, res) => {
  // req.body contains the parsed JSON data
  console.log(req.body);
  
  // You can access specific properties
  const { name, email } = req.body;
  
  res.json({ 
    message: 'User created', 
    user: { name, email } 
  });
});
            

Configuration Options

The express.json() middleware accepts an options object with several configuration parameters:

app.use(express.json({
  // Size limit for the JSON payload (default: '100kb')
  limit: '1mb',
  
  // Only parse objects and arrays (default: true)
  strict: true,
  
  // Content-Type values that should be parsed as JSON (default: 'application/json')
  type: 'application/json',
  
  // Control if detailed error messages should be sent to client (default: true)
  inflate: true,
  
  // Reviver function for JSON.parse (default: null)
  reviver: null
}));
            

Real-world example: REST APIs like Twitter's or GitHub's use express.json() to parse incoming data when you create a new tweet or issue. When you post a new tweet with a JSON payload containing your message and media, this middleware extracts that data so the application can process and store it properly.

express.urlencoded

The express.urlencoded() middleware parses incoming requests with URL-encoded payloads (typically from HTML forms). Like express.json(), it populates the req.body property with the parsed data.

// Setting up express.urlencoded middleware
const express = require('express');
const app = express();

// Parse URL-encoded bodies (default form submissions)
app.use(express.urlencoded({ extended: true }));

// Route handler for a form submission
app.post('/login', (req, res) => {
  // req.body contains the parsed form data
  const { username, password } = req.body;
  
  // Process login
  if (username === 'admin' && password === 'secret') {
    res.send('Login successful');
  } else {
    res.status(401).send('Invalid credentials');
  }
});
            

The extended Option

The extended option determines which library is used to parse the URL-encoded data:

• extended: false - Uses the querystring library, which can only parse simple key-value pairs
• extended: true - Uses the qs library, which can parse nested objects and arrays

// Example of extended parsing difference

// With extended: false
// Form data: user[name]=John&user[email]=john@example.com
// req.body = { 'user[name]': 'John', 'user[email]': 'john@example.com' }

// With extended: true
// Form data: user[name]=John&user[email]=john@example.com
// req.body = { user: { name: 'John', email: 'john@example.com' } }
            

Analogy: If express.json() is like a translator who understands the structured format of JSON documents, express.urlencoded() is like a form-filler who knows how to extract information from traditional paper forms (HTML forms) and organize it into a structured format your application can understand.

express.static

The express.static() middleware serves static files such as HTML, CSS, images, and JavaScript files. It takes the directory path of the files you want to serve as its argument.

// Setting up express.static middleware
const express = require('express');
const path = require('path');
const app = express();

// Serve static files from the 'public' directory
app.use(express.static('public'));

// Files in public directory are now accessible at root URL:
// public/styles.css -> http://localhost:3000/styles.css
// public/images/logo.png -> http://localhost:3000/images/logo.png
// public/js/app.js -> http://localhost:3000/js/app.js

// You can also specify a virtual path prefix
app.use('/assets', express.static('public'));
// Now files are accessible with the /assets prefix:
// public/styles.css -> http://localhost:3000/assets/styles.css

// Using an absolute path is recommended in production
app.use(express.static(path.join(__dirname, 'public')));
            

Configuration Options

The express.static() middleware accepts a second parameter with several configuration options:

app.use(express.static('public', {
  // Set custom caching headers (default: undefined)
  maxAge: '1d',
  
  // Enable or disable etag generation (default: true)
  etag: true,
  
  // Enable or disable Last-Modified headers (default: true)
  lastModified: true,
  
  // Set proper MIME type based on file extension (default: true)
  setHeaders: (res, path, stat) => {
    // Custom header setting function
    if (path.endsWith('.pdf')) {
      res.set('Content-Disposition', 'attachment');
    }
  }
}));
            

Multiple Static Directories

You can serve files from multiple directories by calling express.static() multiple times:

// Serve static files from multiple directories
app.use(express.static('public'));
app.use(express.static('uploads'));
app.use(express.static('vendor'));

// Express looks for files in each directory in the order they are defined
// If a file exists in multiple directories, the first one takes precedence
            

Real-world example: When you visit a website like Airbnb, all the images, CSS styles, and client-side JavaScript you see and interact with are served using static file middleware. Without it, every asset would require custom route handlers, making the application much more complex to build and maintain.

Other Built-in Middleware

express.raw

The express.raw() middleware parses incoming request bodies into a Buffer and populates the req.body property. This is useful for processing binary data.

// Parse raw bodies (like binary data)
app.use(express.raw({
  type: 'application/octet-stream',
  limit: '10mb'
}));

// Route handler for binary data
app.post('/upload-binary', (req, res) => {
  // req.body is a Buffer
  console.log('Received binary data of size:', req.body.length);
  
  // Process the binary data...
  
  res.send('Binary data received');
});
            

express.text

The express.text() middleware parses incoming request bodies into a string and populates the req.body property. This is useful for text-based data formats other than JSON.

// Parse text bodies
app.use(express.text({
  type: 'text/plain',
  limit: '1mb'
}));

// Route handler for text data
app.post('/receive-text', (req, res) => {
  // req.body is a string
  console.log('Received text:', req.body);
  
  // Process the text data...
  
  res.send('Text received');
});
            

express.Router

While not technically middleware in the traditional sense, express.Router is a built-in class that creates modular route handlers. It allows you to group related routes together and apply middleware to specific route groups.

// Creating a router instance
const express = require('express');
const router = express.Router();

// Router-specific middleware
router.use((req, res, next) => {
  console.log('Router middleware:', Date.now());
  next();
});

// Define routes on the router
router.get('/', (req, res) => {
  res.send('API Home');
});

router.get('/users', (req, res) => {
  res.send('Users list');
});

// Mount the router on the app
app.use('/api', router);
// Now routes are accessible as /api/ and /api/users
            

Popular Third-party Middleware

While Express comes with essential built-in middleware, the ecosystem offers numerous third-party middleware packages for additional functionality. These packages extend Express with specialized features for common web application needs.

Analogy: Third-party middleware is like aftermarket parts and accessories for your car. While the built-in components handle basic functionality, these additional parts offer specialized features—like advanced entertainment systems, security alarms, or custom lighting—that enhance the driving experience but aren't necessary for every driver.

morgan - HTTP Request Logger

Morgan is a popular HTTP request logger middleware that logs request details to the console or other output streams.

const express = require('express');
const morgan = require('morgan');
const app = express();

// Use morgan with a predefined format
app.use(morgan('dev'));
// Output: GET /home 200 6.123 ms - 1234

// Other predefined formats:
// tiny: Shows minimal output
// common: Apache common log format
// combined: Apache combined log format
// short: Shorter than default, includes response time

// Custom format
app.use(morgan(':method :url :status :res[content-length] - :response-time ms'));

// Write logs to a file
const fs = require('fs');
const path = require('path');
const accessLogStream = fs.createWriteStream(
  path.join(__dirname, 'access.log'), 
  { flags: 'a' }
);
app.use(morgan('combined', { stream: accessLogStream }));
            

Real-world example: Cloud hosting providers like Heroku use request logging to track application performance and diagnose issues. When you deploy an Express app to Heroku, adding Morgan helps you monitor traffic patterns, identify slow endpoints, and troubleshoot HTTP errors through their logging system.

cors - Cross-Origin Resource Sharing

The CORS middleware enables Cross-Origin Resource Sharing, allowing your Express APIs to be accessed from different domains.

const express = require('express');
const cors = require('cors');
const app = express();

// Enable CORS for all routes
app.use(cors());

// Configure CORS with options
app.use(cors({
  // Allowed origins (can be an array, string, or function)
  origin: 'https://example.com',
  
  // Allowed HTTP methods
  methods: ['GET', 'POST', 'PUT', 'DELETE'],
  
  // Allowed request headers
  allowedHeaders: ['Content-Type', 'Authorization'],
  
  // Headers exposed to the client
  exposedHeaders: ['X-Custom-Header'],
  
  // Allow credentials (cookies, authorization headers)
  credentials: true,
  
  // How long the results of a preflight request can be cached
  maxAge: 86400, // 24 hours
}));

// Enable CORS for specific routes only
app.get('/api/public', cors(), (req, res) => {
  res.json({ message: 'This is public API' });
});

// Dynamic origin based on environment
app.use(cors({
  origin: function(origin, callback) {
    const allowedOrigins = ['https://example.com', 'https://dev.example.com'];
    if (!origin || allowedOrigins.includes(origin)) {
      callback(null, true);
    } else {
      callback(new Error('Not allowed by CORS'));
    }
  }
}));
            

Analogy: CORS is like the security policy at a corporate building. By default, visitors (requests) from other companies (domains) aren't allowed in. The CORS middleware is the security guard who checks visitor IDs and compares them against an approved list, allowing only authorized visitors from specific companies to enter certain areas of the building.

helmet - Security Headers

Helmet helps secure Express apps by setting various HTTP headers to protect against common web vulnerabilities.

const express = require('express');
const helmet = require('helmet');
const app = express();

// Use helmet with default settings
app.use(helmet());

// This is equivalent to using all these middleware:
app.use(helmet.contentSecurityPolicy()); // Prevents XSS attacks
app.use(helmet.crossOriginEmbedderPolicy());
app.use(helmet.crossOriginOpenerPolicy());
app.use(helmet.crossOriginResourcePolicy());
app.use(helmet.dnsPrefetchControl());
app.use(helmet.expectCt());
app.use(helmet.frameguard()); // Prevents clickjacking
app.use(helmet.hidePoweredBy()); // Hides X-Powered-By header
app.use(helmet.hsts()); // HTTP Strict Transport Security
app.use(helmet.ieNoOpen());
app.use(helmet.noSniff()); // Prevents MIME sniffing
app.use(helmet.permittedCrossDomainPolicies());
app.use(helmet.referrerPolicy());
app.use(helmet.xssFilter()); // Provides basic XSS protection

// Custom configuration
app.use(
  helmet({
    contentSecurityPolicy: {
      directives: {
        defaultSrc: ["'self'"],
        scriptSrc: ["'self'", "trusted-cdn.com"],
      },
    },
    hsts: {
      maxAge: 31536000, // 1 year
      includeSubDomains: true,
      preload: true
    }
  })
);
            

Real-world example: Financial institutions like banks use Helmet to enhance the security of their web applications. It helps protect sensitive customer data by preventing various attacks such as cross-site scripting (XSS), clickjacking, and other injection-based vulnerabilities that could compromise user accounts.

cookie-parser - Parse Cookie Header

Cookie-parser parses the Cookie header and populates req.cookies with an object containing the cookies.

const express = require('express');
const cookieParser = require('cookie-parser');
const app = express();

// Basic usage without signing
app.use(cookieParser());

// With a secret for signed cookies
app.use(cookieParser('my_secret_key'));

// Setting cookies
app.get('/set-cookie', (req, res) => {
  // Set a regular cookie
  res.cookie('user', 'john', { maxAge: 900000, httpOnly: true });
  
  // Set a signed cookie
  res.cookie('role', 'admin', { signed: true });
  
  res.send('Cookies set');
});

// Reading cookies
app.get('/get-cookies', (req, res) => {
  // Access regular cookies
  console.log('Cookies:', req.cookies);
  
  // Access signed cookies (tamper-proof)
  console.log('Signed Cookies:', req.signedCookies);
  
  res.json({
    cookies: req.cookies,
    signedCookies: req.signedCookies
  });
});

// Clearing cookies
app.get('/clear-cookie', (req, res) => {
  res.clearCookie('user');
  res.send('Cookie cleared');
});
            

Analogy: Cookie-parser is like a hotel receptionist who manages your room key. When you arrive (make a request), the receptionist checks if you have a key card (cookie), validates it's for the right room (parses it), and either lets you in or issues a new key (sets a new cookie) when needed.

express-session - Session Management

Express-session creates a session middleware that stores session data on the server and uses cookies to identify the session.

const express = require('express');
const session = require('express-session');
const app = express();

// Basic session setup
app.use(session({
  // Secret used to sign the session ID cookie
  secret: 'keyboard cat',
  
  // Forces the session to be saved back to the session store
  resave: false,
  
  // Forces a session that is "uninitialized" to be saved to the store
  saveUninitialized: true,
  
  // Cookie settings
  cookie: { 
    secure: process.env.NODE_ENV === 'production', // HTTPS only in production
    httpOnly: true, // Prevents client-side JS from reading the cookie
    maxAge: 1000 * 60 * 60 * 24 // 24 hours
  }
}));

// Using session to store user data
app.post('/login', (req, res) => {
  // Authenticate user (simplified example)
  const { username, password } = req.body;
  
  if (username === 'admin' && password === 'secret') {
    // Store user data in session
    req.session.user = {
      id: 1,
      username: 'admin',
      role: 'administrator'
    };
    res.redirect('/dashboard');
  } else {
    res.redirect('/login?error=true');
  }
});

// Access session data
app.get('/dashboard', (req, res) => {
  // Check if user is logged in
  if (!req.session.user) {
    return res.redirect('/login');
  }
  
  res.send(`Welcome ${req.session.user.username}!`);
});

// Logout by destroying the session
app.get('/logout', (req, res) => {
  req.session.destroy(err => {
    if (err) {
      return res.redirect('/dashboard');
    }
    res.clearCookie('connect.sid');
    res.redirect('/login');
  });
});
            

Using Session Stores

By default, Express-session uses an in-memory store (MemoryStore), which is not suitable for production. For production, you should use a dedicated session store:

// Redis session store
const session = require('express-session');
const RedisStore = require('connect-redis').default;
const { createClient } = require('redis');

// Create Redis client
const redisClient = createClient({
  url: process.env.REDIS_URL || 'redis://localhost:6379'
});

redisClient.connect().catch(console.error);

// Configure session with Redis store
app.use(session({
  store: new RedisStore({ client: redisClient }),
  secret: 'keyboard cat',
  resave: false,
  saveUninitialized: false,
  cookie: { secure: process.env.NODE_ENV === 'production' }
}));

// Other session stores include:
// - connect-mongodb-session: MongoDB store
// - express-mysql-session: MySQL store
// - connect-pg-simple: PostgreSQL store
            

Real-world example: E-commerce platforms like Shopify use session management to maintain shopping carts. When you add items to your cart while browsing, they're stored in your session. Even if you navigate away or refresh the page, your cart items persist because they're linked to your session ID stored in a cookie.

Other Popular Middleware

compression - Response Compression

Compression middleware compresses response bodies for all requests that traverse through the middleware.

const express = require('express');
const compression = require('compression');
const app = express();

// Use compression
app.use(compression());

// With options
app.use(compression({
  // Compression level (0-9)
  level: 6,
  
  // Minimum response size in bytes to compress
  threshold: 1024,
  
  // Don't compress responses with this Content-Type
  filter: (req, res) => {
    return /text|json|javascript|css/.test(res.getHeader('Content-Type'));
  }
}));
            

multer - File Upload Handling

Multer is a middleware for handling multipart/form-data, primarily used for file uploads.

const express = require('express');
const multer = require('multer');
const app = express();

// Basic setup with default disk storage
const upload = multer({ dest: 'uploads/' });

// Single file upload
app.post('/upload', upload.single('avatar'), (req, res) => {
  // req.file contains the uploaded file info
  console.log(req.file);
  
  // req.body contains other text fields
  console.log(req.body);
  
  res.send('File uploaded');
});

// Multiple files upload
app.post('/upload-gallery', upload.array('photos', 12), (req, res) => {
  // req.files contains the uploaded files info
  console.log(req.files);
  
  res.send(`${req.files.length} files uploaded`);
});

// Advanced configuration with storage
const storage = multer.diskStorage({
  destination: function (req, file, cb) {
    cb(null, 'uploads/');
  },
  filename: function (req, file, cb) {
    const uniqueSuffix = Date.now() + '-' + Math.round(Math.random() * 1E9);
    cb(null, file.fieldname + '-' + uniqueSuffix + '.jpg');
  }
});

const upload2 = multer({ 
  storage: storage,
  limits: {
    fileSize: 1024 * 1024 * 5, // 5MB
    files: 5
  },
  fileFilter: function (req, file, cb) {
    // Accept only image files
    if (file.mimetype.startsWith('image/')) {
      cb(null, true);
    } else {
      cb(new Error('Only image files are allowed'));
    }
  }
});
            

passport - Authentication

Passport is an authentication middleware that provides a comprehensive, modular authentication system.

const express = require('express');
const passport = require('passport');
const LocalStrategy = require('passport-local').Strategy;
const app = express();

// Initialize passport
app.use(express.session({ secret: 'keyboard cat' }));
app.use(passport.initialize());
app.use(passport.session());

// Configure local strategy
passport.use(new LocalStrategy(
  function(username, password, done) {
    // Database lookup logic (simplified example)
    User.findOne({ username: username }, function (err, user) {
      if (err) { return done(err); }
      if (!user) { return done(null, false, { message: 'Incorrect username.' }); }
      if (!user.validatePassword(password)) {
        return done(null, false, { message: 'Incorrect password.' });
      }
      return done(null, user);
    });
  }
));

// Serialize/deserialize user
passport.serializeUser(function(user, done) {
  done(null, user.id);
});

passport.deserializeUser(function(id, done) {
  User.findById(id, function(err, user) {
    done(err, user);
  });
});

// Login route
app.post('/login',
  passport.authenticate('local', {
    successRedirect: '/dashboard',
    failureRedirect: '/login',
    failureFlash: true
  })
);

// Protected route
app.get('/dashboard',
  ensureAuthenticated,
  function(req, res) {
    res.render('dashboard', { user: req.user });
  }
);

// Middleware to check if the user is authenticated
function ensureAuthenticated(req, res, next) {
  if (req.isAuthenticated()) {
    return next();
  }
  res.redirect('/login');
}
            

Real-world example: Social media platforms like LinkedIn use Passport.js to implement "Sign in with Google/Facebook/Twitter" functionality. It provides a consistent authentication interface while supporting multiple providers, allowing users to log in with their preferred accounts rather than creating a new one.

Middleware Selection and Composition

When building Express applications, it's important to select and combine middleware strategically to create a cohesive request processing pipeline.

Common Middleware Stack

Here's an example of a typical middleware stack for a modern Express application:

const express = require('express');
const morgan = require('morgan');
const helmet = require('helmet');
const compression = require('compression');
const cors = require('cors');
const cookieParser = require('cookie-parser');
const session = require('express-session');
const app = express();

// Request logging
app.use(morgan('dev'));

// Security headers
app.use(helmet());

// Enable CORS
app.use(cors());

// Parse requests
app.use(express.json());
app.use(express.urlencoded({ extended: true }));

// Cookie handling
app.use(cookieParser());

// Session management
app.use(session({
  secret: 'secret-key',
  resave: false,
  saveUninitialized: false,
  cookie: { secure: process.env.NODE_ENV === 'production' }
}));

// Response compression
app.use(compression());

// Static file serving
app.use(express.static('public'));

// Application routes
app.use('/api', apiRoutes);
app.use('/auth', authRoutes);
app.use('/', webRoutes);

// Error handling (should be last)
app.use((err, req, res, next) => {
  console.error(err.stack);
  res.status(500).send('Something broke!');
});
            

Analogy: Selecting middleware is like planning a security checkpoint at an airport. You need to decide which inspections (middleware) to include and in what order. Do you check IDs (authentication) before or after scanning baggage (parsing the request body)? The order matters and affects both security and efficiency.

Practice Activities

Key Takeaways

• Express provides essential built-in middleware for parsing requests and serving static files
• The most common built-in middleware includes express.json(), express.urlencoded(), and express.static()
• Third-party middleware extends Express with specialized functionality for security, logging, file uploads, and more
• Popular third-party middleware includes morgan, helmet, cors, and express-session
• The order of middleware in your application is crucial for proper functionality
• A well-designed middleware stack creates a robust processing pipeline for HTTP requests
• Selecting appropriate middleware helps balance functionality, performance, and security

In our next lecture, we'll explore creating custom middleware to add specialized functionality to your Express applications.