Building a Feature-Rich Express.js API with Proper Architecture and Error Handling

Step 1: Understand the Problem - Defining Our API Requirements

Before writing any code, we need to thoroughly understand what we're building and why. Let's define the requirements for our API project.

Project Description: Bookshelf API

We'll create a RESTful API for managing a virtual bookshelf system. Users can create accounts, manage their collection of books, track reading progress, write reviews, and discover new books through recommendations.

Core Requirements

• Authentication and Authorization: Secure user registration, login, and role-based access control
• Resource Management: CRUD operations for books, collections, reviews, and user profiles
• Error Handling: Comprehensive system for handling and reporting errors
• Data Validation: Thorough validation of all input data
• Documentation: Clear API documentation with examples

Technical Requirements

• Framework: Express.js
• Database: MongoDB with Mongoose ODM
• Authentication: JWT-based authentication
• Architecture: Well-structured MVC pattern with service layer
• Error Handling: Centralized error handling middleware
• Validation: Using express-validator or Joi
• Testing: Unit and integration tests with Jest and Supertest
• Documentation: Swagger/OpenAPI specification

Key API Endpoints

Step 2: Devise a Plan - Designing the Architecture

With a clear understanding of what we need to build, let's design a clean, maintainable architecture for our Express.js API.

Project Structure

bookshelf-api/
├── src/
│   ├── config/            # Configuration files
│   │   ├── db.js          # Database configuration
│   │   ├── env.js         # Environment variables
│   │   └── logger.js      # Logging configuration
│   ├── controllers/       # Route controllers
│   │   ├── authController.js
│   │   ├── bookController.js
│   │   ├── collectionController.js
│   │   └── reviewController.js
│   ├── errors/            # Error handling
│   │   ├── AppError.js    # Base error class
│   │   ├── errorTypes.js  # Error type definitions
│   │   ├── errorHandler.js # Error middleware
│   │   └── asyncHandler.js # Async error handler
│   ├── middleware/        # Middleware functions
│   │   ├── auth.js        # Authentication middleware
│   │   ├── validate.js    # Validation middleware
│   │   └── rateLimiter.js # Rate limiting middleware
│   ├── models/            # Database models
│   │   ├── User.js
│   │   ├── Book.js
│   │   ├── Collection.js
│   │   └── Review.js
│   ├── routes/            # API routes
│   │   ├── authRoutes.js
│   │   ├── bookRoutes.js
│   │   ├── collectionRoutes.js
│   │   └── reviewRoutes.js
│   ├── services/          # Business logic
│   │   ├── authService.js
│   │   ├── bookService.js
│   │   ├── collectionService.js
│   │   └── reviewService.js
│   ├── utils/             # Utility functions
│   │   ├── validation.js  # Validation schemas
│   │   ├── jwt.js         # JWT utilities
│   │   └── helpers.js     # Helper functions
│   ├── docs/              # API documentation
│   │   └── swagger.json   # Swagger/OpenAPI spec
│   └── app.js             # Express application setup
├── tests/                 # Test files
│   ├── unit/              # Unit tests
│   ├── integration/       # Integration tests
│   └── fixtures/          # Test fixtures
├── .env                   # Environment variables
├── .gitignore             # Git ignore file
├── package.json           # Project dependencies
├── jest.config.js         # Jest configuration
└── README.md              # Project documentation

Architectural Pattern

We'll implement a modified MVC architecture with a service layer:

Key Architectural Components

• Routes: Define API endpoints and map them to controllers
• Controllers: Handle HTTP requests/responses, but delegate business logic to services
• Services: Contain business logic and interact with models/external services
• Models: Define database schema and data access methods
• Middleware: Cross-cutting concerns like authentication, validation, error handling
• Error Handling: Centralized system for handling all types of errors

Error Handling Strategy

We'll implement a comprehensive error handling system with:

• Custom Error Classes: Hierarchy of error types for different scenarios
• Async Error Handling: Wrapper for handling async errors consistently
• Central Error Middleware: Process all errors in one place
• Environment-Specific Responses: Different formats for development vs. production
• Error Logging: Structured logging for troubleshooting

Development Approach

We'll take an iterative approach, focusing on one feature at a time:

1. Set up project structure and core architecture
2. Implement authentication system
3. Build book management features
4. Add collections and reviews functionality
5. Implement advanced features (search, recommendations)
6. Add comprehensive testing
7. Create documentation

Step 3: Execute the Plan - Implementation

Now let's implement our API design, focusing on the core components and error handling.

Core Setup

First, let's set up the Express application and essential middleware:

// src/app.js
const express = require('express');
const morgan = require('morgan');
const helmet = require('helmet');
const cors = require('cors');
const { connectDB } = require('./config/db');
const errorHandler = require('./errors/errorHandler');
const { notFoundHandler } = require('./errors/errorHandler');

// Import routes
const authRoutes = require('./routes/authRoutes');
const bookRoutes = require('./routes/bookRoutes');
const collectionRoutes = require('./routes/collectionRoutes');
const reviewRoutes = require('./routes/reviewRoutes');

// Initialize express app
const app = express();

// Connect to database
connectDB();

// Middleware
app.use(express.json());
app.use(express.urlencoded({ extended: false }));
app.use(morgan('dev'));
app.use(helmet());
app.use(cors());

// Routes
app.use('/api/auth', authRoutes);
app.use('/api/books', bookRoutes);
app.use('/api/collections', collectionRoutes);
app.use('/api/reviews', reviewRoutes);

// Swagger documentation
app.use('/api-docs', swaggerUI.serve, swaggerUI.setup(swaggerDocs));

// 404 handler
app.use(notFoundHandler);

// Error handling middleware
app.use(errorHandler);

module.exports = app;

Error Handling Implementation

Let's implement our error handling system:

// src/errors/AppError.js
class AppError extends Error {
  constructor(message, statusCode, errorCode, isOperational = true) {
    super(message);
    this.statusCode = statusCode;
    this.errorCode = errorCode || this.constructor.name;
    this.isOperational = isOperational;
    
    Error.captureStackTrace(this, this.constructor);
  }
}

module.exports = AppError;

// src/errors/errorTypes.js
const AppError = require('./AppError');

class NotFoundError extends AppError {
  constructor(resource = 'Resource', id = '') {
    const message = id 
      ? `${resource} with ID ${id} not found`
      : `${resource} not found`;
    super(message, 404, 'NOT_FOUND');
  }
}

class ValidationError extends AppError {
  constructor(message = 'Validation failed', errors = null) {
    super(message, 400, 'VALIDATION_ERROR');
    this.errors = errors;
  }
}

class AuthenticationError extends AppError {
  constructor(message = 'Authentication failed') {
    super(message, 401, 'AUTHENTICATION_ERROR');
  }
}

class AuthorizationError extends AppError {
  constructor(message = 'Not authorized to access this resource') {
    super(message, 403, 'AUTHORIZATION_ERROR');
  }
}

class ConflictError extends AppError {
  constructor(message = 'Resource already exists') {
    super(message, 409, 'CONFLICT_ERROR');
  }
}

class DatabaseError extends AppError {
  constructor(message = 'Database error', originalError = null) {
    super(message, 500, 'DATABASE_ERROR', false);
    this.originalError = originalError;
  }
}

module.exports = {
  NotFoundError,
  ValidationError,
  AuthenticationError,
  AuthorizationError,
  ConflictError,
  DatabaseError
};

// src/errors/asyncHandler.js
const asyncHandler = fn => (req, res, next) =>
  Promise.resolve(fn(req, res, next)).catch(next);

module.exports = asyncHandler;

// src/errors/errorHandler.js
const { isProduction } = require('../config/env');
const logger = require('../config/logger');
const AppError = require('./AppError');

// 404 error handler - convert to our custom format
const notFoundHandler = (req, res, next) => {
  const error = new AppError(
    `Resource not found - ${req.originalUrl}`,
    404,
    'RESOURCE_NOT_FOUND'
  );
  next(error);
};

// Central error handling middleware
const errorHandler = (err, req, res, next) => {
  // Clone error to avoid modifying the original
  let error = { ...err };
  error.message = err.message;
  error.stack = err.stack;
  
  // Log error
  logger.error({
    message: `${err.statusCode || 500} - ${err.message}`,
    stack: err.stack,
    method: req.method,
    path: req.path,
    ip: req.ip,
    requestId: req.id,
    timestamp: new Date().toISOString()
  });
  
  // Format Mongoose errors
  if (err.name === 'CastError') {
    error = new AppError(`Invalid ${err.path}: ${err.value}`, 400, 'INVALID_ID');
  }
  
  if (err.name === 'ValidationError') {
    const errors = Object.values(err.errors).map(val => val.message);
    error = new AppError(`Validation failed: ${errors.join(', ')}`, 400, 'VALIDATION_ERROR');
  }
  
  if (err.code === 11000) {
    const field = Object.keys(err.keyValue)[0];
    error = new AppError(`Duplicate field value: ${field}`, 409, 'DUPLICATE_ERROR');
  }
  
  // JWT errors
  if (err.name === 'JsonWebTokenError') {
    error = new AppError('Invalid token', 401, 'INVALID_TOKEN');
  }
  
  if (err.name === 'TokenExpiredError') {
    error = new AppError('Token expired', 401, 'EXPIRED_TOKEN');
  }
  
  // Prepare response
  const statusCode = error.statusCode || err.statusCode || 500;
  const isOperational = error.isOperational || err.isOperational || false;

  // Response for development vs production
  const response = {
    success: false,
    error: {
      message: error.message || 'Internal server error',
      code: error.errorCode || err.errorCode || 'SERVER_ERROR',
      statusCode
    }
  };
  
  // Add stack trace in development for debugging
  if (!isProduction) {
    response.error.stack = error.stack || err.stack;
  }
  
  // Include error details if available (e.g., validation errors)
  if (error.errors || err.errors) {
    response.error.details = error.errors || err.errors;
  }
  
  // Don't expose internal error details in production for non-operational errors
  if (isProduction && !isOperational) {
    response.error.message = 'Something went wrong';
    response.error.code = 'SERVER_ERROR';
    delete response.error.details;
  }
  
  res.status(statusCode).json(response);
};

module.exports = {
  notFoundHandler,
  errorHandler
};

Authentication Implementation

Let's implement user authentication with JWT:

// src/models/User.js
const mongoose = require('mongoose');
const bcrypt = require('bcryptjs');
const jwt = require('jsonwebtoken');
const { jwtSecret, jwtExpiresIn } = require('../config/env');

const userSchema = new mongoose.Schema({
  name: {
    type: String,
    required: [true, 'Name is required'],
    trim: true
  },
  email: {
    type: String,
    required: [true, 'Email is required'],
    unique: true,
    trim: true,
    lowercase: true,
    match: [/^[\w-\.]+@([\w-]+\.)+[\w-]{2,4}$/, 'Please provide a valid email']
  },
  password: {
    type: String,
    required: [true, 'Password is required'],
    minlength: [8, 'Password must be at least 8 characters'],
    select: false
  },
  role: {
    type: String,
    enum: ['user', 'admin'],
    default: 'user'
  },
  createdAt: {
    type: Date,
    default: Date.now
  }
});

// Hash password before saving
userSchema.pre('save', async function(next) {
  if (!this.isModified('password')) return next();
  
  try {
    const salt = await bcrypt.genSalt(10);
    this.password = await bcrypt.hash(this.password, salt);
    next();
  } catch (error) {
    next(error);
  }
});

// Sign JWT token
userSchema.methods.getSignedJwtToken = function() {
  return jwt.sign(
    { id: this._id, role: this.role },
    jwtSecret,
    { expiresIn: jwtExpiresIn }
  );
};

// Compare password
userSchema.methods.matchPassword = async function(enteredPassword) {
  return await bcrypt.compare(enteredPassword, this.password);
};

module.exports = mongoose.model('User', userSchema);

// src/middleware/auth.js
const jwt = require('jsonwebtoken');
const { jwtSecret } = require('../config/env');
const asyncHandler = require('../errors/asyncHandler');
const { AuthenticationError, AuthorizationError } = require('../errors/errorTypes');
const User = require('../models/User');

// Protect routes - JWT verification
exports.protect = asyncHandler(async (req, res, next) => {
  let token;
  
  // Get token from Authorization header
  if (req.headers.authorization && req.headers.authorization.startsWith('Bearer')) {
    token = req.headers.authorization.split(' ')[1];
  }
  
  // Check if token exists
  if (!token) {
    throw new AuthenticationError('Not authorized to access this route');
  }
  
  try {
    // Verify token
    const decoded = jwt.verify(token, jwtSecret);
    
    // Add user to request
    req.user = await User.findById(decoded.id);
    
    if (!req.user) {
      throw new AuthenticationError('User not found');
    }
    
    next();
  } catch (error) {
    throw new AuthenticationError('Not authorized to access this route');
  }
});

// Authorize by role
exports.authorize = (...roles) => {
  return (req, res, next) => {
    if (!roles.includes(req.user.role)) {
      throw new AuthorizationError(
        `User role ${req.user.role} is not authorized to access this route`
      );
    }
    next();
  };
};

// src/controllers/authController.js
const asyncHandler = require('../errors/asyncHandler');
const { ValidationError, AuthenticationError } = require('../errors/errorTypes');
const User = require('../models/User');
const authService = require('../services/authService');

// @desc    Register user
// @route   POST /api/auth/register
// @access  Public
exports.register = asyncHandler(async (req, res) => {
  const { name, email, password } = req.body;
  
  // Create user
  const user = await authService.registerUser(name, email, password);
  
  // Generate token and send response
  sendTokenResponse(user, 201, res);
});

// @desc    Login user
// @route   POST /api/auth/login
// @access  Public
exports.login = asyncHandler(async (req, res) => {
  const { email, password } = req.body;
  
  // Validate email & password
  if (!email || !password) {
    throw new ValidationError('Please provide email and password');
  }
  
  // Check for user
  const user = await User.findOne({ email }).select('+password');
  
  if (!user) {
    throw new AuthenticationError('Invalid credentials');
  }
  
  // Check if password matches
  const isMatch = await user.matchPassword(password);
  
  if (!isMatch) {
    throw new AuthenticationError('Invalid credentials');
  }
  
  // Generate token and send response
  sendTokenResponse(user, 200, res);
});

// @desc    Get current logged in user
// @route   GET /api/auth/me
// @access  Private
exports.getMe = asyncHandler(async (req, res) => {
  const user = await User.findById(req.user.id);
  
  res.status(200).json({
    success: true,
    data: user
  });
});

// Helper function to send token response
const sendTokenResponse = (user, statusCode, res) => {
  // Create token
  const token = user.getSignedJwtToken();
  
  res.status(statusCode).json({
    success: true,
    token
  });
};

// src/services/authService.js
const User = require('../models/User');
const { ConflictError } = require('../errors/errorTypes');

exports.registerUser = async (name, email, password) => {
  // Check if user already exists
  const existingUser = await User.findOne({ email });
  
  if (existingUser) {
    throw new ConflictError('User with that email already exists');
  }
  
  // Create new user
  return await User.create({
    name,
    email,
    password
  });
};

Validation Middleware

Implementing request validation using express-validator:

// src/middleware/validate.js
const { validationResult } = require('express-validator');
const { ValidationError } = require('../errors/errorTypes');

// Validation middleware
const validate = validations => {
  return async (req, res, next) => {
    // Run all validations
    await Promise.all(validations.map(validation => validation.run(req)));
    
    // Check for validation errors
    const errors = validationResult(req);
    
    if (!errors.isEmpty()) {
      // Format errors for easy consumption
      const formattedErrors = {};
      
      errors.array().forEach(error => {
        formattedErrors[error.param] = error.msg;
      });
      
      throw new ValidationError('Validation failed', formattedErrors);
    }
    
    next();
  };
};

module.exports = validate;

// src/utils/validation.js
const { body } = require('express-validator');

// User validation schemas
exports.registerValidator = [
  body('name')
    .trim()
    .notEmpty().withMessage('Name is required')
    .isLength({ min: 2, max: 50 }).withMessage('Name must be between 2 and 50 characters'),
    
  body('email')
    .trim()
    .notEmpty().withMessage('Email is required')
    .isEmail().withMessage('Please provide a valid email')
    .normalizeEmail(),
    
  body('password')
    .trim()
    .notEmpty().withMessage('Password is required')
    .isLength({ min: 8 }).withMessage('Password must be at least 8 characters')
    .matches(/\d/).withMessage('Password must contain a number')
    .matches(/[A-Z]/).withMessage('Password must contain an uppercase letter')
];

exports.loginValidator = [
  body('email')
    .trim()
    .notEmpty().withMessage('Email is required')
    .isEmail().withMessage('Please provide a valid email'),
    
  body('password')
    .trim()
    .notEmpty().withMessage('Password is required')
];

// Book validation schemas
exports.createBookValidator = [
  body('title')
    .trim()
    .notEmpty().withMessage('Title is required')
    .isLength({ min: 1, max: 200 }).withMessage('Title must be between 1 and 200 characters'),
    
  body('author')
    .trim()
    .notEmpty().withMessage('Author is required')
    .isLength({ min: 1, max: 100 }).withMessage('Author must be between 1 and 100 characters'),
    
  body('genre')
    .optional()
    .trim()
    .isLength({ max: 50 }).withMessage('Genre must be less than 50 characters'),
    
  body('publishedYear')
    .optional()
    .isInt({ min: 1000, max: new Date().getFullYear() })
    .withMessage(`Year must be between 1000 and ${new Date().getFullYear()}`)
];

Route Implementation

Implementing a set of routes for one of our resources:

// src/routes/bookRoutes.js
const express = require('express');
const router = express.Router();
const bookController = require('../controllers/bookController');
const { protect, authorize } = require('../middleware/auth');
const validate = require('../middleware/validate');
const { createBookValidator, updateBookValidator } = require('../utils/validation');

// Public routes
router
  .route('/public')
  .get(bookController.getPublicBooks);

// Protected routes
router.use(protect); // Apply authentication middleware to all routes below

router
  .route('/')
  .get(bookController.getBooks)
  .post(validate(createBookValidator), bookController.createBook);

router
  .route('/:id')
  .get(bookController.getBook)
  .put(validate(updateBookValidator), bookController.updateBook)
  .delete(bookController.deleteBook);

// Admin only routes
router
  .route('/admin/reports')
  .get(authorize('admin'), bookController.getAdminReports);

module.exports = router;

// src/controllers/bookController.js
const asyncHandler = require('../errors/asyncHandler');
const { NotFoundError } = require('../errors/errorTypes');
const Book = require('../models/Book');
const bookService = require('../services/bookService');

// @desc    Get all books
// @route   GET /api/books
// @access  Private
exports.getBooks = asyncHandler(async (req, res) => {
  // Query parameters for filtering, sorting, pagination
  const { page = 1, limit = 10, sort, genre, search } = req.query;
  
  // Get books from service layer
  const result = await bookService.getBooks({
    user: req.user.id,
    page: parseInt(page, 10),
    limit: parseInt(limit, 10),
    sort,
    genre,
    search
  });
  
  res.status(200).json({
    success: true,
    count: result.books.length,
    pagination: {
      page: result.page,
      limit: result.limit,
      totalPages: result.totalPages,
      totalCount: result.totalCount
    },
    data: result.books
  });
});

// @desc    Get single book
// @route   GET /api/books/:id
// @access  Private
exports.getBook = asyncHandler(async (req, res) => {
  const book = await bookService.getBookById(req.params.id, req.user.id);
  
  res.status(200).json({
    success: true,
    data: book
  });
});

// @desc    Create new book
// @route   POST /api/books
// @access  Private
exports.createBook = asyncHandler(async (req, res) => {
  // Add user to request body
  req.body.user = req.user.id;
  
  const book = await bookService.createBook(req.body);
  
  res.status(201).json({
    success: true,
    data: book
  });
});

// @desc    Update book
// @route   PUT /api/books/:id
// @access  Private
exports.updateBook = asyncHandler(async (req, res) => {
  const book = await bookService.updateBook(req.params.id, req.body, req.user.id);
  
  res.status(200).json({
    success: true,
    data: book
  });
});

// @desc    Delete book
// @route   DELETE /api/books/:id
// @access  Private
exports.deleteBook = asyncHandler(async (req, res) => {
  await bookService.deleteBook(req.params.id, req.user.id);
  
  res.status(200).json({
    success: true,
    data: {}
  });
});

// src/services/bookService.js
const Book = require('../models/Book');
const { NotFoundError, AuthorizationError } = require('../errors/errorTypes');

exports.getBooks = async ({ user, page = 1, limit = 10, sort, genre, search }) => {
  // Build query
  const query = { user };
  
  // Add genre filter if provided
  if (genre) {
    query.genre = genre;
  }
  
  // Add search functionality
  if (search) {
    query.$or = [
      { title: { $regex: search, $options: 'i' } },
      { author: { $regex: search, $options: 'i' } }
    ];
  }
  
  // Calculate pagination
  const skip = (page - 1) * limit;
  
  // Build sort object
  let sortObj = {};
  if (sort) {
    // Handle sort format like 'field:asc' or 'field:desc'
    const [field, direction] = sort.split(':');
    sortObj[field] = direction === 'desc' ? -1 : 1;
  } else {
    // Default sort by creation date
    sortObj = { createdAt: -1 };
  }
  
  // Execute query with pagination
  const books = await Book.find(query)
    .sort(sortObj)
    .skip(skip)
    .limit(limit);
  
  // Get total count for pagination
  const totalCount = await Book.countDocuments(query);
  
  return {
    books,
    page,
    limit,
    totalPages: Math.ceil(totalCount / limit),
    totalCount
  };
};

exports.getBookById = async (id, userId) => {
  const book = await Book.findById(id);
  
  if (!book) {
    throw new NotFoundError('Book', id);
  }
  
  // Check if the book belongs to the user
  if (book.user.toString() !== userId) {
    throw new AuthorizationError('Not authorized to access this book');
  }
  
  return book;
};

exports.createBook = async (bookData) => {
  return await Book.create(bookData);
};

exports.updateBook = async (id, bookData, userId) => {
  let book = await Book.findById(id);
  
  if (!book) {
    throw new NotFoundError('Book', id);
  }
  
  // Check if the book belongs to the user
  if (book.user.toString() !== userId) {
    throw new AuthorizationError('Not authorized to update this book');
  }
  
  // Update book
  book = await Book.findByIdAndUpdate(id, bookData, {
    new: true,
    runValidators: true
  });
  
  return book;
};

exports.deleteBook = async (id, userId) => {
  const book = await Book.findById(id);
  
  if (!book) {
    throw new NotFoundError('Book', id);
  }
  
  // Check if the book belongs to the user
  if (book.user.toString() !== userId) {
    throw new AuthorizationError('Not authorized to delete this book');
  }
  
  await book.remove();
  
  return true;
};

Step 4: Review/Reflect - Testing and Improvement

Now that we've implemented our API, let's test it, evaluate the solution, and look for areas of improvement.

Testing Strategy

We'll use a combination of unit, integration, and end-to-end tests:

// tests/unit/services/authService.test.js
const mongoose = require('mongoose');
const { MongoMemoryServer } = require('mongodb-memory-server');
const authService = require('../../../src/services/authService');
const User = require('../../../src/models/User');
const { ConflictError } = require('../../../src/errors/errorTypes');

// Test data
const testUser = {
  name: 'Test User',
  email: 'test@example.com',
  password: 'Password123'
};

describe('Auth Service', () => {
  let mongoServer;
  
  // Set up in-memory MongoDB server
  beforeAll(async () => {
    mongoServer = await MongoMemoryServer.create();
    await mongoose.connect(mongoServer.getUri());
  });
  
  // Clean up after tests
  afterAll(async () => {
    await mongoose.disconnect();
    await mongoServer.stop();
  });
  
  // Clear database between tests
  beforeEach(async () => {
    await User.deleteMany({});
  });
  
  describe('registerUser', () => {
    it('should register a new user successfully', async () => {
      const user = await authService.registerUser(
        testUser.name,
        testUser.email,
        testUser.password
      );
      
      expect(user).toBeDefined();
      expect(user.name).toBe(testUser.name);
      expect(user.email).toBe(testUser.email);
      expect(user.password).not.toBe(testUser.password); // Should be hashed
    });
    
    it('should throw ConflictError if email already exists', async () => {
      // Create a user first
      await User.create(testUser);
      
      // Try to register with same email
      await expect(
        authService.registerUser(
          'Another Name',
          testUser.email,
          'DifferentPass123'
        )
      ).rejects.toThrow(ConflictError);
    });
  });
});

// tests/integration/books.test.js
const request = require('supertest');
const mongoose = require('mongoose');
const { MongoMemoryServer } = require('mongodb-memory-server');
const app = require('../../src/app');
const User = require('../../src/models/User');
const Book = require('../../src/models/Book');

describe('Book Routes', () => {
  let mongoServer;
  let token;
  let userId;
  let bookId;
  
  // Test data
  const testUser = {
    name: 'Test User',
    email: 'test@example.com',
    password: 'Password123'
  };
  
  const testBook = {
    title: 'Test Book',
    author: 'Test Author',
    genre: 'Fiction',
    publishedYear: 2020
  };
  
  // Set up in-memory MongoDB server
  beforeAll(async () => {
    mongoServer = await MongoMemoryServer.create();
    await mongoose.connect(mongoServer.getUri());
    
    // Create test user and get token
    const userResponse = await request(app)
      .post('/api/auth/register')
      .send(testUser);
    
    token = userResponse.body.token;
    
    // Extract user ID from token
    const decoded = jwt.verify(token, process.env.JWT_SECRET);
    userId = decoded.id;
  });
  
  // Clean up after tests
  afterAll(async () => {
    await mongoose.disconnect();
    await mongoServer.stop();
  });
  
  // Clear books collection between tests
  beforeEach(async () => {
    await Book.deleteMany({});
  });
  
  describe('POST /api/books', () => {
    it('should create a new book', async () => {
      const res = await request(app)
        .post('/api/books')
        .set('Authorization', `Bearer ${token}`)
        .send(testBook);
      
      expect(res.statusCode).toEqual(201);
      expect(res.body.success).toBeTruthy();
      expect(res.body.data).toHaveProperty('_id');
      expect(res.body.data.title).toBe(testBook.title);
      expect(res.body.data.user).toBe(userId);
      
      // Save book ID for later tests
      bookId = res.body.data._id;
    });
    
    it('should return validation error if title is missing', async () => {
      const res = await request(app)
        .post('/api/books')
        .set('Authorization', `Bearer ${token}`)
        .send({
          author: testBook.author,
          genre: testBook.genre
        });
      
      expect(res.statusCode).toEqual(400);
      expect(res.body.success).toBeFalsy();
      expect(res.body.error.code).toBe('VALIDATION_ERROR');
      expect(res.body.error.details).toHaveProperty('title');
    });
  });
  
  describe('GET /api/books', () => {
    it('should get all books for the user', async () => {
      // Create a test book first
      await request(app)
        .post('/api/books')
        .set('Authorization', `Bearer ${token}`)
        .send(testBook);
      
      const res = await request(app)
        .get('/api/books')
        .set('Authorization', `Bearer ${token}`);
      
      expect(res.statusCode).toEqual(200);
      expect(res.body.success).toBeTruthy();
      expect(res.body.data).toBeInstanceOf(Array);
      expect(res.body.data.length).toBeGreaterThan(0);
      expect(res.body.pagination).toBeDefined();
    });
  });
  
  // Additional tests for GET, PUT, DELETE...
});

Performance Optimization

Let's review our implementation for performance bottlenecks:

1. Database Indexing: Add indexes for frequently queried fields
2. Caching: Implement Redis caching for frequently accessed data
3. Query Optimization: Refine database queries to fetch only needed fields
4. Pagination: Ensure all list endpoints support pagination
5. Rate Limiting: Implement rate limiting to prevent abuse

// src/models/Book.js - Adding indexes
const bookSchema = new mongoose.Schema({
  title: {
    type: String,
    required: [true, 'Title is required'],
    trim: true,
    index: true // Add index for search performance
  },
  author: {
    type: String,
    required: [true, 'Author is required'],
    trim: true,
    index: true // Add index for search performance
  },
  // ... other fields
});

// src/middleware/rateLimiter.js - Rate limiting
const rateLimit = require('express-rate-limit');
const RedisStore = require('rate-limit-redis');
const redis = require('../config/redis');

// Create rate limiter
const createRateLimiter = (options) => {
  return rateLimit({
    store: new RedisStore({
      client: redis,
      prefix: 'ratelimit:'
    }),
    windowMs: options.windowMs || 15 * 60 * 1000, // 15 minutes
    max: options.max || 100, // Limit each IP to 100 requests per windowMs
    message: options.message || 'Too many requests, please try again later',
    standardHeaders: true,
    legacyHeaders: false
  });
};

// Different rate limiters for different routes
exports.loginLimiter = createRateLimiter({
  windowMs: 15 * 60 * 1000, // 15 minutes
  max: 5, // 5 login attempts per 15 minutes
  message: 'Too many login attempts, please try again after 15 minutes'
});

exports.apiLimiter = createRateLimiter({
  windowMs: 60 * 1000, // 1 minute
  max: 60 // 60 requests per minute
});

// src/routes/authRoutes.js - Apply rate limiting
const { loginLimiter } = require('../middleware/rateLimiter');

router.post('/login', loginLimiter, validate(loginValidator), authController.login);

Security Enhancements

Let's review and enhance security measures:

1. Password Policies: Enforce strong password requirements
2. Input Sanitization: Sanitize all user inputs to prevent injection attacks
3. JWT Refresh Tokens: Implement refresh tokens for secure session management
4. CORS Configuration: Restrict cross-origin requests to trusted domains
5. Security Headers: Add security headers with Helmet middleware

// src/config/security.js
const helmet = require('helmet');
const xss = require('xss-clean');
const mongoSanitize = require('express-mongo-sanitize');
const hpp = require('hpp');

// Configure security middleware
const configureSecurity = (app) => {
  // Set security headers with Helmet
  app.use(helmet());
  
  // Prevent XSS attacks
  app.use(xss());
  
  // Sanitize data to prevent NoSQL injection
  app.use(mongoSanitize());
  
  // Prevent HTTP parameter pollution
  app.use(hpp());
  
  // Set CORS options
  const corsOptions = {
    origin: process.env.CORS_ORIGIN || '*',
    methods: ['GET', 'POST', 'PUT', 'DELETE'],
    allowedHeaders: ['Content-Type', 'Authorization'],
    credentials: true,
    maxAge: 86400 // 24 hours
  };
  
  app.use(cors(corsOptions));
  
  return app;
};

module.exports = configureSecurity;

// Apply in app.js
const configureSecurity = require('./config/security');
// ...
configureSecurity(app);

Documentation

Create comprehensive API documentation with Swagger/OpenAPI:

// src/docs/swagger.json
{
  "openapi": "3.0.0",
  "info": {
    "title": "Bookshelf API",
    "description": "API for managing a virtual bookshelf system",
    "version": "1.0.0"
  },
  "servers": [
    {
      "url": "http://localhost:5000/api",
      "description": "Local development server"
    }
  ],
  "paths": {
    "/auth/register": {
      "post": {
        "summary": "Register a new user",
        "tags": ["Authentication"],
        "requestBody": {
          "required": true,
          "content": {
            "application/json": {
              "schema": {
                "$ref": "#/components/schemas/RegisterRequest"
              }
            }
          }
        },
        "responses": {
          "201": {
            "description": "User registered successfully",
            "content": {
              "application/json": {
                "schema": {
                  "$ref": "#/components/schemas/AuthResponse"
                }
              }
            }
          },
          "400": {
            "description": "Validation error",
            "content": {
              "application/json": {
                "schema": {
                  "$ref": "#/components/schemas/ErrorResponse"
                }
              }
            }
          }
        }
      }
    }
    // Additional endpoint documentation...
  },
  "components": {
    "schemas": {
      "RegisterRequest": {
        "type": "object",
        "required": ["name", "email", "password"],
        "properties": {
          "name": {
            "type": "string",
            "description": "User's full name"
          },
          "email": {
            "type": "string",
            "format": "email",
            "description": "User's email address"
          },
          "password": {
            "type": "string",
            "format": "password",
            "description": "User's password"
          }
        }
      },
      "AuthResponse": {
        "type": "object",
        "properties": {
          "success": {
            "type": "boolean",
            "example": true
          },
          "token": {
            "type": "string"
          }
        }
      },
      "ErrorResponse": {
        "type": "object",
        "properties": {
          "success": {
            "type": "boolean",
            "example": false
          },
          "error": {
            "type": "object",
            "properties": {
              "message": {
                "type": "string"
              },
              "code": {
                "type": "string"
              },
              "statusCode": {
                "type": "integer"
              },
              "details": {
                "type": "object"
              }
            }
          }
        }
      }
      // Additional schema definitions...
    },
    "securitySchemes": {
      "bearerAuth": {
        "type": "http",
        "scheme": "bearer",
        "bearerFormat": "JWT"
      }
    }
  }
}

Final Reflections

After completing our implementation and review, here are the key lessons learned:

• Architecture Matters: A well-designed architecture makes development, testing, and maintenance easier
• Error Handling is Critical: Comprehensive error handling improves reliability and user experience
• Separation of Concerns: Keeping business logic in services makes the codebase more maintainable
• Input Validation: Thorough validation prevents many potential issues
• Testing is Essential: Tests give confidence in the reliability of the code
• Security Requires Multiple Layers: Security must be addressed at every level of the application
• Documentation Makes APIs Usable: Good documentation is as important as good code

Summary: Polya's Method in API Development

Let's reflect on how George Polya's problem-solving approach guided our API development process:

The Value of a Structured Approach

Using Polya's method provided several benefits:

• Clear Starting Point: By first understanding the problem, we avoided jumping into code too quickly
• Coherent Architecture: Planning before coding led to a well-designed system
• Systematic Implementation: Executing a clear plan made development more efficient
• Continuous Improvement: Reflecting on our solution helped identify optimizations

Beyond the Weekend Project

The approach demonstrated in this project can be extended to larger applications:

• Iterative Development: Apply Polya's method to each feature or module
• Team Collaboration: Use the structured approach to align team members
• Maintenance: Return to the understanding phase when bugs or new requirements arise
• Scaling: The modular architecture supports growing the application over time

Further Learning Resources

• Express.js Error Handling Guide
• Node.js Best Practices
• OpenAPI Specification
• Mongoose Documentation
• Jest Testing Framework
• OWASP REST Security Cheat Sheet