Introduction to Request Validation
When building APIs with Express.js, one of the most critical aspects of security and reliability is proper validation and sanitization of incoming data. In this lecture, we'll learn why this is important and how to implement it effectively.
"Never trust user input. Never. Not even if your user is another system."
Why Validate and Sanitize?
- Prevent Security Vulnerabilities: SQL injection, XSS attacks, command injection
- Ensure Data Integrity: Maintain consistency in your database
- Provide Better User Experience: Immediate feedback on invalid inputs
- Simplify Backend Logic: Work with clean, predictable data
- Reduce Server Errors: Avoid crashes from unexpected data types
Common Validation Scenarios
Before diving into implementation, let's understand what we typically need to validate:
Data Types and Structures
- Is this field a string, number, boolean, etc.?
- Is this array properly structured?
- Does this object contain the required fields?
Value Constraints
- Is this string within the expected length range?
- Is this number between valid minimum and maximum values?
- Does this email address match a valid pattern?
- Does this date fall within an acceptable range?
Business Rules
- Does this password meet complexity requirements?
- Is this username unique in our system?
- Does the user have permission to access this resource?
- Are the related entities consistent with each other?
Real-World Example: E-commerce Product Creation
Imagine validating a request to create a new product:
- Product name: Required, string, 3-100 characters
- Price: Required, number, greater than 0
- Category ID: Required, must exist in categories database
- Description: Optional, string, max 2000 characters
- Features: Optional array of strings, each max 200 characters
Validation Approaches in Express
Manual Validation
The most basic approach is writing custom validation logic directly in your route handlers:
app.post('/api/products', (req, res) => {
const { name, price, categoryId } = req.body;
// Manual validation
const errors = [];
if (!name || typeof name !== 'string') {
errors.push('Product name is required and must be a string');
} else if (name.length < 3 || name.length > 100) {
errors.push('Product name must be between 3 and 100 characters');
}
if (!price || typeof price !== 'number' || price <= 0) {
errors.push('Price is required and must be a positive number');
}
if (!categoryId) {
errors.push('Category ID is required');
}
if (errors.length > 0) {
return res.status(400).json({ errors });
}
// If validation passes, continue with business logic
// ...
});
Pros of Manual Validation:
- Complete control over validation logic
- No dependencies required
- Can be tailored to very specific use cases
Cons of Manual Validation:
- Verbose and repetitive code
- Easy to miss validation cases
- Harder to maintain as application grows
- Validation logic mixed with handler logic
Using Express-Validator
Express-validator is one of the most popular validation libraries for Express.js, providing a robust set of validation and sanitization functions.
Getting Started with Express-Validator
First, install the package:
npm install express-validator
Basic implementation:
const express = require('express');
const { body, validationResult } = require('express-validator');
const app = express();
app.use(express.json());
app.post('/api/products', [
// Define validation chain
body('name')
.exists().withMessage('Product name is required')
.isString().withMessage('Product name must be a string')
.isLength({ min: 3, max: 100 }).withMessage('Product name must be between 3 and 100 characters'),
body('price')
.exists().withMessage('Price is required')
.isNumeric().withMessage('Price must be a number')
.custom(value => value > 0).withMessage('Price must be greater than 0'),
body('categoryId')
.exists().withMessage('Category ID is required')
], (req, res) => {
// Check for validation errors
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
}
// Validation passed, continue with business logic
// ...
});
Validation Chains
Express-validator uses the concept of validation chains, which allow you to apply multiple validation rules to a single field in a fluent API style.
Common Validators
| Validator | Purpose | Example |
|---|---|---|
| isString() | Checks if value is a string | body('name').isString() |
| isNumeric() | Checks if value is numeric | body('age').isNumeric() |
| isEmail() | Validates email format | body('email').isEmail() |
| isLength() | Checks string length | body('password').isLength({ min: 8 }) |
| isIn() | Checks if value is in an array | body('role').isIn(['admin', 'user']) |
| custom() | Custom validation logic | body('id').custom(value => isValidUUID(value)) |
Advanced Validation Techniques
Custom Validators
When built-in validators aren't enough, you can create custom validation logic:
// Check if username is unique
body('username').custom(async value => {
const user = await User.findOne({ where: { username: value } });
if (user) {
throw new Error('Username already in use');
}
return true; // Indicates validation success
});
Conditional Validation
Sometimes validation rules depend on other fields or conditions:
// Only validate shippingAddress if delivery type is 'physical'
body('shippingAddress')
.if(body('deliveryType').equals('physical'))
.notEmpty().withMessage('Shipping address is required for physical delivery');
Schema-Based Validation
For complex objects, schema-based validation can be more maintainable:
const { checkSchema } = require('express-validator');
app.post('/api/users', checkSchema({
username: {
in: ['body'],
isString: true,
isLength: {
options: { min: 3, max: 20 },
errorMessage: 'Username must be between 3 and 20 characters'
},
custom: {
options: async value => {
const exists = await User.findOne({ where: { username: value } });
if (exists) {
throw new Error('Username already exists');
}
return true;
}
}
},
email: {
in: ['body'],
isEmail: true,
normalizeEmail: true,
errorMessage: 'Invalid email address'
},
password: {
in: ['body'],
isLength: {
options: { min: 8 },
errorMessage: 'Password must be at least 8 characters'
},
matches: {
options: /^(?=.*\d)(?=.*[a-z])(?=.*[A-Z])(?=.*[!@#$%^&*]).*$/,
errorMessage: 'Password must include at least one number, one uppercase letter, and one special character'
}
}
}), (req, res) => {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
}
// Process valid data
});
Data Sanitization
Validation ensures data is valid, but sanitization ensures it's safe and properly formatted. Express-validator provides sanitization methods that modify the incoming data.
Common Sanitizers
| Sanitizer | Purpose | Example |
|---|---|---|
| trim() | Removes whitespace from both ends | body('name').trim() |
| escape() | Converts HTML special characters | body('comment').escape() |
| normalizeEmail() | Normalizes email addresses | body('email').normalizeEmail() |
| toInt() | Converts to integer | body('age').toInt() |
| toLowerCase() | Converts to lowercase | body('username').toLowerCase() |
Combined Validation and Sanitization
app.post('/api/register', [
// Validate and sanitize email
body('email')
.isEmail().withMessage('Invalid email address')
.normalizeEmail() // Sanitize: lowercase domain, remove dots in Gmail, etc.
.trim(),
// Validate and sanitize name
body('name')
.isString().withMessage('Name must be a string')
.isLength({ min: 2, max: 50 }).withMessage('Name must be between 2 and 50 characters')
.trim() // Remove whitespace
.escape(), // Escape HTML special characters
// Validate password but don't sanitize (sanitizing passwords could alter them)
body('password')
.isLength({ min: 8 }).withMessage('Password must be at least 8 characters')
], (req, res) => {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
}
// Use sanitized data
const { email, name, password } = req.body;
// ...
});
Important: XSS Prevention
While escape() helps prevent XSS attacks, it's not sufficient as the only defense. Always implement proper output encoding in your templates or frontend. Consider using a library like OWASP ESAPI for more comprehensive security.
Best Practices for Validation and Sanitization
Separation of Concerns
Keep validation logic separated from your route handlers for better maintainability:
// validators/user.js
const { body } = require('express-validator');
exports.createUserValidation = [
body('username').isString().isLength({ min: 3, max: 20 }),
body('email').isEmail().normalizeEmail(),
body('password').isLength({ min: 8 })
];
// routes/user.js
const express = require('express');
const { validationResult } = require('express-validator');
const { createUserValidation } = require('../validators/user');
const router = express.Router();
router.post('/', createUserValidation, (req, res) => {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
}
// Process valid data
});
module.exports = router;
Consistent Error Responses
Create a middleware for handling validation errors consistently:
// middlewares/validation.js
const { validationResult } = require('express-validator');
exports.validate = validations => {
return async (req, res, next) => {
await Promise.all(validations.map(validation => validation.run(req)));
const errors = validationResult(req);
if (errors.isEmpty()) {
return next();
}
return res.status(400).json({
status: 'error',
errors: errors.array().map(err => ({
field: err.param,
message: err.msg
}))
});
};
};
// Usage in routes
const { validate } = require('../middlewares/validation');
const { createUserValidation } = require('../validators/user');
router.post('/', validate(createUserValidation), (req, res) => {
// No need to check for validation errors here
// Process valid data
});
Database Validation vs. API Validation
Remember that API validation doesn't replace database validation:
- API Validation: Validates user input before processing
- Database Validation: Ensures data integrity at the database level
Always implement both layers of validation for robust applications.
Real-World Example: Multi-Layer Validation
- Frontend Validation: Immediate feedback to users
- API Validation: Guards against malicious requests or bypassed frontend validation
- Service Layer Validation: Ensures business rules are met
- Database Constraints: Last line of defense for data integrity
All layers should be implemented for critical systems.
Alternative Validation Libraries
Joi
Joi offers schema-based validation with a very expressive API:
const Joi = require('joi');
const express = require('express');
const app = express();
app.use(express.json());
const userSchema = Joi.object({
username: Joi.string().alphanum().min(3).max(30).required(),
email: Joi.string().email().required(),
password: Joi.string().pattern(new RegExp('^[a-zA-Z0-9]{3,30}$')).required(),
birthYear: Joi.number().integer().min(1900).max(2013)
});
app.post('/api/users', (req, res) => {
const { error, value } = userSchema.validate(req.body);
if (error) {
return res.status(400).json({
error: error.details.map(detail => detail.message)
});
}
// Process valid data
res.json({ success: true, user: value });
});
Yup
Yup provides schema validation with a focus on strongly typed objects:
const yup = require('yup');
const express = require('express');
const app = express();
app.use(express.json());
const userSchema = yup.object({
username: yup.string().required().min(3).max(20),
email: yup.string().email().required(),
password: yup.string().required().min(8)
.matches(/^(?=.*[a-z])(?=.*[A-Z])(?=.*\d)/, 'Password requirements not met'),
age: yup.number().positive().integer().min(18).required()
});
app.post('/api/users', async (req, res) => {
try {
const validatedData = await userSchema.validate(req.body);
// Process valid data
res.json({ success: true, user: validatedData });
} catch (error) {
res.status(400).json({ error: error.message });
}
});
Comparison of Validation Libraries
| Library | Strengths | Ideal Use Cases |
|---|---|---|
| Express-validator | Tight Express integration, middleware-based approach | Express applications with field-by-field validation needs |
| Joi | Powerful schema validation, detailed error messages | Complex nested objects, applications needing schema definition |
| Yup | TypeScript friendly, async validation support | TypeScript projects, frontend and backend code sharing |
Security Considerations
Input Sanitization for Security
Beyond validation, proper sanitization helps prevent several security vulnerabilities:
SQL Injection Prevention
When using raw SQL queries (though ORMs are preferred):
// BAD: Vulnerable to SQL injection
const query = `SELECT * FROM users WHERE email = '${req.body.email}'`;
// GOOD: Using parameterized queries
const query = 'SELECT * FROM users WHERE email = ?';
connection.query(query, [req.body.email]);
XSS Prevention
Always sanitize user-generated content that will be displayed:
// Using express-validator
app.post('/comments', [
body('comment').not().isEmpty().trim().escape()
], (req, res) => {
// Store and display sanitized comment
});
NoSQL Injection Prevention
MongoDB queries can also be vulnerable:
// BAD: Vulnerable construction
const query = { username: req.body.username };
// BETTER: Type checking and sanitization
const username = typeof req.body.username === 'string'
? req.body.username.trim()
: '';
const query = { username: username };
OWASP Input Validation Cheat Sheet
The OWASP Input Validation Cheat Sheet provides excellent guidance on secure validation practices. Key principles include:
- Validate all input against a whitelist of allowed characters
- Apply both syntactic and semantic validation
- Validate on the server side, not just the client side
- Consider the context where data will be used for appropriate escaping
Practical Exercise
Build a Validated API Endpoint
Create an Express.js API endpoint for user registration with the following validation requirements:
Requirements
- Username: 3-20 characters, alphanumeric with underscores
- Email: Valid email format, normalized
- Password: 8+ characters, must include uppercase, lowercase, number
- Age: Optional, but if provided must be 13 or older
- Terms: Must be accepted (boolean true)
Tasks
- Set up an Express.js server with the necessary dependencies
- Create a POST route for user registration
- Implement validation and sanitization using express-validator
- Create meaningful error responses that clearly communicate validation failures
- Implement a custom validator to check if the username is already taken
- Bonus: Add password strength scoring
Testing
Test your endpoint with various inputs, including:
- Valid data that should pass all validation
- Invalid data for each field
- Missing required fields
- Malicious inputs (SQL injection attempts, XSS scripts)
Real-World Application
Let's see how validation and sanitization fit into a complete user registration API:
// routes/auth.js
const express = require('express');
const { body, validationResult } = require('express-validator');
const bcrypt = require('bcrypt');
const User = require('../models/User');
const router = express.Router();
// Validation middleware
const registerValidation = [
// Username validation
body('username')
.trim()
.isLength({ min: 3, max: 20 })
.withMessage('Username must be between 3 and 20 characters')
.isAlphanumeric('en-US', { ignore: '_' })
.withMessage('Username can only contain letters, numbers and underscores')
.custom(async value => {
const existingUser = await User.findOne({ username: value });
if (existingUser) {
throw new Error('Username is already taken');
}
return true;
}),
// Email validation
body('email')
.isEmail()
.withMessage('Please provide a valid email address')
.normalizeEmail()
.custom(async value => {
const existingUser = await User.findOne({ email: value });
if (existingUser) {
throw new Error('Email is already registered');
}
return true;
}),
// Password validation
body('password')
.isLength({ min: 8 })
.withMessage('Password must be at least 8 characters long')
.matches(/^(?=.*\d)(?=.*[a-z])(?=.*[A-Z])/)
.withMessage('Password must contain at least one uppercase letter, one lowercase letter, and one number'),
// Password confirmation
body('confirmPassword')
.custom((value, { req }) => {
if (value !== req.body.password) {
throw new Error('Passwords do not match');
}
return true;
})
];
// Registration route
router.post('/register', registerValidation, async (req, res) => {
// Check for validation errors
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({
success: false,
errors: errors.array()
});
}
try {
const { username, email, password } = req.body;
// Hash password
const salt = await bcrypt.genSalt(10);
const hashedPassword = await bcrypt.hash(password, salt);
// Create new user
const user = new User({
username,
email,
password: hashedPassword
});
// Save user to database
await user.save();
// Return success response
res.status(201).json({
success: true,
message: 'User registered successfully'
});
} catch (error) {
console.error('Registration error:', error);
res.status(500).json({
success: false,
message: 'Server error occurred during registration'
});
}
});
module.exports = router;
This example demonstrates:
- Comprehensive validation rules for each field
- Custom validators for checking existing users
- Clear error messages for each validation rule
- Separation of validation logic from route handler logic
- Proper error handling for validation failures
- Data sanitization (trimming, normalizing)
Summary
Key Takeaways
- Validation and sanitization are critical for security and reliability
- Express-validator provides powerful middleware-based validation
- Separate validation logic from route handlers for better maintainability
- Sanitize user input to prevent security vulnerabilities
- Consider schema-based validation for complex data structures
- Implement multiple layers of validation for robust applications
Additional Resources
- Express-validator Documentation
- Joi Documentation
- Yup GitHub
- OWASP Input Validation Cheat Sheet
- OWASP Top 10 Web Application Security Risks
Next Steps
In our next lecture, we'll explore Response Formatting and Status Codes, where you'll learn how to create consistent, informative API responses that follow RESTful best practices.
Further Practice
Exercises
- Expand the registration API to include additional fields (address, phone number) with appropriate validation.
- Implement a password strength validator that gives feedback on password quality.
- Create a middleware that logs validation errors for monitoring purposes.
- Compare the performance of express-validator vs. Joi for validating a complex nested object.
- Create a validation middleware for file uploads, ensuring only safe files are accepted.
Project Idea
Build a small "form builder" API that allows users to define custom forms with validation rules. The API should:
- Allow defining fields with various types (text, number, email, etc.)
- Support setting validation rules for each field
- Dynamically generate validation middleware based on form definitions
- Provide a submission endpoint that validates according to the defined rules