Core Node.js Modules (fs, path, http)

Introduction to Core Modules

One of Node.js's greatest strengths is its rich set of built-in modules that provide essential functionality without requiring external dependencies. These core modules are the foundation upon which the Node.js ecosystem is built, offering standardized ways to interact with the file system, handle network operations, manage paths, and much more.

Core modules are like the basic toolkit that comes with your car—essential tools designed specifically for the system, ready to use without additional purchases.

graph TD A[Node.js Core Modules] A --> B[File System
fs] A --> C[Path
path] A --> D[HTTP
http, https] A --> E[Operating System
os] A --> F[Utilities
util] A --> G[Events
events] A --> H[Streams
stream] A --> I[More...] style A fill:#f9f,stroke:#333,stroke-width:2px

These modules are immediately available in your Node.js applications without installation—they're part of the core Node.js runtime. To use a core module, you simply require it:

// Importing core modules
const fs = require('fs');
const path = require('path');
const http = require('http');
const os = require('os');

In today's lecture, we'll focus on three of the most commonly used core modules: fs (file system), path (path manipulation), and http (HTTP server/client). These modules form the backbone of many Node.js applications and understanding them is essential for effective Node.js development.

The File System Module (fs)

The fs module provides an API for interacting with the file system in a way that models the standard POSIX functions. It allows you to work with files and directories—reading, writing, updating, deleting, and more.

One key aspect of the fs module is that most methods come in three forms:

Synchronous: Blocks execution until the operation completes
Callback-based asynchronous: Takes a callback function for handling results
Promise-based asynchronous: Returns a Promise for modern async/await syntax

Reading Files

const fs = require('fs');

// Synchronous (blocking) version
try {
    const data = fs.readFileSync('file.txt', 'utf8');
    console.log(data);
} catch (err) {
    console.error('Error reading file:', err);
}

// Callback-based asynchronous version
fs.readFile('file.txt', 'utf8', (err, data) => {
    if (err) {
        console.error('Error reading file:', err);
        return;
    }
    console.log(data);
});

// Promise-based asynchronous version (modern)
const fsPromises = require('fs').promises;

async function readFile() {
    try {
        const data = await fsPromises.readFile('file.txt', 'utf8');
        console.log(data);
    } catch (err) {
        console.error('Error reading file:', err);
    }
}

readFile();

In most cases, you should use the asynchronous versions to avoid blocking the event loop, especially in server environments where performance and concurrency are important.

Writing Files

// Synchronous write
try {
    fs.writeFileSync('output.txt', 'Hello, Node.js!');
    console.log('File written successfully');
} catch (err) {
    console.error('Error writing file:', err);
}

// Asynchronous write with callback
fs.writeFile('output.txt', 'Hello, Node.js!', (err) => {
    if (err) {
        console.error('Error writing file:', err);
        return;
    }
    console.log('File written successfully');
});

// Promise-based asynchronous write
async function writeFile() {
    try {
        await fsPromises.writeFile('output.txt', 'Hello, Node.js!');
        console.log('File written successfully');
    } catch (err) {
        console.error('Error writing file:', err);
    }
}

writeFile();

Appending to Files

// Append to a file
fs.appendFile('logs.txt', `\nLog entry at ${new Date()}`, (err) => {
    if (err) {
        console.error('Error appending to file:', err);
        return;
    }
    console.log('Data appended successfully');
});

Working with Directories

// Check if a directory exists
if (!fs.existsSync('data')) {
    // Create a directory
    fs.mkdirSync('data');
    console.log('Directory created');
}

// Read directory contents
fs.readdir('data', (err, files) => {
    if (err) {
        console.error('Error reading directory:', err);
        return;
    }
    console.log('Directory contents:', files);
});

The fs module also offers methods for:

Renaming files and directories (fs.rename)
Deleting files (fs.unlink)
Getting file information (fs.stat)
Watching for file changes (fs.watch)
Creating readable and writable streams (fs.createReadStream, fs.createWriteStream)

Think of the fs module as a filing clerk for your application—it handles storing, retrieving, organizing, and modifying documents as needed.

File Streams for Efficient Processing

When working with large files, it's often more efficient to use streams rather than reading or writing the entire file at once. Streams process data in chunks, which is more memory-efficient and allows for processing data before it's fully read.

Reading Files with Streams

const fs = require('fs');

// Create a readable stream
const readStream = fs.createReadStream('large-file.txt', { encoding: 'utf8' });

// Handle stream events
readStream.on('data', (chunk) => {
    console.log(`Received ${chunk.length} characters of data`);
    // Process chunk here
});

readStream.on('end', () => {
    console.log('Finished reading file');
});

readStream.on('error', (err) => {
    console.error('Error reading file:', err);
});

Writing Files with Streams

// Create a writable stream
const writeStream = fs.createWriteStream('output-file.txt');

// Write data to the stream
writeStream.write('Hello, ');
writeStream.write('World!\n');
writeStream.end('End of file');

writeStream.on('finish', () => {
    console.log('Finished writing to file');
});

writeStream.on('error', (err) => {
    console.error('Error writing to file:', err);
});

Piping Streams

One of the most powerful features of streams is the ability to pipe them together, creating data processing pipelines:

// Copy a file efficiently
const readStream = fs.createReadStream('input.txt');
const writeStream = fs.createWriteStream('output.txt');

// Pipe the read stream into the write stream
readStream.pipe(writeStream);

writeStream.on('finish', () => {
    console.log('File copied successfully');
});

This code copies a file by piping the read stream directly to the write stream, which is much more memory-efficient than reading the entire file into memory first, especially for large files.

Streams are like water pipes in a plumbing system—they allow data to flow continuously in small amounts (chunks) rather than requiring a massive container to hold all the water at once.

The Path Module

The path module provides utilities for working with file and directory paths. It's essential for writing cross-platform applications because path formats vary between operating systems (e.g., backslashes on Windows vs. forward slashes on UNIX-based systems).

const path = require('path');

Key Path Operations

// Join path segments (cross-platform)
const fullPath = path.join(__dirname, 'data', 'users.json');
console.log(fullPath);
// On Windows: C:\projects\my-app\data\users.json
// On UNIX: /projects/my-app/data/users.json

// Resolve an absolute path (handles .. and . segments)
const absolutePath = path.resolve('data', '..', 'config', 'settings.json');
console.log(absolutePath);

// Get the directory name from a path
const dirName = path.dirname('/users/files/document.txt');
console.log(dirName); // /users/files

// Get the file name from a path
const fileName = path.basename('/users/files/document.txt');
console.log(fileName); // document.txt

// Get the file name without extension
const fileNameWithoutExt = path.basename('/users/files/document.txt', '.txt');
console.log(fileNameWithoutExt); // document

// Get the file extension
const extension = path.extname('/users/files/document.txt');
console.log(extension); // .txt

// Parse a path into components
const pathObj = path.parse('/users/files/document.txt');
console.log(pathObj);
// {
//   root: '/',
//   dir: '/users/files',
//   base: 'document.txt',
//   ext: '.txt',
//   name: 'document'
// }

// Format path components into a path string
const newPath = path.format({
    dir: '/users/files',
    base: 'new-document.txt'
});
console.log(newPath); // /users/files/new-document.txt

Special Path Variables

// Platform-specific path delimiter
console.log(path.delimiter); // ':' on UNIX, ';' on Windows

// Platform-specific path separator
console.log(path.sep); // '/' on UNIX, '\' on Windows

// Current script directory
console.log(__dirname); // Absolute path to the directory containing the current file

// Current script file path
console.log(__filename); // Absolute path to the current file

Using the path module is like having a GPS for your file system—it helps you navigate and manipulate paths reliably across different environments, avoiding the pitfalls of manual string concatenation or platform-specific assumptions.

flowchart TD A[Path Operations] A --> B[Joining
path.join()] A --> C[Resolving
path.resolve()] A --> D[Extracting
path.dirname()
path.basename()
path.extname()] A --> E[Parsing/Formatting
path.parse()
path.format()]

The HTTP Module

The http module allows Node.js to transfer data over HTTP (Hypertext Transfer Protocol). It can be used to create both HTTP servers that listen for requests and HTTP clients that make requests to other servers.

const http = require('http');

Creating a Basic HTTP Server

const http = require('http');

// Create a server
const server = http.createServer((req, res) => {
    // Request object (req) contains information about the client request
    console.log(`Request received: ${req.method} ${req.url}`);
    
    // Set response headers
    res.setHeader('Content-Type', 'text/html');
    
    // Write response body
    res.write('<h1>Hello from Node.js!</h1>');
    res.write('<p>This is a simple HTTP server.</p>');
    
    // End the response
    res.end();
});

// Start the server on port 3000
const PORT = 3000;
server.listen(PORT, () => {
    console.log(`Server running at http://localhost:${PORT}/`);
});

This basic server responds to all requests with the same HTML content. In a real application, you would handle different URLs, HTTP methods, and more.

Handling Different Routes

const http = require('http');

const server = http.createServer((req, res) => {
    res.setHeader('Content-Type', 'text/html');
    
    // Handle different URL paths
    if (req.url === '/') {
        res.statusCode = 200;
        res.end('<h1>Home Page</h1>');
    } else if (req.url === '/about') {
        res.statusCode = 200;
        res.end('<h1>About Page</h1>');
    } else {
        // Not found
        res.statusCode = 404;
        res.end('<h1>404 Not Found</h1>');
    }
});

server.listen(3000, () => {
    console.log('Server running at http://localhost:3000/');
});

Handling POST Requests and JSON Data

const http = require('http');

const server = http.createServer((req, res) => {
    // Check if it's a POST request to /api/users
    if (req.method === 'POST' && req.url === '/api/users') {
        let body = '';
        
        // Collect data chunks
        req.on('data', (chunk) => {
            body += chunk.toString();
        });
        
        // Process the complete request body
        req.on('end', () => {
            try {
                const userData = JSON.parse(body);
                console.log('Received user data:', userData);
                
                // Send JSON response
                res.setHeader('Content-Type', 'application/json');
                res.statusCode = 201; // Created
                res.end(JSON.stringify({ 
                    message: 'User created successfully',
                    user: userData
                }));
            } catch (error) {
                // Handle JSON parsing error
                res.setHeader('Content-Type', 'application/json');
                res.statusCode = 400; // Bad Request
                res.end(JSON.stringify({ error: 'Invalid JSON data' }));
            }
        });
    } else {
        // Handle other requests
        res.statusCode = 404;
        res.end('Not Found');
    }
});

server.listen(3000, () => {
    console.log('Server running at http://localhost:3000/');
});

This server specifically handles POST requests to the /api/users endpoint, parsing JSON data from the request body and returning a JSON response.

Making HTTP Requests as a Client

const http = require('http');

// Options for the request
const options = {
    hostname: 'jsonplaceholder.typicode.com',
    path: '/users/1',
    method: 'GET'
};

// Make the request
const req = http.request(options, (res) => {
    let data = '';
    
    // A chunk of data has been received
    res.on('data', (chunk) => {
        data += chunk;
    });
    
    // The whole response has been received
    res.on('end', () => {
        console.log(JSON.parse(data));
    });
});

// Handle errors
req.on('error', (error) => {
    console.error('Error making request:', error);
});

// End the request
req.end();

The http module can be used to make outgoing HTTP requests as well, though in practice, many developers use higher-level libraries like axios or node-fetch for this purpose because they offer a more convenient API.

sequenceDiagram participant Client participant Server Client->>Server: HTTP Request Note right of Client: http.request() Server-->>Client: HTTP Response Note left of Server: in request callback

The HTTP module is like a telephone system for your application—it allows two-way communication between clients and servers using the standardized HTTP protocol that powers the web.

The HTTPS Module

For secure communication over HTTP, Node.js provides the https module, which is very similar to the http module but adds TLS/SSL encryption:

const https = require('https');
const fs = require('fs');

// SSL options with certificate and key
const options = {
    key: fs.readFileSync('key.pem'),
    cert: fs.readFileSync('cert.pem')
};

// Create HTTPS server
const server = https.createServer(options, (req, res) => {
    res.statusCode = 200;
    res.setHeader('Content-Type', 'text/plain');
    res.end('Hello, secure world!');
});

server.listen(443, () => {
    console.log('HTTPS Server running on port 443');
});

For development purposes, you can generate self-signed certificates using tools like OpenSSL, but for production, you should use certificates from a trusted Certificate Authority (CA).

Like the http module, https can also be used for making secure outgoing requests:

const https = require('https');

https.get('https://api.github.com/users/nodejs', (res) => {
    let data = '';
    
    res.on('data', (chunk) => {
        data += chunk;
    });
    
    res.on('end', () => {
        console.log(JSON.parse(data));
    });
}).on('error', (err) => {
    console.error('Error: ', err.message);
});

The https module is like an encrypted telephone line—it provides the same communication capabilities as http, but with added security to protect sensitive information during transmission.

Practical Example: Building a File Server

Let's combine the fs, path, and http modules to create a simple file server that can serve files from a directory:

const http = require('http');
const fs = require('fs');
const path = require('path');

// Directory where public files are stored
const PUBLIC_DIR = path.join(__dirname, 'public');

// Create the server
const server = http.createServer((req, res) => {
    // Normalize the URL and create the file path
    const filePath = path.join(PUBLIC_DIR, req.url === '/' ? 'index.html' : req.url);
    
    // Get the file extension
    const extname = path.extname(filePath);
    
    // Define content types based on file extension
    const contentType = {
        '.html': 'text/html',
        '.js': 'text/javascript',
        '.css': 'text/css',
        '.json': 'application/json',
        '.png': 'image/png',
        '.jpg': 'image/jpeg',
        '.gif': 'image/gif'
    }[extname] || 'text/plain';
    
    // Check if the file exists
    fs.access(filePath, fs.constants.F_OK, (err) => {
        if (err) {
            // File doesn't exist
            res.statusCode = 404;
            res.setHeader('Content-Type', 'text/html');
            res.end('<h1>404 Not Found</h1>');
            return;
        }
        
        // File exists, serve it
        fs.readFile(filePath, (err, content) => {
            if (err) {
                // Server error
                res.statusCode = 500;
                res.setHeader('Content-Type', 'text/html');
                res.end(`<h1>500 Server Error</h1><p>${err.message}</p>`);
            } else {
                // Success
                res.statusCode = 200;
                res.setHeader('Content-Type', contentType);
                res.end(content);
            }
        });
    });
});

const PORT = process.env.PORT || 3000;

server.listen(PORT, () => {
    console.log(`Server running at http://localhost:${PORT}/`);
});

This server serves static files from a 'public' directory, determining the appropriate content type based on file extension. It's a simple example of how these core modules can work together to create useful functionality.

For a more robust static file server, you might use the fs.createReadStream method instead of fs.readFile to handle large files more efficiently:

// Replace the fs.readFile block with this streaming approach
fs.stat(filePath, (err, stats) => {
    if (err) {
        res.statusCode = 500;
        res.setHeader('Content-Type', 'text/html');
        res.end(`<h1>500 Server Error</h1><p>${err.message}</p>`);
        return;
    }
    
    res.statusCode = 200;
    res.setHeader('Content-Type', contentType);
    res.setHeader('Content-Length', stats.size);
    
    // Stream the file to the response
    const fileStream = fs.createReadStream(filePath);
    fileStream.pipe(res);
    
    fileStream.on('error', (error) => {
        res.statusCode = 500;
        res.end(`<h1>500 Server Error</h1><p>${error.message}</p>`);
    });
});

This approach streams the file directly to the response, which is more memory-efficient for large files and starts sending data to the client immediately rather than waiting for the entire file to be read.

Other Essential Core Modules

While we've focused on fs, path, and http/https, Node.js includes several other important core modules:

Events Module

The foundation of Node.js's event-driven architecture:

const EventEmitter = require('events');

// Create a custom event emitter
class MyEmitter extends EventEmitter {}
const myEmitter = new MyEmitter();

// Add an event listener
myEmitter.on('event', (a, b) => {
    console.log('Event fired with arguments:', a, b);
});

// Emit the event
myEmitter.emit('event', 'arg1', 'arg2');

URL Module

For parsing and formatting URLs:

const url = require('url');

const myUrl = new URL('https://example.com/path?query=string#hash');

console.log(myUrl.hostname); // example.com
console.log(myUrl.pathname); // /path
console.log(myUrl.searchParams.get('query')); // string
console.log(myUrl.hash); // #hash

OS Module

For operating system-related information and utilities:

const os = require('os');

console.log('CPU architecture:', os.arch());
console.log('Free memory:', os.freemem());
console.log('Total memory:', os.totalmem());
console.log('OS platform:', os.platform());
console.log('OS type:', os.type());
console.log('Uptime (seconds):', os.uptime());
console.log('User info:', os.userInfo());

Process Module

Provides information about, and control over, the current Node.js process:

// The 'process' object is global, no require needed

// Access environment variables
console.log('NODE_ENV:', process.env.NODE_ENV);

// Get command line arguments
console.log('Arguments:', process.argv);

// Current working directory
console.log('Current directory:', process.cwd());

// Exit the process
process.exit(0); // 0 indicates success, non-zero indicates failure

These modules, along with many others in the Node.js core, provide a comprehensive set of tools for building server-side applications without needing to rely heavily on external packages.

Best Practices for Using Core Modules

Performance Considerations

Use asynchronous methods for I/O operations to avoid blocking the event loop
Prefer streams for handling large files
Implement proper error handling for all operations
Consider caching frequently accessed file content or computation results

Security Considerations

Validate file paths to prevent directory traversal attacks
Sanitize user input before using it in file operations or URLs
Set appropriate file permissions when creating files
Use HTTPS instead of HTTP for sensitive data
Implement rate limiting for HTTP servers to prevent DoS attacks

Code Organization

Create utility modules for common operations
Use async/await for cleaner asynchronous code
Handle all potential errors to prevent crashes
Add appropriate logging for debugging and monitoring

Following these best practices will help you build more reliable, secure, and maintainable Node.js applications using core modules.

Practice Activities

Activity 1: File System Operations

Create a Node.js script that:

Creates a directory called "logs" if it doesn't exist
Creates a log file with the current timestamp as the name
Writes system information (using the os module) to the log file
Reads the directory and lists all log files
Reads and displays the content of the oldest log file

Activity 2: Path Manipulation Challenge

Write a function that:

Takes a file path as input
Normalizes the path across platforms
Creates a new path for a backup file (e.g., "file.txt" becomes "file.backup.txt")
Returns an object with the directory, original filename, and backup filename
Test with both absolute and relative paths

Activity 3: Mini Web Server

Create a simple HTTP server that:

Listens on port 3000
Handles at least 3 different routes ("/", "/about", "/api/time")
Returns HTML content for web routes
Returns JSON data for API routes
Includes proper status codes and content types
Handles 404 errors for undefined routes

Key Takeaways

Node.js core modules provide essential functionality without requiring external dependencies.
The fs module offers methods for interacting with the file system, with both synchronous and asynchronous options.
For large files, streams provide memory-efficient processing by handling data in chunks.
The path module helps manage file paths in a cross-platform way, abstracting away OS-specific differences.
The http and https modules enable creating web servers and making outgoing requests.
Combining these modules allows you to build sophisticated applications like file servers, APIs, and more.
Best practices include using asynchronous methods, implementing proper error handling, and considering security implications.

These core modules form the foundation of Node.js development and understanding them well will make you more effective when working with higher-level frameworks and libraries that build upon them.