HTTP Request/Response Cycle

Understanding the Foundation of Web Communication

Introduction to HTTP

HTTP (Hypertext Transfer Protocol) is the backbone of data communication on the web. It defines the format and rules for how messages are sent and received between clients (usually web browsers) and servers.

Since its creation in the early 1990s by Tim Berners-Lee, HTTP has evolved through several versions (HTTP/1.0, HTTP/1.1, HTTP/2, and HTTP/3), each bringing improvements in performance, security, and functionality.

As backend developers, understanding HTTP is essential because it's the primary way clients will communicate with your server-side code. Almost every web application you build will involve handling HTTP requests and constructing HTTP responses.

sequenceDiagram participant Client participant Server Client->>+Server: HTTP Request Note right of Server: Server processes request Server->>-Client: HTTP Response Note over Client,Server: The HTTP Request/Response Cycle

The Mail System Analogy

A helpful way to understand HTTP is to compare it to a postal mail system:

Just like postal mail, HTTP follows established formats and protocols to ensure that messages are properly delivered, processed, and responded to.

The HTTP Request in Detail

Anatomy of an HTTP Request

graph TB request[HTTP Request] --> A[Start Line] request --> B[Headers] request --> C[Empty Line] request --> D[Body optional] A --> A1[Method URL HTTP-Version] B --> B1[Name: Value] D --> D1[Request Data] style request fill:#f9d5e5,stroke:#333,stroke-width:2px

Start Line (Request Line)

The first line of an HTTP request contains three important pieces of information:

Request Headers

Headers provide additional information about the request or the client:

Request Body

Present in some requests (POST, PUT), it contains the data being sent to the server:

HTTP Methods

HTTP methods indicate the desired action to be performed on a resource:

Method Description Has Body Idempotent Safe
GET Retrieve a resource No Yes Yes
POST Create a new resource or submit data for processing Yes No No
PUT Update a resource by replacing it entirely Yes Yes No
PATCH Partially update a resource Yes No No
DELETE Remove a resource Sometimes Yes No
HEAD Same as GET but without response body (headers only) No Yes Yes
OPTIONS Describe communication options for a resource No Yes Yes

Important concepts:

HTTP Request Examples

Simple GET Request


GET /api/products HTTP/1.1
Host: example.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36
Accept: application/json

POST Request with JSON Data


POST /api/users HTTP/1.1
Host: example.com
Content-Type: application/json
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...
Content-Length: 84

{
  "name": "John Doe",
  "email": "john.doe@example.com",
  "password": "securePassword123"
}

PUT Request to Update a Resource


PUT /api/products/42 HTTP/1.1
Host: example.com
Content-Type: application/json
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...
Content-Length: 145

{
  "id": 42,
  "name": "Updated Product Name",
  "price": 99.99,
  "description": "This is an updated product description",
  "in_stock": true
}

The HTTP Response in Detail

Anatomy of an HTTP Response

graph TB response[HTTP Response] --> A[Status Line] response --> B[Headers] response --> C[Empty Line] response --> D[Body optional] A --> A1[HTTP-Version Status-Code Reason-Phrase] B --> B1[Name: Value] D --> D1[Response Data] style response fill:#b5ead7,stroke:#333,stroke-width:2px

Status Line

The first line of an HTTP response contains:

Response Headers

Headers provide metadata about the response:

Response Body

Contains the requested resource or data:

HTTP Status Codes

Status codes are grouped into five categories:

1xx: Informational

2xx: Success

3xx: Redirection

4xx: Client Errors

5xx: Server Errors

HTTP Response Examples

Successful Response with JSON Data


HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 165
Date: Mon, 15 Aug 2023 10:23:45 GMT
Cache-Control: max-age=3600

{
  "id": 42,
  "name": "Wireless Headphones",
  "price": 99.99,
  "description": "High-quality wireless headphones with noise cancellation",
  "in_stock": true,
  "categories": ["electronics", "audio"]
}

Redirection Response


HTTP/1.1 301 Moved Permanently
Location: https://example.com/new-location
Content-Length: 0
Date: Mon, 15 Aug 2023 10:24:35 GMT

Client Error Response


HTTP/1.1 404 Not Found
Content-Type: application/json
Content-Length: 49
Date: Mon, 15 Aug 2023 10:25:12 GMT

{
  "error": "Product with ID 999 could not be found"
}

Server Error Response


HTTP/1.1 500 Internal Server Error
Content-Type: application/json
Content-Length: 83
Date: Mon, 15 Aug 2023 10:26:05 GMT

{
  "error": "An unexpected error occurred while processing your request"
}

The Complete HTTP Request/Response Cycle

sequenceDiagram participant Browser participant DNS as DNS Server participant Server participant Database Browser->>DNS: 1. DNS Lookup (example.com) DNS->>Browser: Return IP address Browser->>+Server: 2. TCP Handshake Server->>-Browser: Connection established Browser->>+Server: 3. HTTP Request (GET /products) Server->>+Database: 4. Query for products Database->>-Server: Return product data Server->>-Browser: 5. HTTP Response (200 OK + data) Note over Browser: 6. Browser renders the page alt HTTPS is used Browser->>+Server: TLS Handshake before HTTP Server->>-Browser: Secure connection established end

Let's break down the complete cycle from when a user enters a URL until the page appears:

Step 1: DNS Resolution

Step 2: Connection Establishment

Step 3: HTTP Request

Step 4: Server Processing

Step 5: HTTP Response

Step 6: Client Processing

Step 7: Connection Handling

HTTP in Different Context Types

Traditional Web Pages

sequenceDiagram participant Browser participant Server Browser->>+Server: GET /page.html HTTP/1.1 Server->>-Browser: HTTP/1.1 200 OK
Content-Type: text/html Browser->>+Server: GET /styles.css HTTP/1.1 Server->>-Browser: HTTP/1.1 200 OK
Content-Type: text/css Browser->>+Server: GET /script.js HTTP/1.1 Server->>-Browser: HTTP/1.1 200 OK
Content-Type: application/javascript Browser->>+Server: GET /image.jpg HTTP/1.1 Server->>-Browser: HTTP/1.1 200 OK
Content-Type: image/jpeg

In traditional web pages:

AJAX and Single-Page Applications

sequenceDiagram participant Browser participant Server Browser->>+Server: GET /app.html HTTP/1.1 Server->>-Browser: HTTP/1.1 200 OK
Content-Type: text/html Note over Browser: User interacts with app Browser->>+Server: GET /api/data HTTP/1.1
Accept: application/json Server->>-Browser: HTTP/1.1 200 OK
Content-Type: application/json Note over Browser: App updates UI without
page reload Browser->>+Server: POST /api/items HTTP/1.1
Content-Type: application/json Server->>-Browser: HTTP/1.1 201 Created
Content-Type: application/json

In AJAX and SPA contexts:

RESTful APIs

sequenceDiagram participant Client participant API Client->>+API: GET /api/users HTTP/1.1 API->>-Client: HTTP/1.1 200 OK
[User list data] Client->>+API: GET /api/users/42 HTTP/1.1 API->>-Client: HTTP/1.1 200 OK
[Single user data] Client->>+API: POST /api/users HTTP/1.1
[New user data] API->>-Client: HTTP/1.1 201 Created
[Created user data] Client->>+API: PUT /api/users/42 HTTP/1.1
[Updated user data] API->>-Client: HTTP/1.1 200 OK
[Updated user data] Client->>+API: DELETE /api/users/42 HTTP/1.1 API->>-Client: HTTP/1.1 204 No Content

In RESTful API contexts:

Mobile Applications

In mobile app contexts:

HTTP Headers in Depth

HTTP headers play a critical role in controlling the behavior of requests and responses. Let's examine some important headers in detail:

Request Headers

General Headers (used in both requests and responses)

Request-Specific Headers

Response Headers

Response-Specific Headers

Security-Related Headers

Evolution of HTTP

timeline title Evolution of HTTP 1991 : HTTP/0.9 : The "original" HTTP 1996 : HTTP/1.0 : Headers, status codes, content-types 1997 : HTTP/1.1 : Persistent connections, pipelining 2015 : HTTP/2 : Multiplexing, server push, binary protocol 2022 : HTTP/3 : QUIC transport protocol, improved performance

HTTP/1.1

The stable workhorse of the web for over two decades:

HTTP/2

Major performance improvements:

HTTP/3

The latest evolution, built on QUIC:

Working with HTTP in Different Languages

As backend developers, you'll need to handle HTTP requests and responses in your code. Let's look at some examples in different languages:

Node.js with Express


// Simple Express server handling different HTTP methods
const express = require('express');
const app = express();
app.use(express.json());

// GET request handler
app.get('/api/users', (req, res) => {
  // Access query parameters
  const page = req.query.page || 1;
  const limit = req.query.limit || 10;
  
  // Access headers
  const acceptLanguage = req.headers['accept-language'];
  
  // Send response
  res.status(200).json({
    users: [{ id: 1, name: 'John' }, { id: 2, name: 'Jane' }],
    page: page,
    limit: limit,
    total: 100
  });
});

// POST request handler
app.post('/api/users', (req, res) => {
  // Access request body
  const { name, email } = req.body;
  
  // Validate input
  if (!name || !email) {
    return res.status(400).json({ error: 'Name and email are required' });
  }
  
  // Process the data (in a real app, you'd save to a database)
  const newUser = { id: 3, name, email };
  
  // Send response with status code 201 Created
  res.status(201).json(newUser);
});

// PUT request handler
app.put('/api/users/:id', (req, res) => {
  // Access route parameters
  const userId = req.params.id;
  const userData = req.body;
  
  // In a real app, you'd update the user in the database
  
  res.status(200).json({
    id: userId,
    ...userData,
    updated: true
  });
});

// DELETE request handler
app.delete('/api/users/:id', (req, res) => {
  // Access route parameters
  const userId = req.params.id;
  
  // In a real app, you'd delete the user from the database
  
  // Send 204 No Content response
  res.status(204).send();
});

app.listen(3000, () => {
  console.log('Server running on port 3000');
});

Python with Flask


from flask import Flask, request, jsonify

app = Flask(__name__)

@app.route('/api/users', methods=['GET'])
def get_users():
    # Access query parameters
    page = request.args.get('page', 1, type=int)
    limit = request.args.get('limit', 10, type=int)
    
    # Access headers
    accept_language = request.headers.get('Accept-Language')
    
    # Send response
    return jsonify({
        'users': [{'id': 1, 'name': 'John'}, {'id': 2, 'name': 'Jane'}],
        'page': page,
        'limit': limit,
        'total': 100
    }), 200  # Status code

@app.route('/api/users', methods=['POST'])
def create_user():
    # Access JSON body
    data = request.get_json()
    name = data.get('name')
    email = data.get('email')
    
    # Validate input
    if not name or not email:
        return jsonify({'error': 'Name and email are required'}), 400
    
    # Process the data (in a real app, you'd save to a database)
    new_user = {'id': 3, 'name': name, 'email': email}
    
    # Return with 201 Created status
    return jsonify(new_user), 201

@app.route('/api/users/', methods=['PUT'])
def update_user(user_id):
    # Access route parameter
    data = request.get_json()
    
    # In a real app, you'd update the user in the database
    
    # Return updated user
    user_data = {**data, 'id': user_id, 'updated': True}
    return jsonify(user_data), 200

@app.route('/api/users/', methods=['DELETE'])
def delete_user(user_id):
    # In a real app, you'd delete the user from the database
    
    # Return 204 No Content
    return '', 204

if __name__ == '__main__':
    app.run(debug=True)

PHP (Laravel-inspired)


<?php

// Simulate route handling for different HTTP methods
$method = $_SERVER['REQUEST_METHOD'];
$path = parse_url($_SERVER['REQUEST_URI'], PHP_URL_PATH);

// GET request handler
if ($method === 'GET' && $path === '/api/users') {
    // Access query parameters
    $page = $_GET['page'] ?? 1;
    $limit = $_GET['limit'] ?? 10;
    
    // Access headers
    $acceptLanguage = $_SERVER['HTTP_ACCEPT_LANGUAGE'] ?? '';
    
    // Set response headers
    header('Content-Type: application/json');
    http_response_code(200);
    
    // Send response
    echo json_encode([
        'users' => [
            ['id' => 1, 'name' => 'John'],
            ['id' => 2, 'name' => 'Jane']
        ],
        'page' => (int)$page,
        'limit' => (int)$limit,
        'total' => 100
    ]);
}

// POST request handler
else if ($method === 'POST' && $path === '/api/users') {
    // Get JSON body
    $data = json_decode(file_get_contents('php://input'), true);
    $name = $data['name'] ?? null;
    $email = $data['email'] ?? null;
    
    // Validate input
    if (!$name || !$email) {
        header('Content-Type: application/json');
        http_response_code(400);
        echo json_encode(['error' => 'Name and email are required']);
        exit;
    }
    
    // Process the data (in a real app, you'd save to a database)
    $newUser = ['id' => 3, 'name' => $name, 'email' => $email];
    
    // Send response
    header('Content-Type: application/json');
    http_response_code(201);
    echo json_encode($newUser);
}

// PUT request handler - assumes a route like /api/users/42
else if ($method === 'PUT' && preg_match('#^/api/users/(\d+)$#', $path, $matches)) {
    $userId = $matches[1];
    
    // Get JSON body
    $data = json_decode(file_get_contents('php://input'), true);
    
    // In a real app, you'd update the user in the database
    
    // Send response
    header('Content-Type: application/json');
    http_response_code(200);
    echo json_encode(array_merge($data, [
        'id' => (int)$userId,
        'updated' => true
    ]));
}

// DELETE request handler
else if ($method === 'DELETE' && preg_match('#^/api/users/(\d+)$#', $path, $matches)) {
    $userId = $matches[1];
    
    // In a real app, you'd delete the user from the database
    
    // Send 204 No Content response
    http_response_code(204);
}

else {
    // Route not found
    header('Content-Type: application/json');
    http_response_code(404);
    echo json_encode(['error' => 'Route not found']);
}

HTTP Best Practices for Backend Developers

RESTful API Design

Security Considerations

Performance Optimization

Versioning and Evolution

Practical Exercise: Analyzing HTTP Communication

Task: Decode and Analyze HTTP Messages

For this exercise, you'll practice interpreting HTTP requests and responses and understanding what's happening in each part.

Example 1: Analyze this HTTP Request


POST /api/login HTTP/1.1
Host: example.com
Content-Type: application/json
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64)
Content-Length: 56

{
  "username": "john.doe",
  "password": "secretpassword123"
}

Questions:

  1. What operation is being performed?
  2. What data format is being sent?
  3. What is the likely purpose of this request?
  4. Are there any security concerns with this request?

Example 2: Match Requests with Appropriate Responses

Given this request:


GET /api/products?category=electronics&sort=price HTTP/1.1
Host: example.com
Accept: application/json

Which of these responses is most appropriate?

Response A:


HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 256

{
  "products": [
    { "id": 101, "name": "Wireless Earbuds", "price": 29.99, "category": "electronics" },
    { "id": 102, "name": "USB Cable", "price": 9.99, "category": "electronics" },
    { "id": 103, "name": "Smartphone", "price": 499.99, "category": "electronics" }
  ],
  "total": 3
}

Response B:


HTTP/1.1 201 Created
Content-Type: application/json
Content-Length: 85

{
  "id": 104,
  "name": "New Product",
  "price": 199.99,
  "category": "electronics"
}

Response C:


HTTP/1.1 405 Method Not Allowed
Content-Type: application/json
Content-Length: 49
Allow: POST

{
  "error": "GET method not allowed for this endpoint"
}

Example 3: Create a Complete HTTP Cycle

Design a complete HTTP request and response cycle for a user updating their profile information. Include:

  1. The appropriate HTTP request with all necessary components
  2. A successful response
  3. A response for when validation fails
  4. A response for when the user is not authenticated

Summary

In this session, we've explored the HTTP request/response cycle in detail:

Understanding HTTP is fundamental to backend development as it forms the primary interface between clients and your server-side code. In upcoming sessions, we'll build on this foundation to implement specific backend functionality using Node.js, Python, and PHP.

Additional Resources