API Design Best Practices

Introduction to API Design

Designing a good API is similar to designing a good user interface—but for developers rather than end users. While REST provides the architectural foundation, effective API design requires additional consideration of user experience, consistency, and practicality.

A well-designed API is:

• Easy to learn and use - Follows conventions and has intuitive behavior
• Hard to misuse - Prevents common errors and provides clear feedback
• Complete enough for the use case - Addresses real developer needs
• Consistent - Uses predictable patterns throughout
• Evolvable - Can adapt to changing requirements over time

In this lecture, we'll explore concrete best practices for designing APIs that embody these qualities, focusing on both RESTful principles and practical considerations from real-world implementations.

Resource Design

Effective resource design is the foundation of a good API. It involves identifying the right resources, naming them clearly, and structuring their relationships appropriately.

Resource Identification

Start by identifying the key entities in your domain model. Good candidates for resources typically include:

• Business objects (users, products, orders, etc.)
• Collections of objects
• Non-object resources that represent important concepts

When identifying resources, consider:

• What domain objects do users interact with?
• What operations do users need to perform?
• What resources would make the API intuitive to understand?

For an e-commerce system, core resources might include:

/products
/customers
/orders
/payments
/reviews
/categories
/inventory
            

Resource Naming Best Practices

Clear, consistent resource naming is crucial for a usable API:

• Use nouns, not verbs for resource names
• Use plural nouns for collection resources
• Use concrete names rather than abstract concepts
• Be consistent with naming patterns across the API
• Use lower-case with hyphens or camelCase (pick one and stick with it)
• Avoid abbreviations unless they're universally understood

Resource Hierarchies

Organize resources hierarchically when they have strong parent-child relationships. This helps express containment and ownership clearly.

# Parent-child relationships
/users/123/orders                # All orders for user 123
/orders/456/items                # All items in order 456
/products/789/reviews            # All reviews for product 789

# Avoid excessive nesting
/users/123/orders/456/items/789  # Too deep, prefer /orders/456/items/789
            

As a rule of thumb, limit nesting to a maximum of 2-3 levels. Beyond that, consider flattening with query parameters or separate endpoints.

Collection and Singleton Resources

APIs typically have two types of resources:

• Collection resources - Represent multiple instances (e.g., /users)
• Singleton resources - Represent a single instance (e.g., /users/123)

Some resources might also be singletons within a parent context:

/users/current                   # Current authenticated user
/users/123/profile               # Profile of user 123 (only one per user)
/system/status                   # System status information
            

Custom Resource Types

Some operations don't fit neatly into the standard CRUD model. In these cases, consider creating specialized resource types:

# Controller resources (for operations that don't map to standard CRUD)
/search                          # Search across multiple resource types
/calculator                      # Perform calculations

# Process resources (for long-running operations)
/jobs                            # Background processing jobs
/imports                         # Data import processes

# Virtual resources (don't map directly to backend entities)
/dashboard                       # Combined stats and insights
/suggestions                     # Personalized recommendations
            

These resources still follow RESTful principles, even if they represent operations or virtual concepts rather than traditional database entities.

HTTP Methods & Operations

HTTP methods define the operations that can be performed on resources. Using them consistently is key to a clear and predictable API.

Standard CRUD Operations

For most resources, implement these standard operations:

PUT vs. PATCH

Understanding the distinction between PUT and PATCH is important:

• PUT is for complete replacement of a resource. All fields must be specified, even unchanged ones. Omitted fields are typically set to default values or null.
• PATCH is for partial updates to a resource. Only specify the fields you want to change. Omitted fields remain unchanged.

# Original resource
{
  "name": "John Smith",
  "email": "john@example.com",
  "role": "user",
  "status": "active"
}

# PUT request (complete replacement)
PUT /users/123
{
  "name": "John Smith",
  "email": "new.email@example.com",
  "role": "user"
  # Note: missing 'status' field might be set to null or default value
}

# PATCH request (partial update)
PATCH /users/123
{
  "email": "new.email@example.com"
  # Note: other fields remain unchanged
}
            

Bulk Operations

APIs often need to support operations on multiple resources at once:

# Bulk create
POST /users/bulk
{
  "users": [
    { "name": "User 1", "email": "user1@example.com" },
    { "name": "User 2", "email": "user2@example.com" }
  ]
}

# Bulk update
PATCH /users/bulk
{
  "ids": [123, 456, 789],
  "changes": { "status": "inactive" }
}

# Bulk delete
DELETE /users?ids=123,456,789
            

For bulk operations, prioritize consistency, clarity, and error handling. Decide whether the operation should be atomic (all succeed or all fail) or partial (return which succeeded and which failed).

Complex Operations

Some operations don't map cleanly to CRUD. Here are patterns for handling them:

Each pattern has its place. Choose based on:

• Whether the operation impacts a specific resource or is system-wide
• Whether the operation changes resource state or performs a side effect
• What approach would be most intuitive for API consumers

Query Parameters and Filtering

Query parameters allow clients to modify the behavior of resource requests, such as filtering, sorting, and pagination.

Common Query Parameter Uses

Filtering Strategies

Consider these approaches to filtering, from simple to complex:

# Basic field equality
/products?category=electronics&in_stock=true

# Operators for numeric fields
/products?price_min=100&price_max=500

# Dates and time ranges
/orders?created_after=2025-01-01&created_before=2025-02-01

# Complex filtering with a query language
/products?filter=category:electronics AND (price<500 OR price>1000)
            

For advanced filtering, consider:

• Using standard operators (gt, lt, eq, ne, etc.)
• Supporting multiple values (e.g., status=active,pending)
• Establishing conventions for range filters
• Implementing a simple query mini-language for complex queries

Pagination Approaches

Proper pagination is essential for handling large data sets efficiently. Common approaches include:

Offset-Based Pagination

/users?page=2&per_page=20
/users?offset=40&limit=20
            

Advantages: Simple to implement, familiar to developers
Disadvantages: Performance issues with large offsets, inconsistent results with data changes

Cursor-Based Pagination

/users?after=eyJpZCI6MTIzNDV9&limit=20
/users?cursor=eyJpZCI6MTIzNDV9&limit=20
            

Advantages: Better performance, consistent results with data changes
Disadvantages: More complex to implement, cannot jump to arbitrary pages

Keyset Pagination

/users?after_id=12345&limit=20
/orders?after_date=2025-03-15T14:30:00Z&limit=20
            

Advantages: Good performance, simple to implement and use
Disadvantages: Requires a stable sorting key, doesn't work well with complex sorts

Include pagination metadata in responses to help clients navigate:

{
  "data": [...],
  "pagination": {
    "total_count": 573,
    "page": 2,
    "per_page": 20,
    "total_pages": 29,
    "links": {
      "first": "/users?page=1&per_page=20",
      "prev": "/users?page=1&per_page=20",
      "next": "/users?page=3&per_page=20",
      "last": "/users?page=29&per_page=20"
    }
  }
}
            

Sorting

Provide flexible sorting options to meet varied client needs:

# Basic single-field sorting
/users?sort=last_name

# Multiple fields with directions
/users?sort=status:asc,created_at:desc

# Default sorting direction
/products?sort=price  # Assumes ascending

# Alias common sort patterns
/articles?sort=newest  # Equivalent to created_at:desc
            

Document your sorting options clearly, including:

• Available sort fields
• Default sort order if not specified
• How to specify ascending vs. descending
• Any predefined sort aliases

Response Design and Status Codes

Well-designed responses make your API easy to use and understand. They should be consistent, informative, and follow established patterns.

Response Format Consistency

Maintain a consistent response structure across your API:

# Single resource response
{
  "id": 123,
  "name": "Product Name",
  "price": 99.99,
  "created_at": "2025-03-15T14:30:00Z",
  "updated_at": "2025-03-20T09:15:00Z"
}

# Collection response
{
  "data": [
    { "id": 123, "name": "Product 1", ... },
    { "id": 124, "name": "Product 2", ... }
  ],
  "pagination": {
    "total": 57,
    "page": 1,
    "per_page": 20
  }
}

# Error response
{
  "error": {
    "code": "validation_error",
    "message": "Invalid input data",
    "details": [
      { "field": "email", "message": "Must be a valid email address" },
      { "field": "age", "message": "Must be at least 18" }
    ]
  }
}
            

Consider wrapping success responses in a common envelope for consistency:

# Success response envelope
{
  "data": {
    "id": 123,
    "name": "Product Name",
    ...
  },
  "meta": {
    "request_id": "a1b2c3d4",
    "timestamp": "2025-03-25T12:34:56Z"
  }
}
            

However, be careful not to add unnecessary nesting. Some APIs prefer to keep single resource responses flat for simplicity.

HTTP Status Codes

Use HTTP status codes consistently and appropriately. Focus on a core set of codes for common scenarios:

For a more detailed approach, consider using additional status codes for specific scenarios:

• 409 Conflict - The request conflicts with the current state (e.g., version conflicts)
• 410 Gone - The resource existed but was permanently removed
• 412 Precondition Failed - Condition in request headers not met (e.g., If-Match)
• 413 Payload Too Large - Request body exceeds size limits
• 415 Unsupported Media Type - Content type in request not supported
• 503 Service Unavailable - Server temporarily unavailable (maintenance, overload)

Error Response Design

Error responses should be consistent, informative, and actionable:

# Basic error response
{
  "error": {
    "code": "resource_not_found",
    "message": "The requested resource was not found",
    "details": "No user exists with ID 12345"
  }
}

# Validation error with field-level details
{
  "error": {
    "code": "validation_error",
    "message": "The request contains invalid parameters",
    "details": [
      {
        "field": "email",
        "code": "invalid_format",
        "message": "Must be a valid email address"
      },
      {
        "field": "password",
        "code": "too_short",
        "message": "Must be at least 8 characters long"
      }
    ]
  }
}
            

Include these elements in error responses:

• Machine-readable error code - For programmatic handling
• Human-readable message - For debugging and displaying to users
• Detailed information - As appropriate for the error type
• Request identifier - For correlating with server logs
• Link to documentation - For complex errors (optional)

Success Response Patterns

For different operations, return appropriate content in success responses:

Versioning and Evolution

APIs evolve over time. Effective versioning strategies help maintain backward compatibility while allowing for improvements.

Versioning Strategies

Several approaches to API versioning exist, each with advantages and disadvantages:

URL path versioning is the most common approach due to its simplicity and visibility, despite not being the most technically pure RESTful solution.

Versioning Guidelines

• Don't version unless necessary - Only introduce a new version for breaking changes
• Version the entire API - Avoid versioning individual endpoints when possible
• Support multiple versions - Allow for transition periods when both old and new versions work
• Document version differences - Clearly explain what changed between versions
• Set deprecation timelines - Give clients time to migrate to newer versions

Non-Breaking Changes

Many changes can be made without breaking backward compatibility:

• Adding new resources - Entirely new endpoints
• Adding new optional request fields - As long as missing them doesn't change behavior
• Adding new response fields - Clients should ignore fields they don't understand
• Adding new optional query parameters - With sensible defaults
• Relaxing validation rules - Accepting formats you previously rejected
• Adding new operations - New HTTP methods on existing resources

Breaking Changes

These changes typically require a new version:

• Removing or renaming fields - Breaks clients expecting those fields
• Changing field types - E.g., changing a string to a number
• Adding required request fields - Breaks existing clients not sending them
• Changing resource URL structure - Breaks clients with hardcoded URLs
• Changing error codes - Breaks error handling logic
• Removing operations - Breaks clients using those operations
• Changing authentication methods - Breaks existing auth flow

API Evolution Best Practices

To minimize the need for versioning and make your API more resilient to change:

• Use semantic versioning (MAJOR.MINOR.PATCH) for your API
• Design for extensibility from the beginning
• Support feature detection through capabilities endpoint
• Use enums instead of booleans for fields that might get additional states
• Implement graceful degradation when clients use deprecated features
• Use hypermedia controls (HATEOAS) to decouple clients from resource URLs
• Make clients robust to change by ignoring unknown fields

Deprecation Process

When you need to phase out part of your API:

1. Announce deprecation well in advance (typically 6-12 months)
2. Document alternatives for deprecated functionality
3. Include deprecation notices in responses using a custom header:

   Deprecation: true
   Sunset: Sat, 31 Dec 2025 23:59:59 GMT
   Link: <https://api.example.com/v2/users>; rel="successor-version"

4. Track usage of deprecated features to identify affected clients
5. Contact users of deprecated features directly if possible
6. Maintain the deprecated version until the sunset date

Security Best Practices

Security is a critical aspect of API design that can't be bolted on as an afterthought.

Authentication & Authorization

Implement robust authentication and authorization mechanisms:

• OAuth 2.0 for delegated authorization and API access
• JWT (JSON Web Tokens) for stateless authentication
• API Keys for simple machine-to-machine communication
• Scopes to limit permissions of access tokens
• Role-Based Access Control (RBAC) for fine-grained permissions

Always use HTTPS for all API endpoints, with no exceptions for "public" data.

Input Validation

Thoroughly validate all client input to prevent injection attacks and other security issues:

• Validate data types (strings, numbers, booleans, etc.)
• Validate ranges and lengths for numeric fields and strings
• Validate formats for emails, URLs, and other structured data
• Whitelist allowed values for enumerated fields
• Validate nested objects and arrays, including their depth
• Set reasonable size limits for request payloads

Rate Limiting

Implement rate limiting to protect your API from abuse, DoS attacks, and excessive use:

# Rate limit headers in response
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 87
X-RateLimit-Reset: 1616723058

# When limit is exceeded
HTTP/1.1 429 Too Many Requests
Retry-After: 60
            

Consider different rate limit tiers based on:

• Authentication status (anonymous vs. authenticated)
• User role or subscription level (free vs. premium)
• Resource sensitivity (higher limits for read-only operations)
• Client impact (stricter limits for expensive operations)

Error Disclosure

Be careful about what information you disclose in error responses:

• Don't expose sensitive data in error messages
• Don't reveal implementation details that could help attackers
• Use generic error messages for security-related failures (e.g., authentication)
• Log detailed error information server-side instead
• Include request IDs to correlate user reports with internal logs

Additional Security Headers

Include security-related HTTP headers in API responses:

Strict-Transport-Security: max-age=31536000; includeSubDomains
Content-Security-Policy: default-src 'none'
X-Content-Type-Options: nosniff
Cache-Control: no-store
Pragma: no-cache
            

Documentation

Great API documentation is essential for developer adoption and effective use of your API.

Documentation Components

Comprehensive API documentation should include:

• Getting started guide - How to quickly begin using the API
• Authentication information - How to obtain and use credentials
• Resource descriptions - All available endpoints and their purpose
• Request and response formats - With examples
• Query parameters - What's available and how to use them
• Error codes and handling - All possible errors and their meaning
• Rate limits and quotas - Usage restrictions and how they're applied
• Code examples - In multiple languages if possible
• Changelog - History of API changes and versions

Documentation Formats

Consider these documentation approaches:

• OpenAPI/Swagger - Machine-readable API specification that can generate interactive documentation
• API Blueprint - Markdown-based format for API descriptions
• RAML - YAML-based language for API definition
• GraphQL Schema - For GraphQL APIs, the schema itself serves as documentation
• Custom documentation - Hand-crafted documentation specific to your API

OpenAPI has become the de facto standard for RESTful API documentation due to its widespread adoption and rich tooling ecosystem.

Interactive Documentation

Interactive API explorers like Swagger UI allow developers to:

• Read documentation for each endpoint
• See all available parameters and their descriptions
• View request and response schemas
• Make test API calls directly from the browser
• Authenticate and use their own credentials
• See actual responses from the API

This "try it out" functionality dramatically improves developer experience and reduces onboarding time.

Documentation as Code

Treat API documentation as code by:

• Storing documentation in version control alongside the API code
• Automating documentation updates as part of your build process
• Testing documentation examples to ensure they work
• Versioning documentation alongside API versions
• Using code generation to create client libraries from API specs

This approach ensures documentation stays accurate and up-to-date as the API evolves.

Performance Considerations

Performance directly impacts the usability and adoption of your API.

Request Efficiency

Design your API to minimize the number of requests needed to accomplish common tasks:

• Field selection - Allow clients to request only the fields they need
• Embedding related resources - Include related data to avoid multiple requests
• Bulk operations - Support operations on multiple resources at once
• Filtering capabilities - Enable server-side filtering to reduce data transfer

# Field selection
GET /users?fields=id,name,email

# Embedding related resources
GET /orders?embed=customer,items

# Bulk operations
POST /users/bulk
PATCH /products?ids=1,2,3
            

Caching

Implement proper HTTP caching to reduce load and improve response times:

# Example cache headers
Cache-Control: max-age=3600, public
ETag: "33a64df551425fcc55e4d42a148795d9f25f89d4"
Last-Modified: Wed, 15 Mar 2025 14:30:00 GMT
            

Consider these caching strategies:

• ETags for content-based validation
• Last-Modified for time-based validation
• Cache-Control for cache behavior directives
• Vary header to handle different representations
• Conditional requests (If-None-Match, If-Modified-Since)

Pagination Performance

Choose pagination strategies that scale well with large datasets:

• Avoid count queries for total items if possible (they get expensive)
• Use cursor-based pagination for large datasets (instead of offset)
• Support reasonable page sizes with sensible defaults
• Enforce maximum page sizes to prevent abuse
• Use keyset pagination for time-ordered data

Asynchronous Operations

For long-running operations, consider asynchronous processing patterns:

# Create a job for a long operation
POST /import-jobs
{
  "file_url": "https://example.com/data.csv",
  "options": { "update_existing": true }
}

# Response with job resource
HTTP/1.1 202 Accepted
Location: /import-jobs/12345

# Check job status
GET /import-jobs/12345

# Job status response
{
  "id": "12345",
  "status": "processing",
  "progress": 60,
  "created_at": "2025-03-15T14:30:00Z",
  "estimated_completion_time": "2025-03-15T14:35:00Z"
}
            

This pattern allows clients to start operations and check progress without keeping connections open for long periods.

Case Study: Building a RESTful E-commerce API

Let's apply these best practices to design an e-commerce API.

Resource Design

Core resources for an e-commerce API:

# Primary resources
/products            # Product catalog
/categories          # Product categories
/customers           # Customer accounts
/orders              # Customer orders
/cart                # Shopping cart
/payments            # Payment transactions
/shipping-methods    # Available shipping options
/reviews             # Product reviews

# Sub-resources
/products/{id}/variants          # Product variations
/orders/{id}/items               # Line items in an order
/customers/{id}/addresses        # Customer addresses
/customers/{id}/payment-methods  # Saved payment methods

# Specialized resources
/search              # Product search
/recommendations     # Personalized product recommendations
/inventory           # Stock levels
/webhooks            # Event notifications
            

Common Operations

Example Requests and Responses

# Get product with related data
GET /products/123?fields=id,name,price,stock,variants&embed=reviews,categories

# Response
{
  "id": 123,
  "name": "Ergonomic Office Chair",
  "price": 299.99,
  "stock": 15,
  "variants": [
    {
      "id": 456,
      "name": "Black",
      "price_modifier": 0
    },
    {
      "id": 457,
      "name": "Red",
      "price_modifier": 20
    }
  ],
  "reviews": [
    {
      "id": 789,
      "rating": 5,
      "comment": "Very comfortable!",
      "author": "John D."
    }
  ],
  "categories": [
    {
      "id": 22,
      "name": "Office Furniture"
    }
  ]
}

# Add item to cart
POST /cart/items
{
  "product_id": 123,
  "variant_id": 456,
  "quantity": 2
}

# Response
{
  "cart": {
    "id": "c789",
    "items": [
      {
        "id": "ci456",
        "product_id": 123,
        "variant_id": 456,
        "name": "Ergonomic Office Chair - Black",
        "quantity": 2,
        "unit_price": 299.99,
        "total_price": 599.98
      }
    ],
    "subtotal": 599.98,
    "tax": 48.00,
    "shipping": 0,
    "total": 647.98,
    "_links": {
      "self": { "href": "/cart" },
      "checkout": { "href": "/checkout" }
    }
  }
}
            

Error Scenarios

# Product not found
GET /products/999
HTTP/1.1 404 Not Found
{
  "error": {
    "code": "resource_not_found",
    "message": "The requested product could not be found",
    "request_id": "req_abcd1234"
  }
}

# Add to cart with insufficient stock
POST /cart/items
{
  "product_id": 123,
  "quantity": 100
}

HTTP/1.1 422 Unprocessable Entity
{
  "error": {
    "code": "insufficient_stock",
    "message": "The requested quantity exceeds available stock",
    "details": {
      "available": 15,
      "requested": 100
    }
  }
}
            

Complex Operations

For the checkout process, we might implement a workflow using multiple endpoints:

1. Review cart: GET /cart
2. Set shipping address: PUT /cart/shipping-address
3. Get shipping options: GET /shipping-methods?destination={shipping_address_id}
4. Select shipping method: PUT /cart/shipping-method
5. Create order (reserves inventory): POST /orders
6. Process payment: POST /payments
7. Confirm order: POST /orders/{id}/confirm

This multi-step process allows for flexibility while maintaining RESTful principles.

Practice Activities

Activity 1: Resource Design

Design resources for a blog API that supports these features:

• Posts with categories and tags
• Authors with profiles
• Comments on posts
• Likes/reactions to posts
• Media uploads (images for posts)
• Subscriptions to authors or topics
• Search functionality

For each resource, define:

• The resource name and URL
• Available operations (HTTP methods)
• Relationships to other resources
• Any query parameters for filtering/sorting

Activity 2: Response Design

Create response formats for the following API scenarios:

1. Successfully creating a new user account
2. Validation error when creating a user account
3. Listing orders with pagination
4. Getting a single product with related products
5. Authentication failure
6. Rate limit exceeded

Include appropriate status codes, headers, and response bodies for each scenario.

Activity 3: API Evolution

Consider an existing API endpoint for user profiles:

GET /users/123
{
  "id": 123,
  "username": "johndoe",
  "email": "john@example.com",
  "name": "John Doe",
  "created_at": "2025-01-15T12:00:00Z"
}
            

Design how you would implement these changes:

1. Adding user preferences (non-breaking change)
2. Splitting "name" into "first_name" and "last_name" (breaking change)
3. Adding user roles and permissions (mixed changes)
4. Supporting both old and new formats during a transition period

Explain your versioning strategy and how you would communicate these changes to API consumers.

Summary

In this lecture, we've explored comprehensive best practices for designing effective RESTful APIs:

• Resource Design - Identifying, naming, and structuring API resources
• HTTP Methods & Operations - Using HTTP verbs appropriately for CRUD and complex operations
• Query Parameters - Implementing filtering, sorting, and pagination
• Response Design - Creating consistent and informative responses
• Status Codes - Using HTTP status codes properly
• Error Handling - Designing helpful error messages
• Versioning - Strategies for API evolution
• Security - Best practices for authentication, validation, and protection
• Documentation - Creating comprehensive API documentation
• Performance - Optimizing API efficiency

By following these practices, you can create APIs that are intuitive, stable, and maintainable—providing an excellent developer experience while meeting your system's technical requirements.

Further Reading

• Zalando RESTful API Guidelines - Comprehensive API design rules
• Microsoft REST API Guidelines - Microsoft's approach to RESTful design
• Best Practices for a Pragmatic RESTful API - Practical advice for real-world APIs
• OpenAPI Specification - Standard for API documentation
• JSON:API - Specification for building APIs in JSON
• The API Stylebook - Collection of API design guidelines