🛠️ Weekend Project: Asynchronous Javascript

The Project Brief

Your mission: build a real-time data dashboard that streams live financial-style data, processes it on the fly, and renders it into auto-updating widgets. It's the perfect capstone for this module because a dashboard is nothing but asynchronous work — many slow, unreliable sources feeding a UI that has to stay smooth.

You don't need paid market-data APIs to do this. Every source can be replaced with a small mock async generator that yields random walks on a timer — the async plumbing is identical, and the plumbing is the point. Here's the shape every source will share:

// The one contract every data source implements.
// Consumers never care whether it's REST, WebSocket, or SSE underneath.
async function* stream() {
  while (running) {
    yield { timestamp: Date.now(), data: /* one update */ };
  }
}

A Plan of Attack: Polya's Method

Before writing code, borrow a framework from mathematician George Pólya's 1945 classic How to Solve It. His four steps map perfectly onto software and keep you from the classic trap of coding before you understand the problem.

Step 1 — Understand the problem

Write down the answers before you touch an editor:

• What must it do? Show live prices, news, and indicators that update without a page refresh.
• What are the inputs? Three sources with three different protocols and three different failure modes.
• What are the hard constraints? The UI must stay responsive, memory must not grow forever, and one dead source must not take down the others.

Steps 2–4 — plan, build, reflect

The rest of this lesson is steps 2 through 4: we devise a plan (an architecture plus four milestones), carry it out (the code for each milestone), and look back (the checklist and "what good looks like"). Work the milestones in order — each one leaves you with a program you can actually run.

Architecture & Milestones

Keep the layers strictly separated. Data sources know nothing about charts; charts know nothing about WebSockets. A thin controller wires them together. That separation is what lets you swap a mock source for a real API later without touching the UI.

The four milestones

Ship each milestone before starting the next. If you run out of weekend, a working dashboard with one source beats a half-written one with three.

Milestone 1 — Async Utilities

Every resilient app rests on a few reusable async helpers. Build these first and the rest of the project gets dramatically shorter. Put them in utils/async.js.

Retry with exponential backoff and jitter

Networks fail transiently. Instead of giving up on the first error, retry a few times with a delay that grows each attempt — and add a little randomness (jitter) so a fleet of clients doesn't retry in lockstep and stampede the server.

// utils/async.js

/**
 * Retry an async function with exponential backoff + jitter.
 * @param {(attempt: number) => Promise<T>} fn
 * @param {{ retries?: number, baseMs?: number, maxMs?: number,
 *           shouldRetry?: (err: unknown) => boolean }} [opts]
 */
export async function retry(fn, opts = {}) {
  const { retries = 3, baseMs = 500, maxMs = 15000, shouldRetry = () => true } = opts;

  for (let attempt = 0; ; attempt++) {
    try {
      return await fn(attempt);
    } catch (err) {
      if (attempt >= retries || !shouldRetry(err)) throw err;
      const backoff = Math.min(maxMs, baseMs * 2 ** attempt);
      const jitter = backoff * (0.5 + Math.random() * 0.5); // 50–100% of backoff
      await delay(jitter);
    }
  }
}

/** Promise-based sleep. */
export const delay = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

Timeouts the modern way: AbortSignal.timeout

The old pattern raced a promise against a setTimeout. Modern browsers and Node give you AbortSignal.timeout(ms), which produces a signal that aborts itself — pass it straight to fetch and you cancel the actual request, not just stop waiting for it.

/**
 * fetch() that aborts if it takes longer than `ms`, honoring an
 * optional outer signal too (so a global Stop can cancel it).
 */
export async function fetchWithTimeout(url, { ms = 8000, signal, ...init } = {}) {
  const timeout = AbortSignal.timeout(ms);
  // Combine the caller's signal with our timeout signal.
  const combined = signal ? AbortSignal.any([signal, timeout]) : timeout;
  return fetch(url, { ...init, signal: combined });
}

Limiting concurrency

Firing 500 requests at once will get you rate-limited or will exhaust sockets. A concurrency limiter runs at most N tasks at a time and, crucially, never rejects — it returns a settled result per task so one failure doesn't sink the batch.

/**
 * Run async task factories with at most `limit` in flight.
 * Returns results in input order, each { status, value | reason }.
 */
export async function mapLimit(items, limit, task) {
  const results = new Array(items.length);
  let next = 0;

  async function worker() {
    while (next < items.length) {
      const i = next++;
      try {
        results[i] = { status: 'fulfilled', value: await task(items[i], i) };
      } catch (reason) {
        results[i] = { status: 'rejected', reason };
      }
    }
  }

  // Spin up `limit` workers that pull from the shared cursor.
  const size = Math.min(limit, items.length);
  await Promise.all(Array.from({ length: size }, worker));
  return results;
}

Milestone 2 — Sources as Async Generators

The key insight of the whole project: any source of values over time can be exposed as an async generator. Once it is, consuming it is just a for await…of loop, regardless of whether the data arrived by polling, a socket, or SSE.

A polling REST source

Polling is a loop: fetch, yield, wait, repeat — wrapped in retry so a blip doesn't end the stream, and watching an AbortSignal so it can be stopped.

// data/sources.js
import { retry, fetchWithTimeout, delay } from '../utils/async.js';

/** Poll a JSON endpoint every `intervalMs`, yielding each response. */
export async function* pollingSource(url, { intervalMs = 5000, signal } = {}) {
  while (!signal?.aborted) {
    try {
      const data = await retry(
        () => fetchWithTimeout(url, { signal }).then((r) => {
          if (!r.ok) throw new ApiError(`HTTP ${r.status}`, { status: r.status });
          return r.json();
        }),
        { retries: 2, shouldRetry: (e) => e.retryable !== false },
      );
      yield { timestamp: Date.now(), data };
    } catch (err) {
      // Surface the error to the consumer instead of throwing away the stream.
      yield { timestamp: Date.now(), error: err };
    }
    await delay(intervalMs);
  }
}

Push sources: turning events into an async iterable

WebSocket and SSE push data at you via event listeners. To make them iterable you bridge events into a queue that the generator drains — resolving a pending promise when a consumer is waiting, otherwise buffering. Here's the pattern once; a WebSocket and an EventSource both use it.

/** Bridge an EventTarget's 'message' events into an async generator. */
export async function* socketSource(url, { signal } = {}) {
  const socket = new WebSocket(url);
  const queue = [];
  let notify;                     // resolver for the "next message" promise
  let closed = false;

  const push = (value) => {
    if (notify) { notify(value); notify = null; }
    else queue.push(value);
  };

  socket.addEventListener('message', (e) =>
    push({ timestamp: Date.now(), data: JSON.parse(e.data) }));
  socket.addEventListener('close', () => { closed = true; push(null); });
  socket.addEventListener('error', () => push({ timestamp: Date.now(), error: new NetworkError('socket error') }));
  signal?.addEventListener('abort', () => socket.close());

  try {
    while (!closed && !signal?.aborted) {
      const msg = queue.length ? queue.shift()
                               : await new Promise((res) => { notify = res; });
      if (msg === null) break;    // socket closed
      yield msg;
    }
  } finally {
    socket.close();               // always clean up, even on early break/throw
  }
}

Testing without any real API

Prove the plumbing with a mock that random-walks a price. It has the exact same signature, so your dashboard can't tell it from the real thing.

/** A fake price feed: a random walk emitted every `intervalMs`. */
export async function* mockPriceSource({ start = 100, intervalMs = 1000, signal } = {}) {
  let price = start;
  while (!signal?.aborted) {
    price = Math.max(1, price + (Math.random() - 0.5) * 2);
    yield { timestamp: Date.now(), data: { symbol: 'AAPL', price: +price.toFixed(2) } };
    await delay(intervalMs);
  }
}

// Milestone 2 smoke test:
const ac = new AbortController();
setTimeout(() => ac.abort(), 5000);        // stop after 5s
for await (const tick of mockPriceSource({ signal: ac.signal })) {
  console.log(tick.data.price);
}

Milestone 3 — The Processing Pipeline

Raw ticks are rarely what a widget wants. A pipeline is itself an async generator that wraps a source and applies a sequence of transforms — filtering symbols, mapping shapes, computing indicators — yielding the finished result. Because generators are lazy, this adds essentially no overhead and no intermediate arrays.

// data/pipeline.js

/**
 * Compose a source generator with an ordered list of async transforms.
 * A transform returns a value to keep, or `undefined` to drop the item.
 */
export async function* pipeline(source, transforms) {
  for await (const item of source) {
    if (item.error) { yield item; continue; }   // pass errors straight through
    let value = item;
    for (const transform of transforms) {
      value = await transform(value);
      if (value === undefined) break;            // dropped by a filter
    }
    if (value !== undefined) yield value;
  }
}

// Reusable transforms — small, pure, testable.
export const keepSymbol = (sym) => (item) =>
  item.data.symbol === sym ? item : undefined;

export const pick = (...fields) => (item) => ({
  ...item,
  data: Object.fromEntries(fields.map((f) => [f, item.data[f]])),
});

A stateful transform: simple moving average

Indicators need memory of past values. A closure that keeps a sliding window is the cleanest way to add state to an otherwise stateless pipeline.

/** Attach a simple moving average of `data[field]` over the last `period` ticks. */
export function movingAverage(field, period) {
  const window = [];
  return (item) => {
    window.push(item.data[field]);
    if (window.length > period) window.shift();
    const sma = window.length === period
      ? window.reduce((a, b) => a + b, 0) / period
      : null;                      // not enough data yet
    return { ...item, data: { ...item.data, sma } };
  };
}

// Compose it all:
const stream = pipeline(mockPriceSource({ signal }), [
  keepSymbol('AAPL'),
  movingAverage('price', 20),
]);
for await (const tick of stream) {
  console.log(tick.data.price, '→ SMA:', tick.data.sma);
}

Milestone 4 — Wiring the Dashboard

The controller is deliberately thin. It owns a single AbortController, starts one consumer loop per widget, and — the part that ties the whole module together — stops everything at once.

// DashboardController.js
export class DashboardController {
  #ac = new AbortController();
  #tasks = [];

  constructor(widgets) {
    this.widgets = widgets;       // [{ id, source, transforms, render }]
  }

  start() {
    const { signal } = this.#ac;
    // Each widget consumes its own stream; failures are isolated per widget.
    this.#tasks = this.widgets.map((w) => this.#run(w, signal));
  }

  async #run(widget, signal) {
    const stream = pipeline(widget.source(signal), widget.transforms);
    try {
      for await (const item of stream) {
        if (signal.aborted) break;
        if (item.error) { widget.render({ status: 'error', error: item.error }); continue; }
        widget.render({ status: 'live', data: item.data });
      }
    } catch (err) {
      widget.render({ status: 'error', error: err });
    }
  }

  async stop() {
    this.#ac.abort();             // signals every source + fetch to cancel
    await Promise.allSettled(this.#tasks); // wait for loops to unwind cleanly
    this.#ac = new AbortController();       // fresh controller for next start()
  }
}

Notice how stop() is a single line of real work: abort the shared signal and every polling loop, socket, and in-flight fetch tears itself down through the finally blocks you wrote in Milestone 2. That is the payoff of threading one AbortSignal through the whole system.

Handling partial failure well

Because each widget runs its own isolated loop, a WebSocket that dies only turns that widget's status badge red. The prices keep ticking. That is the concrete, visible result of the "one dead source must not take down the others" decision you made back in Step 1.

Checklist & What Good Looks Like

What good looks like

Summary & Quiz

📚 Further Reading

• MDN — for await...of
• MDN — AbortController & AbortSignal
• MDN — Server-Sent Events
• Pólya — How to Solve It

🚀 What's Next?

You've closed out asynchronous JavaScript by building something real. Next, Module 12 turns to the frontend framework that will consume APIs like the one behind this dashboard: we start with a tour of the React library and how it re-thinks building user interfaces.