🔁 Async Generators and Iteration

Data That Arrives Over Time

Think of an async generator as a water dispenser that fills one cup on demand. You don't wait for it to fill a thousand cups before drinking; you take one, and only when you ask for the next does it refill. That is lazy, pull-based streaming — the consumer sets the pace, and the producer only does work when asked.

This model shines whenever the full dataset is large, slow, or unbounded: paginated results, file streams, real-time feeds. Instead of buffering everything into memory, you process each item as it arrives and keep a flat memory footprint.

Iterators & Iterables

Two small protocols underpin everything in this lesson.

An iterator is any object with a next() method that returns { value, done }. An iterable is any object with a [Symbol.iterator]() method that returns an iterator — that's what makes it usable in for...of and the spread operator. Arrays, strings, Maps, and Sets are all built-in iterables.

// A hand-written iterable range: for..of works because of Symbol.iterator.
function range(start, end) {
  return {
    [Symbol.iterator]() {
      let current = start;
      return {
        next() {
          return current <= end
            ? { value: current++, done: false }
            : { value: undefined, done: true };
        },
      };
    },
  };
}

for (const n of range(1, 5)) console.log(n); // 1 2 3 4 5
const nums = [...range(1, 5)];                // [1, 2, 3, 4, 5]

Writing iterators by hand is verbose — you manage all the state yourself. Generators exist precisely to remove that boilerplate.

Generators: A Refresher

A generator function (function*) can yield a value and pause, resuming where it left off on the next call. It is both an iterator and an iterable, so the state-tracking is handled for you.

function* rangeGen(start, end) {
  for (let i = start; i <= end; i++) {
    yield i; // pause here, hand `i` to the consumer, resume on next()
  }
}

for (const n of rangeGen(1, 5)) console.log(n); // 1 2 3 4 5

Two features become important later. First, yield* delegates to another iterable, splicing its values in — the basis of composing generators:

function* letters() { yield 'a'; yield 'b'; }
function* combined() {
  yield* letters(); // delegate
  yield 'c';
}
console.log([...combined()]); // ['a', 'b', 'c']

Second, a finally block in a generator runs when the consumer stops early (e.g. break), which is how generators clean up resources reliably.

Async Iteration & for await

The synchronous protocols assume each value is ready immediately. For values that arrive asynchronously, JavaScript adds a parallel set: an async iterator has a next() that returns a Promise of { value, done }, and an async iterable implements [Symbol.asyncIterator](). You consume them with for await...of inside an async function.

// A hand-written async iterable that emits values on a delay.
function delayedValues(values, delayMs = 500) {
  return {
    [Symbol.asyncIterator]() {
      let i = 0;
      return {
        async next() {
          if (i >= values.length) return { value: undefined, done: true };
          await new Promise(r => setTimeout(r, delayMs));
          return { value: values[i++], done: false };
        },
      };
    },
  };
}

for await (const value of delayedValues([1, 2, 3])) {
  console.log(value); // 1, then 2, then 3 — each after a delay
}

Async Generators

An async generator (async function*) is the payoff: it can await inside its body and yield values, giving you the concise syntax of a generator with the asynchronous behaviour of an async iterator. The state machine, the promises, and the cleanup are all handled for you.

async function* delayedNumbers(start, end, delayMs = 500) {
  for (let i = start; i <= end; i++) {
    await new Promise(r => setTimeout(r, delayMs)); // pause asynchronously
    yield i;                                        // then emit
  }
}

for await (const n of delayedNumbers(1, 5)) {
  console.log(n); // 1..5, one every ~500ms
}

Worked example: a paginated API client

The classic use case. Fetching a paginated endpoint by hand means tracking the page number, the "has more" flag, and looping — all tangled with your processing code. An async generator isolates the pagination logic completely; the consumer just iterates.

// The generator owns ALL the pagination bookkeeping.
async function* paginate(endpoint, pageSize = 20) {
  let page = 1;
  let hasMore = true;

  while (hasMore) {
    const res = await fetch(`${endpoint}?page=${page}&pageSize=${pageSize}`);
    if (!res.ok) throw new Error(`API error: ${res.status}`);

    const data = await res.json();
    hasMore = data.hasMore ?? data.next != null;
    page++;

    yield data.items; // hand this page's items to the consumer
  }
}

// The consumer is blissfully unaware of pages — it just streams records.
async function collectAllUsers() {
  const users = [];
  for await (const pageItems of paginate('/api/users')) {
    for (const user of pageItems) users.push(user);
    console.log(`Loaded ${users.length} users so far...`);
  }
  return users;
}

Notice what the consumer gains: it processes each page as it arrives rather than waiting for the entire dataset, memory stays constant regardless of how many pages exist, and a normal try/catch around the loop handles a mid-stream error cleanly.

Backpressure for free

Backpressure is the ability of a slow consumer to throttle a fast producer. With async generators it's automatic: the generator is suspended at its yield until the consumer asks for the next value. A slow loop body naturally paces the producer — no buffering, no overflow.

Composable Pipelines

Because an async generator both consumes an async iterable and produces one, you can chain them into a pipeline — each stage a small, testable transform. This is the same idea as array methods (filter, map), but lazy and asynchronous.

// Reusable stages: each takes an async iterable and returns one.
async function* filter(source, predicate) {
  for await (const item of source) {
    if (predicate(item)) yield item;
  }
}

async function* map(source, fn) {
  for await (const item of source) {
    yield await fn(item); // fn may itself be async
  }
}

async function* batch(source, size) {
  let group = [];
  for await (const item of source) {
    group.push(item);
    if (group.length >= size) {
      yield group;
      group = [];
    }
  }
  if (group.length > 0) yield group; // flush the remainder
}

// Compose: stream users, keep active ones, enrich each, process in batches of 5.
async function processUsers() {
  const active = filter(paginate('/api/users'), u => u.status === 'active');
  const enriched = map(active, async (u) => ({
    ...u,
    details: await fetch(`/api/users/${u.id}/details`).then(r => r.json()),
  }));

  for await (const group of batch(enriched, 5)) {
    console.log(`Processing a batch of ${group.length} users`);
    await saveBatch(group);
  }
}

Nothing runs until the final for await...of pulls a value, and only one item flows through the whole chain at a time. That keeps memory flat even for a stream of millions.

Hands-on Exercise

async function* countForever() {
  let i = 0;
  try {
    while (true) {
      await new Promise(r => setTimeout(r, 200));
      yield i++;
    }
  } finally {
    console.log('cleaned up'); // runs on early termination
  }
}

async function* take(source, n) {
  let count = 0;
  for await (const value of source) {
    if (count++ >= n) return;
    yield value;
  }
}

for await (const value of take(countForever(), 5)) {
  console.log(value); // 0 1 2 3 4
}
// Then: "cleaned up" is logged, and the program exits.

Best Practices

Summary & Quiz

📚 Further Reading

• MDN — AsyncGenerator
• MDN — for await...of
• MDN — Symbol.asyncIterator
• Jake Archibald — Async iterators and generators

🚀 What's Next?

You've now covered the full asynchronous toolkit — promises, error handling, concurrency, and streams. It's time to put it together. The Weekend Project: Asynchronous JavaScript asks you to build a small real-world app that fetches, retries, coordinates, and streams data end to end.