Generator is a new concept introduced to JavaScript in ECMAScript 2015 (or ES6). Generators are the new powerful tools which should be in each JavaScript developer’s toolkit. Generators are mostly used in JavaScript frameworks and libraries. For example, Koa uses generators as middleware. Babel can use generators to transpile async/await functions. But generators are not commonly used in application code yet. The main reason is that generators are not easy to understand and adopt in day-to-day development.

This book focuses on real day-to-day development scenarios which can benefit from using generators.

Most the code examples in this book are tested on Chrome browser and some of them are tested on NodeJS 6.7.0. These examples should also work on other browsers which support generators. Refer to this page for browser compatibility of generators.

# I Generators basics

Before discussing actual usage of generators, we start from the basic concept of generators.

There are two different concepts related to generators.

• Generator function - A special kind of function which generates generator objects.
• Generator object - An instance of generator function.

Execution of generator objects can be suspended and resumed. In JavaScript, we have only limited control over execution of normal functions. Given a function, when it starts execution, by using (), apply or call, it will run to the end of the execution.

For a simple function sum shown below, when it’s invoked using sum(1, 2), it starts execution and returns value 3 to the caller.

As JavaScript engine execution is single-threaded (not considering web worker here), during the execution of a function, there is no way to stop the execution. So if you accidentally create an infinite loop in your function, the whole application will be blocked.

## 1. Basic generators

Let’s start with a simple generator function. The difference between a generator function and a normal function declaration is the * between function and the function name.

Generator objects can return multiple values when next() method is invoked. Those values are specified using yield keyword. In the generator function above, three yield expressions can generate three values 1, 2 and 3 when next() method of a generator object is invoked.

In the code above, invoking the generator function sample generates a new generator object func. Execution of generator object func is initially suspended. When next method is invoked on the func object, it starts execution and runs to the first yield expression and returns the value 1 to the caller. The return value is an object with two properties: value and done. value contains the return value of yield expression, done can be used to check if there are more values to get. done property is false for the first three invocations of next method. For the fourth invocation, done property is set to true, which means there are no values anymore.

### 1.1 Suspend & resume execution

The power of generators comes from the ability to suspend and resume execution of generator objects. Each generator object can be viewed as a state machine. Each instance of the same generator function maintains its own state. Invoking next() on the generator object triggers state transition inside the object, which causes the object runs to the next yield expression. This continues until no more yield expressions found.

In the code below, two generator objects func1 and func2 maintain their own internal states. Invoking next() on one object doesn’t affect the state of the other object.

### 1.2 Check types of generator functions and generator objects

We can use Object.prototype.toString to check the types of generator functions and generator objects.

## 2. Pass values to next()

Let’s start from another simple generator function doMath. If we just look at the code, we may think that after invoking next() on the generator object, the value of x should be 1, the value of y should be 11 and the value of z should be 110. It’s just simple math, right???

But the actual result doesn’t match what we would expect. As shown in the code below, the values are 1, NaN and NaN.

The key to understanding the actual result is that value passed to next() invocation is the actually used value of last yield expression. Since we didn’t pass any argument when invoking next(), so the value of each yield expression is actually undefined.

For the first next() invocation, there is no last yield expression, so the value is actually ignored. For the second next() invocation, value of last yield expression, i.e. yield 1 is set to undefined, which sets x to undefined, then sets the result of yield x + 10 to NaN. For the third next() invocation, value of last yield expression, i.e. yield x + 10 is set to undefined, which sets y to undefined, then sets the result of yield y * 10 to NaN.

Now we can try to pass a value when invoking next() method on a generator object. In the code below, the second next() invocation func.next(1) passes 1 to the generator object, so value 1 is set as the value of yield 1, which sets x to 1, then the result of this next() will be 11. For the third next() invocation func.next(2), 2 is passed as the value of yield x + 10, which sets y to 2, then the result of this next() will be 20.

## 3. return in generators

In the generator function, we can also use return statement. The returned value is also passed to the caller of a generator object’s next() method. return also finishes execution of generator object, i.e. done property is set to true. In the code below, the return value of second next(1) invocation is the value of return statement, i.e. x + 2.

### 3.1 Infinite values

It’s possible for a generator object to generate an infinite number of values, i.e. done property is always false. For example, we can create a generator which generates infinite integer numbers starting from 0. In this case, we can use return to finish generator objects.

In the code below, loop keeps generating incremental values in a while loop. When a truthy value is passed to next() as the value of shouldExit, the last value is returned and generator object is finished.

As shown in the code below, three values are generated using next(). The forth next(true) invocation finishes the generator object func.

## 4. Iterators & generators

From all the generators code above, you may wonder why we should use next() to get values from the generator objects and deal with the nonintuitive return value format {value: 1, done: false}. Meet iterators.

### 4.1 Iterators

Iterators are no strangers to developers. They already exist in different programming languages with similar names, e.g. Java Iterator, Ruby Enumerator and Python Iterator Types. Iterators can be used to iterate over items in a collection. Iterators maintain their own states regarding the current position in the target collection.

An iterator in ES6 is just an object which provides a next() method to get next item in the current iteration. next() method should return an object with two properties: value and done. So generator functions are actually factories of iterators.

### 4.2 Iterables

Iterables are objects which have property @@iterator. The value of @@iterator property is a function that returns an Iterator object.

A generator object conforms to both the Iterator and Iterable interfaces.

### 4.3 Iterate generator objects

As generators are iterable, we can use other ES6 language features to interact with generator objects easily. Following examples use values generator function shown below.

#### for-of loops

We can use for-of loops to easily iterate all the values in a generator object.

Generator objects can also be used with spread operator.

#### Work with new collection types

Generator objects can be used to create new collection objects, e.g. Set, WeakSet, Map and WeakMap.

After introducing basic concepts of generators, we are now looking into more advanced features of generators.

The code in this chapter uses following debug function to log values to the console.

## 5. Arguments of generator functions

Like other normal functions, generator functions can take arguments. These arguments can be used in yield expressions inside the generator functions.

In the code below, seq is a generator function with arguments start and number. start means the start number of generated values and number means the total number of generated values.

## 6. return method

A generator object has a return method to return given value and finish the generator. This behavior is similar with using return statement inside of a generator.

Given the same values generator function shown below,

We can see how invoking return method finishes the generator object. The first next() invocation returns the first value 'a', then func.return('d') returns value 'd' and finishes the generator, i.e. done property is set to true.

return method can be invoked multiple times. Each invocation returns the value passed to return() method.

## 7. throw method

A generator object also has a throw method to pass a value to it and trigger an exception to throw inside of the generator object. Both throw and next methods can send values to generator objects and change their behaviors. A value passed using next is treated as the result of last yield expression, but a value passed using throw is treated as replacing last yield expression with a throw statement.

In the code below, when passing hello to the generator object using throw('hello'), an uncaught error is thrown and the generator object is finished. When func.throw('hello') is invoked, the last yield expression yield x + 1 is replaced with throw 'hello'. Since the thrown object is not caught, it’s propagated to the JavaScript engine.

Although it’s possible to pass any types of values to throw(), it’s recommended to pass Error objects for better debugging, e.g. throw(new Error('boom!')).

We can use try-catch in the generator function to handle errors. In the code below, when func.throw(new Error('boom!')) is invoked, last yield expression yield 2 is replaced with throw new Error('boom!'). The thrown object is caught by try-catch. So the execution continues until the next yield expression yield 3.

If the value passed by throw() is caught and handled by the generator object, it can continue to generate all remaining values. Otherwise, it will finish with a uncaught error.

## 8. yield*

So far aforementioned generator objects only generate a single value using yield expression one at a time. We can also use a yield* expression to generate a sequence of values. When a yield* expression is encountered, sequence generation of current generator object is delegated to another generator object or iterable object.

### 8.1 yield* & iterable objects

In the code below, generator function oneToThree uses yield* [1, 2, 3] to generate three values: 1, 2 and 3, which has the same result as generator function sample in basic generators. Using yield* expression is more concise and easier to read.

We can use multiple yield* expressions in a generator function, then values from each yield* expression are generated in order.

### 8.2 yield* & generator objects

We can also use other generator objects in yield* expressions.

### 8.3 Value of yield*

yield* is also an expression, so it’s evaluated to a value. The value of yield* expression depends on its target, i.e. the expression after yield*. The value is the last value generated by the iterable object or generator object, i.e. the value property with done set to true.

If yield* is used with iterable objects, then the evaluated value is always undefined, because the last generated value is always {value: undefined, done: true}.

If yield* is used with generator objects, we can control the last generated value using return inside of the generator functions.

## 9. Nested yield and yield*

We can nest yield and yield* to create complex values generation.

### 9.1 Nested yield

In the code below, the inner yield expression generates value 1 first, then the middle yield expression generates value of yield 1 - undefined, then the outer yield expression generates value of yield yield 1 - undefined.

### 9.2 Nested yield and yield*

In the code below, generator oneToThree first generates three values 1, 2 and 3, then its value undefined is generated by yield expression.

## 10. co

Generator functions can also be used to control code execution flow. By using yield expressions, we can control when the execution of a generator object should be suspended. When the execution of a generator object is suspended, other code can have the chance to run and choose the best time to resume the execution. yield* expressions allow the delegation to other generator objects or iterable objects, which can create complicated nested or recursive execution flows.

Generator functions are most useful when combining with Promises. As described in MDN,

The Promise object is used for asynchronous computations. A Promise represents a value which may be available now, or in the future, or never.

If the value of a yield expression is a Promise object, then we can suspend the execution of the generator object when waiting for the Promise to be resolved. When the Promise is fulfilled, we can resume the execution of the generator object with the fulfilled value as the value of the yield expression. Otherwise, we can finish the generator with the rejected error.

To support this kind of scenarios, we need to use the library co. In the code below, timeoutToPromise is a helper method that creates a Promise object using setTimeout. Generator function calculate uses yield expression and the Promise object created by timeoutToPromise. co(calculate, 1, 2) turns the generator function calculate into a Promise object.

Below is an example of using co with generator functions which have yield expressions with other generator objects. value is a generator function which takes the argument v as the seed of generating two random values v1 and v2. yield value(1) in calculate uses a generator object value(1) as the target of yield expression.

## 11. regenerator

If generator functions are not supported on the target platform, we can use regenerator to transpile generator functions into ES5. Babel also has a transform-regenerator plugin to perform the transformation. If you use Babel preset ES2015, then this plugin is already included.

This plugin can transform code using generators

into code using regenerator.

You can try it online.

# III Real-world usages

We are going to see how generators in real-world projects.

## 12. Koa

Koa is next generation web framework for NodeJS. Its powerful middleware architecture is built on top of generators. We are going to see how Koa uses generators.

### 12.1 Koa basics

Koa is very easy to configure and use. Each application creates different middleware to handle requests and generate responses. Each middleware is a generator function and registered using use() of Koa application. Middleware are processed in a chain with the same order as they are registered. Each middleware can access context information using this, e.g. request, response, method and url.

The code below is a simple Koa application. It registers two middleware generator functions, the first one is used to log request processing time and the second one is used to set the response body to Hello World.

Each middleware generator function can take an extra argument which represents the next middleware in the chain. If a middleware generator function needs to intercept execution of downstream middleware in the chain, it can perform certain tasks first, then call yield to delegate to other middleware, then perform other tasks after downstream middleware finish. The logging middleware generator function in the code above demonstrates this pattern. It records start time when a request comes in, then it calls yield next for delegation, finally it records the finish time and calculates the duration.

After accessing the http://localhost:3000, the console log looks like below:

### 12.2 koa-compose

koa-compose is the small library which does the composition of middleware generator functions. Its source code is very simple, only 29 sloc. compose is the main method to compose middleware. The argument middleware is an array of middleware generator functions in the order of registration. The return value of compose method is a generator function with argument next. next is an optional generator function which is the last middleware in the chain.

Let’s go through the generator function code line by line. The first line if (!next) next = noop(); sets next to a do-nothing generator function noop if it’s null. i is the loop variable for array middleware starting from the last middleware in the array. In the while loop, the generator function of each middleware is invoked with the current value of next as the argument, the returned generator object is set as the new value of next. Then yield* is used to delegate to final next generator object.

We’ll see how middleware are used in the sample application of Koa basics. The middleware array contains two generator functions, log and setBody. In the while loop, generator function setBody is invoked first with the argument next set to noop and next is set to the generator object of setBody. Then generator function log is invoked with the argument next set to the generator object of setBody and next is set to the generator object of log. The last yield* next expression delegates to the generator object of log.

The returned generator function of compose is turned into a regular function that returns a Promise using co.wrap from co. The wrapped function is the actual request handler. When a request comes in, the generator object of log starts execution first and runs until the yield next, so the start time is recorded. next is a generator object of setBody, invoking yield next triggers the execution of setBody and set the response body. Finally, the generator object of log resumes execution and calculate the duration.

## 13. Babel

Babel is a JavaScript compiler which allows developers to use future JavaScript features. Babel has different plugins to transform JavaScript code written with the latest standards into a version which is supported on today’s platforms.

### 13.1 Transform async/await

Babel has a async to generator plugin which transforms async functions into generator functions. We’ll use a simple NodeJS application to demonstrate the usage of this Babel plugin.

The code below shows the .babelrc file.

Given JavaScript code shown below,

After applying the plugin, the output is shown as below.

You can also view the transformed result online.

The transformation is straightforward and relies on a helper method _asyncToGenerator. async function is transformed into generator function and await is transformed into yield. The _asyncToGenerator helper is responsible for transforming generator functions into a regular function that returns a Promise.

From the source code of asyncToGenerator, we can see that it transforms a generator function into a Promise chain.

# IV Usage scenarios

In this chapter, we are going to see some common usage scenarios for generators.

## 14. Sequence generation

Generator functions are very useful when generating complex sequences. We can encapsulate the generation logic in the function and shield the consumer from internal details.

In the code below, generator function numbers has a complicated logic about generating values in the sequence.

For more complicated scenarios, we can also use yield* to combine sequences. Suppose we have a system which stores users information in both file system and database, we can use following code to return a sequence of all users.

We use the following code in file createTask.js to create tasks using setTimeout and Promise. The task fails when value is greater than or equals to 5.
We can also implement it using generator functions and co.