Understanding JavaScript Module Resolution Systems with Dinosaurs

Modules make code cleaner, more reusable, and more fun to work with. They let you separate your JavaScript into separate files and they protect you and everyone else on your team from muddying up the global scope. It's also worth considering the fact that the dinosaurs didn't use modules and now they're extinct.

To take advantage of modules in modern ES6 JavaScript, simply use the import and from keywords:

import { pterodactly } from 'dinotopia';

It just works! Somehow JavaScript knows how to find the pterodactly export in the dinotopia module and fit them together seamlessly. But... how?

And while we're asking questions, what are modules? How do they work? Why are there so many kinds of modules? And why does my text editor complain about them so much?

In this post, we'll use a few lines of JavaScript and the TypeScript compiler to answer these questions and clear up some of the confusion around modules.

Now before you close this tab because you thought this was a JavaScript post and you just saw the word TypeScript, I want to assure you that there won't be any code on the page that isn't regular JavaScript. We're just going to take advantage of TypeScript's compiler options to quickly turn our JavaScript code into the different types of modules that are in use today.

A Bit of History

Back in the prehistoric days of 2014, modules did not exist in the ECMAScript (JavaScript) standard. Many libraries like jQuery would put an object or function in the global scope as a way of exporting their functionality. This solution, like anything that relies on the global scope, is a bad code smell and is like a meteor the size of Arizona hurtling towards your codebase's figurative Gulf of Mexico.

An artist's rendition of using the global namespace, courtesy of Wikimedia.org

Libraries like Node added module-like functionality that could be accessed with module.exports and require, but there wasn't an industry agreed upon implementation.

You'll see some of that legacy (and confusion) shining through to the way modules are used today.

The State of the Industry

Somehow we ended up with five major module loading systems.

The CommonJS system, used and popularized by Node.js, works well on the server side but can be slow in the browser because it loads each module synchronously.

To solve module resolution in the browser, the Asynchronous Module Definition (AMD) standard was created. The AMD library require.js is used by many front-end libraries because it is much faster than CommonJS, but this unfortunately created two different standards.

To solve the problem of competing standards, the Universal Module Definition (UMD) standard was created. This one works in both the browser and the server by using either CommonJS or AMD depending on what is available.

How standards reproduce, courtesy of xkcd.com

As JavaScript became more complex, SystemJS entered the scene as a more complete module resolution system that can import all of the standard JavaScript module types as well as "global" modules into its own namespace. It has some nice bells and whistles that the other systems don't have, but takes a bit of configuration to set up in your project.

And finally, with ECMAScript 2015 (ES6), JavaScript gained the a native concept of modules. Unfortunately this didn't make everyone agree upon how modules should work, but it did create a de-facto standard that all the other standards would have to be able to handle.

Why TypeScript?

If you've never used TypeScript before, you're missing out. But you won't need to know anything about the language other than that it is a superset of JavaScript that compiles directly into JavaScript. How that compilation works is determined by a few Compiler Options, one of which sets the module resolution standard used by the output.

This means that we can write some JavaScript code, compile it with the TypeScript Compiler using each of the different module resolution systems, and get a snippet of JavaScript code for each standard.

See? Nothing scary going on here.

Our Code

We'll use this snippet of JavaScript as a way to understand what each module resolution system looks like:

// Imports tRex as the 'default' export of "./best-dino"
import tRex from "./best-dino";

// Imports pterodactly as a named export of "./flying-dinos"
import { pterodactly } from "./flying-dinos";

// Uses the two imports and exports the result as a named const
export const moreAwesome = tRex.awesomeness + pterodactly.screech;

This is written with the native ES6 module format, but TypeScript will help us turn it into all of the other formats. It has imports and an export, so we should be able to see both sides of the module resolution flow.

Note that this is entirely JavaScript, without any TypeScript at all.

CommonJS

Let's set the TypeScript compiler to use the CommonJS module resolution format. The entire output we get is:

exports.__esModule = true;
var best_dino_1 = require("./best-dino");
var flying_dinos_1 = require("./flying-dinos");
exports.moreAwesome = best_dino_1["default"].awesomeness + flying_dinos_1.pterodactly.screech;

First, we see that CommonJS assumes there will be an object called exports available in the scope of the file. Libraries like Node make sure this is possible, but exports is not a reserved word in ECMAScript. Try typing exports, module, and stegosaurus into the console of your Chrome Dev Tools and you'll see that JavaScript has no idea what any of them are (unless you're building a Dino Quiz website and you put stegosaurus in global scope).

We then see that we're setting the __esModule property of exports to true. This simply tells any system that imports this file that it just imported a module. If this option is off, some module resolution systems will assume that this file would put an object in the global scope and will execute it without trying to get any of its exports directly.

Next, we see that CommonJS uses require, which is also not in the ECMAScript standard. CommonJS relies on Node or another library to define the require function. If you're interested in how require works under the hood, check out Fred K Schott's excellent post about it. The important part for us is that require accepts a filepath as a string argument and synchronously returns a value that we store in the variable best_dino_1.

Wait, what happened to tRex and pterodactly? TypeScript turned each of the imports into their own namespaced objects. In the next line, we can see that tRex has been replaced by best_dino_1["default"], since we imported tRex as the default export from "./best-dino".

Finally, CommonJS exports with the exports object, using the name of our exported variable moreAwesome as the name of the property on exports.

CommonJS does a bit of magic behind the scenes with the require method and the exports object, but it doesn't look that different from our original code.

AMD

Using the AMD module setting on the TypeScript compiler, our code becomes:

define(["require", "exports", "./best-dino", "./flying-dinos"], function (require, exports, best_dino_1, flying_dinos_1) {
    exports.__esModule = true;
    exports.moreAwesome = best_dino_1["default"].awesomeness + flying_dinos_1.pterodactly.screech;
});

AMD looks quite different from what we put in, but we'll see that it isn't all that different from CommonJS.

First, we see a new function called define. This is AMD's way of creating modules and their dependencies, and you can read more about it here. We see that it takes two parameters, the first being a list of dependencies that includes require and exports. These aren't strictly necessary for an AMD module, but they allow us to integrate this module with CommonJS modules, so TypeScript throws them in there for safety's sake.

The second parameter passed to define is the module creation function. Whenever the potentially asynchronous magic of define finishes, it will call this function with all the imports passed as parameters. These are then available inside the function the same way we used them in CommonJS, and we can see that the body of this function looks very similar to the entire CommonJS module.

The biggest difference between CommonJS and AMD is that the define method wraps our module in a callback function, allowing us to load our dependencies asynchronously in a way that would be impossible with just the require method. In a web browser, this lets an AMD library like require.js (yes, the name is super confusing) request all of a module's dependencies at once instead of having to wait for each dependency to finish loading before requesting the next one. This saves time and makes your Dino Quiz site load faster.

UMD

We'll use the UMD module setting in our compiler to get a new batch of code:

(function (factory) {
    if (typeof module === "object" && typeof module.exports === "object") {
        var v = factory(require, exports);
        if (v !== undefined) module.exports = v;
    }
    else if (typeof define === "function" && define.amd) {
        define(["require", "exports", "./best-dino", "./flying-dinos"], factory);
    }
})(function (require, exports) {
    exports.__esModule = true;
    var best_dino_1 = require("./best-dino");
    var flying_dinos_1 = require("./flying-dinos");
    exports.moreAwesome = best_dino_1["default"].awesomeness + flying_dinos_1.pterodactly.screech;
});

Whoa. How did three lines of JavaScript turn into this velociraptor's nest? Let's see if we can figure out what's going on here.

The entire module has been wrapped in an Immediately Invoked Function Expression, not just a single function call like we had in the AMD code. There's nothing magical about IIFEs, but here's a logically equivalent version of the above code with some helpful function names and no IIFE:

function myModuleCreationFunction(require, exports) {
    exports.__esModule = true;
    var best_dino_1 = require("./best-dino");
    var flying_dinos_1 = require("./flying-dinos");
    exports.moreAwesome = best_dino_1["default"].awesomeness + flying_dinos_1.pterodactly.screech;
};

function createModuleWithCommonJsOrAmd(factory) {
    if (typeof module === "object" && typeof module.exports === "object") {
        var createdModule = factory(require, exports);
        if (createdModule !== undefined) {
          module.exports = createdModule;
        }
    }
    else if (typeof define === "function" && define.amd) {
        define(["require", "exports", "./best-dino", "./flying-dinos"], factory);
    }
}

createModuleWithCommonJsOrAmd(myModuleCreationFunction);

Now we can see that there are two major parts to the UMD resolution strategy. The first is a module creation function or a "factory function." This function looks almost exactly like the CommonJS output.

The second part is a function that decides whether to use CommonJS or AMD. It does this by first checking to see if the CommonJS module.exports object exists, and if that fails, it checks to see if the AMD define function exists. This way, UMD lets you use CommonJS and AMD for module resolution without having to do much thinking. This might be useful for a project that shares code between a client and a server.

SystemJS

Let's use TypeScript to create a SystemJS module:

System.register(["./best-dino", "./flying-dinos"], function (exports_1, context_1) {
    var __moduleName = context_1 && context_1.id;
    var best_dino_1, flying_dinos_1, moreAwesome;
    return {
        setters: [
            function (best_dino_1_1) {
                best_dino_1 = best_dino_1_1;
            },
            function (flying_dinos_1_1) {
                flying_dinos_1 = flying_dinos_1_1;
            }
        ],
        execute: function () {
            exports_1("moreAwesome", moreAwesome = best_dino_1["default"].awesomeness + flying_dinos_1.pterodactly.screech);
        }
    };
});

At first, SystemJS looks entirely different from anything we've seen up to this point and uglier than a Masiakasaurus:

Maybe the ugliest dinosaur, courtesy of one of my favorites sites, dinosaurpictures.org

But if we look a bit closer, we'll find some similarities to the other module resolution methods.

Like AMD, SystemJS wraps our entire module in a function call — a call to System.register — and passes as parameters a list of dependencies and a function to run when those dependencies resolve. Also, like AMD, this lets SystemJS resolve those dependencies asynchronously.

The name of the function SystemJS uses, register, tells us a bit more about how the system (pun intended) works. Registration implies that the result of the execution of a dependency will be stored by SystemJS somehow, enabling things like caching and hot reloading. While these are available with the other module resolution methods, they are optimized in SystemJS.

Next we see that our callback function is a bit bigger than before. It declares some variables in a scope that is available to some setter functions as well as an execute function. This seems like a bit of overkill compared to CommonJS, AMD, and even UMD, but it allows SystemJS to reload a module without having to re-run the entire dependency graph simply by executing the new module and calling the appropriate function in setters.

Finally, our execute function is pretty similar to the factory function in UMD. It is just a bit more explicit about the name of the export. This function can get called by SystemJS once and the exports can be stored and quickly accessed by other modules.

ES6

Since our input was an ES6 module, the output from the TypeScript compiler should be pretty similar:

import tRex from "./best-dino";
import { pterodactly } from "./flying-dinos";
export var moreAwesome = tRex.awesomeness + pterodactly.screech;

Yup. The only difference is that TypeScript turned our const into a var.

But since I didn't explain them earlier, it is worth talking about import, from, and export. As of ES6/ES2015, these are all reserved words. Your JavaScript interpreter, be it the browser, Node, or something else that can run ES6, understands them natively. The specification does not say anything about caching imported modules, but most browsers and Node are smart enough to handle that.

It's important to recognize the similarities between ES6 modules and CommonJS. Both strategies load modules synchronously, which could potentially slow down your project on the front-end.

For most JavaScript interpreters, this feature was the hardest to implement and took the longest out of all of the new ES6 specifications. If you really want to understand how import works under the hood, take a look at the module parser in Node's C++ source and shoot me a message if you find anything interesting.

Conclusion

Now that you've seen all of the module resolution systems available to you, hopefully you'll be able to make a more informed decision about which strategies to use in your projects. There's no perfect solution, but as with every engineering decision, there is probably a best solution for your situation.

If you're building for a web based client, you might consider entirely removing module separation from your production code. If you bundle all of your dependencies with Grunt, Gulp, or WebPack, you prevent the browser from making multiple requests because all of your JavaScript is contained in a single file. Bundling gives you a speed boost and prevents headaches with modules in the client, but you'll have to pick a module resolution strategy for your bundler.