Modules, npm and bundling

An ES module is a JavaScript file that shares names with export and takes names with import. npm installs packages, and a bundler assembles modules.

When a script no longer fits in your head

Your page already has four scripts: a summary, quick links, a menu with search, and loading data. They sit one after another in a single HTML file, and that is still bearable. Now imagine a site with twenty pages and forty scripts. Which one declared the variable items? Which depends on which? What breaks if you reorder the <script> tags? In the early 2000s sites lived exactly like that: one enormous file, or a dozen files loaded in a strict order, with every script sharing one namespace. It was enough for two files to give a variable the same name, and one quietly overwrote the other.

You need a way to say of each file: this is what it offers to the others, this is what it takes from them, and everything else is its own business. Files like that are called modules.

How we got here

There were several answers. First, people wrapped code in a function that called itself at once, which gave it a private scope (the trick was called an IIFE). In 2009 the server-side JavaScript runtime Node.js got a module system, CommonJS, with a require command, and others (AMD among them) were invented for browsers. To make all of that work in a browser, bundlers appeared: Browserify and webpack (2011 and 2012), Rollup, Parcel, and later esbuild and Vite. In 2015 the language itself gained modules, the ES modules, and by 2018 every major browser understood them. Since then a small site doesn’t always need a bundler.

ES modules

A module file exposes what is marked with the word export, and the rest stays inside:

// data.js
export const items = [ /* … */ ];

Another file takes what it needs with import:

// render.js
import { items } from './data.js';

export function renderCards(container) { /* … */ }

You load the main file with one line, and the browser follows the chain of imports by itself:

<script type="module" src="main.js"></script>

The rules that trip people up the first time:

• the path starts with ./ or ../, and the .js extension is required: the browser doesn’t guess what you meant;
• each module runs once, however many files import it, and it has a scope of its own, so items in one module doesn’t clash with items in another;
• a module runs after the page is parsed, like the type="module" scripts in our lessons;
• you need a server: if you open the page with a double click (an address starting file://), the browser won’t let modules load each other. Any local server will do, for example npx serve . or python3 -m http.server.

There is also a dynamic import, import('./big.js'), which returns a promise: it loads a module only when it’s needed, such as the code of an editor that opens on a button press.

Something else looks odd: a line such as import x from 'lodash', with no ./, doesn’t work in a browser on its own. The browser doesn’t know where to look for a package by name. You need either a bundler, which finds the package in node_modules and puts it in the bundle, or an import map (importmap), where you write the address yourself.

Check yourself: why does a page with modules fail when you open it with a double click?

The browser doesn’t let modules read files from a file:// address. The page has to be opened through a local server.

npm: other people’s code in one command

So that nobody writes everything alone, developers share code as packages. The main warehouse of packages is called npm (and so is the program you install them with). Every project starts with a file called package.json, the project’s passport:

{
  "name": "my-site",
  "private": true,
  "type": "module",
  "scripts": { "start": "npx serve ." },
  "dependencies": { "dates-lib": "^1.4.2" }
}

It holds the project’s name, the mark "type": "module" (the files count as modules), the commands in scripts (you run one as npm run start) and the list of dependencies. The command npm install downloads them into a folder called node_modules, and a file called package-lock.json records the exact versions, so that you and a colleague get the same thing.

Versions follow the semver convention: three numbers, major.minor.patch. A patch fixes bugs, a minor version adds features without breaking anything, and a major version breaks compatibility. The ^ in ^1.4.2 lets you update inside the major version: 1.5.0 is fine, 2.0.0 is not.

The bundler

A bundler follows the chain of import statements, gathers everything the page needs into one or more files, and does useful things to them. It makes them smaller (minification removes spaces and shortens names). It throws out what nobody used (tree shaking, as if you shook the tree and the dead leaves fell off). It turns TypeScript, JSX and other formats into ordinary JavaScript, and it runs a local server that refreshes the page on every edit. Today the most popular in new projects is Vite, and under the hood of many bundlers lies esbuild. Try it on the diagram: add a big package and see what each of the two operations does to it:

add the dates-lib package minification tree shaking Build

• main.js 1.2 KB imports: render.js, data.js, form.js
• render.js 2.1 KB imports: data.js
• form.js 1.8 KB imports: dates-lib
• data.js 1.5 KB imports nothing
• dates-lib 72.0 KB an external package: 40 functions, and the form needs one

Source: 78.6 KB in 5 files. A browser would load them one by one, each with its own imports.

The numbers are illustrative: they show orders of magnitude, not measurements. The real size comes from your bundler and the Network tab.

Does your site need a bundler? A small site of plain modules with no outside packages doesn’t: the browser copes on its own, and HTTP/2 long ago removed the old problem of “many small files”. A bundler becomes necessary when you add packages from npm, write TypeScript or take a framework (more on that in the next lessons).

Every dependency has a price

A package goes in with one line, and that is why people add so many. But every package is:

• weight on the page, which the visitor downloads, on a weak phone too;
• someone else’s code, which you must update and in which vulnerabilities are found (the command npm audit reports the known ones);
• reliance on other people’s decisions. In March 2016 the author of a tiny package called left-pad (eleven lines that pad a string with spaces) removed it from the registry, and thousands of builds around the world broke, because other packages leaned on it. And in 2018 malicious code got into the popular package event-stream through a new dependency. That’s no reason to fear npm. It is a reason to ask yourself before every install: how much does this package save, and what does it cost?

A good rule: a package is justified if it replaces a lot of work and there is no sense in writing it yourself (parsing dates with time zones, for example). Three lines that are easy to write yourself don’t deserve a package.

Practice

You’re on a 2019 site. The scripts have grown, and it’s time to split them into files. The sandbox can’t load several files, so you’ll write the modules as text, as if they lay in files: in pre blocks named data.js, render.js, main.js and package.json. Inside pre, the < sign is written as <, and the rest of the code stays as it is.

Then do what the sandbox can’t, and test on your own machine. Create a folder, put in it an index.html with the tag <script type="module" src="main.js"></script> and the three files from the blocks. Run npx serve . in that folder (you’ll need Node.js) or python3 -m http.server, and open the address it shows. If the console shows a red error, read it: most often it’s a forgotten .js in a path or file:// instead of a server address.

At the end, write which outside packages the project needs and how much they weigh. For your site, probably none: everything is done in plain JavaScript. That is an answer too, and a good one.

What was wrong before

Scripts were glued together by hand or by build commands (Grunt, Gulp), shared names lived in one global object (window.MyApp = {}), the order of <script> tags was part of the program, and nobody knew exactly which file depended on which. For every jQuery plugin you had to remember that it needed jQuery before it. Today those dependencies are written in the code itself, in the import lines, and people, bundlers and editors all see them.

What remains true: divide code by responsibility, and keep data apart from what draws it. And a package from the registry is still added with care: what is handy today can be a problem tomorrow.