This article assumes you completely understand all performance trade-offs related to each available technique to load scripts and how to modularize them. I'd highly recommend you to read another article I wrote just to explain them here.

Motivation

Feel free to skip this section if you are not interested in the background.

I've been using a single JS and a single CSS for my application for a long time.

I've optimized the code a lot, by lazily running some parts and performing all best practices with regards to how to load the resources, minifying them, gzipped them, caching them and so on and, still, every week about 10% of the users won't meet the SLA that says the page should load in within 5s. Some users would load the page under a second even when the resources were not cached.

To be honest, it's not really defined under which conditions a user should be able to load the application in under 5s, so I use the worst scenario to measure this time. After the page is fully loaded I send the data from the resources timing API to the back-end so that I can extract some statistics later, since NewRelic is too limiting for this kind of information. Here's how it works in our application. The user logs in another application which will provide a link to ours, containing an authentication token, which we'll parse, verify and redirect to the root address. I use the time provided by the resources timing API, which will include this redirect.

It should be noticed that any actions in the server-side take about 10-20ms, accordingly to the nginx logging (for the actions related to page loading - opening a transaction or searching the database might take 1s in the server-side for example, depending on the criteria). This means most of the time is spent outside the server and are influenced by latency, network bandwidth between the client and server, CDN, presence of cached resources and so on. Of course, running the JS code itself already contributes to the total time, but this part was already highly optimized before switching away from Sprockets. Half of the accesses were able to run all JS loading code in up to 637ms. 90% up to 1.3s. 3% loaded between 2 and 2.2s. That means that for the slowest client all network operations should complete in about 2.8s, including DNS lookup, redirect and bytes transfer. I can't make those browser run faster and I can't save more than 20ms in the server-side, so my best option is to reduce the amount of data that should be transferred from the server to the client, as I don't have much control over our collocation service provider (Cogent - NY), or the client Internet provider or our CDN provider (CloudFront).

But I can choose which libraries to use and which code to include in the initial page loading. When working with performance improvements the first step is always measuring. I created an application to provide me the analytics I needed to understand the page loading performance so that I could confirm that I should be now focusing on the download size. To give you an idea, the fastest access to our application in the last week was 692ms, from an user accessing from London. The resources were already in cache in this request, the main document loaded in 244ms and the JS code ran in 301ms, using IE10. No redirect happened for this request.

Here's another sample for a fast page load including redirect and non cached resources. Some user from NY loaded the full application in 1.09s. 304ms were spent on redirect, 34ms to load the main document 107ms to load the CSS and 129ms to load the JS (JS and CSS are loaded in parallel). It took 479ms for IE11 to process the scripts in this requests.

Now, let's take a look in a request which took 8.8s to load to understand why it took so long. This request used 6s to load the same JS from the same location (NY) while the redirect took 1.9s. The CSS took 4.3s to load. And this is not a mobile browser, but IE11, and it's a fast computer as the scripts took only 453ms to run. When I take a closer look at the other requests taking over 5s, I can confirm the bad network performance is the main reason for this.

If I want to make them load under 5s I must reduce the amount of data they are downloading. After noticing that I realized sprockets was in my way for this last bit of performance improvement. I had already cut a lot of vendored code which were big and I only used a small part of them, so it was time I had to cut out part of the application code. Well, actually the plan was to post-pone its loading to when they were needed, for example, after the user made some action like clicking some button or link. In other words, I was looking for code splitting and I'd had to implement it on my own if I were to keep using my current stack (Sprockets by that time, or the Rails Assets Pipeline) but I decided to switch to another better tool as I also wanted source-maps support and other features I couldn't get with Sprockets.

Source maps are very important to us because we report any JS errors to our servers including backtraces for future analysis and having the source-maps available makes it much easier to figure out the exact place an exception happened.

Goals

In the context of big single page applications, the ideal resources build tool should be able to:

• support code modularization (understands AMD, CommonJS, allows easy shimming and features to integrate with basically any third-party library without having to modify their sources);
• concatenate sources in bundles, which should be optimized to avoid missing all cache upon frequent deploys;
• support code splitting (lazy code loading) - Sprockets and many other tools do not support this, which would require each developer to roll their own solution) to not force the user to download more code than what is required for the initial page rendering;
• minify JS and CSS for production environments;
• provide a fast watch mode for development mode;
• provide source maps;
• allow CSS to be embedded in JS bundles as well as allowing a separate CSS file (more on that in the following sections);
• support CSS and JS preprocessors/compilers, like Babel, CoffeeScript, SASS, templating languages and so on;
• support filenames containing content-based hashes to support permanent caching;
• provide great integration with NPM and bower packages;
• fast build time for the production-ready configuration to speed up deploys though the usage of persistent caching (on disk, Redis or memcached, for example);

Webpack was the only solution I was able to find which supported all of the items above except for the last one. Sprockets and other solutions are able to handle persistent cache to speed up the final build and consequently the deploy process. Unfortunately the deploy will be a bit slow with webpack, but at least the application should be highly optimized for performance.

If you are aware of other tools that allow the same techniques discussed in this article to be implemented, please let me know in the comments, if possible with examples on how to reproduce the set-up presented in this article.

The webpack set-up

This article is already very long, so I don't intend it to become a webpack tutorial. Webpack has an extensive documentation about most of what you'll need and I'll try to cover here the parts which are not covered by the documentation and the tricks I had to implement to make it meet the goals I stated above.

The first step is to create some webpack.config.js configuration file and to install webpack (which also means installing npm and node.js). I decided to create a new directory under app-root/app/resources and perform these commands there:

The build resources would be generated in app-root/public/assets and the test files under app-root/public/assets/specs. It looks for resources in app/resources/src/js, app/resources/node_modules, app/resources/bower_components, app/assets/javascripts, app/assets/stylesheets, app/assets/images and a few other paths.

webpack.config.js:

loaders/ko-loader.js:

loaders/non-cacheable-extract-text-webpack-loader.js (required due to a webpack bug):

Here's how jquery-ui-autocomplete.js looks like (the others are similar):

jQuery UI was installed from bower and lives in bower_components/jquery-ui.

Here's how my package.json looks like:

I told you. It took me about a week to perform this migration ;)

But believe on me. It worths.

Just run "node_packages/.bin/webpack -w" to enable the watch mode. I'd recommend adding "node_packages/.bin" to PATH in .bashrc so that you can simply run webpack, bower without specifying the full path. For the production build, simply run "PROD=1 webpack".

Vim users should set backupcopy to yes (default is auto) otherwise the watch mode won't detect all file changes as sometimes Vim would move the back-up and create a new copy which is not detected by the watch mode. See more details here.

If you are experiencing other issues with the watch mode, please check the Troubleshooting section of Webpack documentation.

Back-end integration

If you're interested in integrating to Rails, you can stop reading here and jump to the Rails integration section of this article. Or if you'd like to get a concrete example. Otherwise, here are the general rules for integrating to your backend.

Webpack will generate a webpack-assets.json file due to the assets-webpack-plugin, which allows us to get the generated bundle full name with the chunk hash included so that we can use it to pass to the script src attribute. The configuration above would generate 3 bundles. One for common libraries, other for clients and another for internal users (containing some additional features not available to client users).

Here's some incomplete JavaScript code demonstrating how it works:

Then, it's used like this in the page:

Specifying dependencies in the code

Webpack has good documentation on how it detects the code dependencies so I won't get into the details but will only demonstrate two common usages. One for a regular require, which will concatenate the code and another for code splitting usage.

Take this code for example:

The require.ensure call is not really required but it allows you to give the lazy chunk a name which is useful if you want to add other files to the same chunk in other parts of the code.

In that example, jquery will go to the vendors bundle, app.js will go into the app bundle and glossary.js (and any other files added to that chunk) will be lazily loaded by the application. You can even preload it after initializing the application so that the click happens faster when the user click on the #glossary element.

Some numbers

Well, after all this text you must be wondering whether it really worths, so let me show you some numbers for my application.

Before those changes, there was a single JS file which was 864 KB (286 KB gzipped). If we consider the case where the user took 6s to load this file, I think it's fair to emulate throttling for Regular 3G (750 kb/s 100 ms RTT) in the Chrome dev tool. I've also enabled the film-strip feature. After disabling cache, the first initial rendering (for the "loading..." state) happened at 1.16s while the application was fully loaded at 5.26s. It also took 594 ms to load the 74.2 KB CSS file (17.7KB gzipped).

Now, after enabling code splitting, and reducing the initial CSS, here are the numbers. Now the initial "loading..." state was rendered at 499ms and the page was fully loaded at 4.7s. The CSS file is now 7.2 KB (2.4 KB gzipped) and the JS files are 498 KB (169 KB) gzipped for vendor and 259 KB (77.8 KB gzipped) for the app bundle. Unfortunately I couldn't cut much more application code in my case and most of the code is from vendored libraries, but I think there's still room to improve now that webpack is in place. So, whether it worths or not for you to go through all these changes will depend on the percentage of your code which is required for the initial full page rendering, and on the frequency you deploy (I deploy very often, so just the ability of creating a commons bundle is good enough to justify this set-up).

Just for the sake of completeness, I'll also show you the numbers with cache enabled and with throttling disabled.

With cache enabled, the initial render happened at 496ms and the page was fully loaded by 1.35s in Regular 3G throttling mode for the webpack version. If I disable throttling, with a 10 Mbps Internet connection and accessing the NY servers from Brazil I get 354ms for the initial rendering and 1.22s for the full load. If I disable the cache and throttling I get 445ms and 2.03s.

For the sprockets version, the initial render happened at 846ms and the page was fully loaded by 1.74s in Regular 3G throttling mode. If I disable throttling I get 553ms for the initial rendering and 1.48s for the full load. If I disable the cache and throttling I get 740ms and 2.80s.

Actually, those numbers are both for webpack, as I am no longer able to test the sprockets version. But I'm calling it sprockets anyway because the first approach should be feasible with sprockets. But after moving to webpack I was able to more easily extract only the parts we use from jQuery UI and replace underscore with lodash to use only the parts we need and I've also got rid of some other big libraries in the process. Before those changes the app bundle was 1.2MB minified (376KB gzipped), so I was able to reduce the amount of transferred data to about 65% of what it used to be, but it wouldn't be fare to compare those numbers because in theory it should be possible to achieve a lot of this reduction without dropping sprockets.

But in our case, we were able to improve the page loading speed after moving to webpack even before applying code splitting due to the flexibility it provides us which I find easier to take advantage of when compared to how we used the assets from sprockets.

And now we're able to use the source-maps for both debugging in the production environment but specially to understand the stack-traces when JS exceptions are thrown.

If you have any questions please write them in the comments or send me an e-mail and I'll try to help if I can.