Thursday, March 18th, 2010

RequireJS: Asynchronous JavaScript loading

Category: JavaScript, Node, Performance

<p>A certain someone was talking to me about how they find it interesting that node.js, the JavaScript server framework du jour which loves all things async, starts life with a bunch of synchronous require() calls. Now, this is actually quite fine since the startup of the server is not the issue at hand.

However, if you are running require()-esque loader code in the browser you want to avoid blocking calls else Steve Souders will come over and beat you up.

I have seen a couple of interesting items in this area:

RequireJS

James Burke of Mozilla Messaging has spent a lot of time in the depths of dojo.require(). He has taken another look at the problem and RequireJS a solution that offers:

  • some sort of #include/import/require
  • ability to load nested dependencies
  • ease of use for developer but then backed by an optimization tool that helps deployment

He walks through the problem and why other solutions like LABjs, CommonJS require, and Dojo itself don’t cover all of his bases.

The end result is:

javascript
< view plain text >
  1. // code that runs asynchronously when the library is loaded
  2. require(["some/script.js"], function() {
  3.     //This function is called after some/script.js has loaded.
  4. });
  5.  
  6. // defining the module and dependencies
  7. require.def(
  8.     // The name of this module
  9.     "types/Manager",
  10.  
  11.     // The array of dependencies
  12.     ["types/Employee"],
  13.  
  14.     // The function to execute when all dependencies have loaded. The arguments
  15.     // to this function are the array of dependencies mentioned above.
  16.     function (Employee) {
  17.         function Manager () {
  18.             this.reports = [];
  19.         }
  20.  
  21.         // This will now work
  22.         Manager.prototype = new Employee();
  23.  
  24.         // return the Manager constructor function so it can be used by other modules.
  25.         return Manager;
  26.     }
  27. );

Google Analytics “async add to []” Pattern

When talking to Davis Frank of Pivotal about some Google Analytics code, he pointed me to details about the new GA asynchronous loader that we very excitedly blogged about since GA was such a blocking offender on the Web.

Part of the asynchronous API is that you, the developer create an array, and use the push() method to put commands on a queue. This means that you can start pushing commands immediately.

Then, when the GA code loads asynchronously, it takes over that array and wraps those standard methods. Now it can take the commands and fire them back to GA and push() can do more. Freaking brilliant.

Related Content:

Posted by Dion Almaer at 6:15 am
19 Comments

+++--
3.8 rating from 31 votes

19 Comments »

Comments feed TrackBack URI

We definitely need good on-demand loader classes, and this looks like a nice one. I wonder if it deals with the following issues: (1) loading components that shouldn’t be considered “ready” until they’ve done some initial processing, or done some additional load, (2) loading the same component multiple times concurrently (from require calls for different modules, that all depend on the same component). Those were the problems I ran into when designing a similar loader class.

Comment by Joeri — March 18, 2010

It’s nice to use it when you don’t have any other js library in hand. In fact most of the JS libraries support async loading of scripts. In jQuery that is done via:

$.ajax({
dataType : ‘script’,
url : ‘somefile.js’,
success : callback
});

and that may do the same thing as Requirejs. If problem is that jquery.js is quite big and if you’d like to load it via other library … but that’s supposed to mean at least to JS libraries.

Comment by stoimen — March 18, 2010

One of the key advantages of RequireJS is that it is not at odds with CommonJS, but works in conjunction with it. RequireJS follows the CommonJS module transport spec (currently this version: http://wiki.commonjs.org/wiki/Modules/Transport/C), which is specifically designed to make it easy to wrap CommonJS modules with the require.def for transport to the browser through asynchronous loading.

Comment by kriszyp — March 18, 2010

Please note that based on jQuery’s doc if you use this method to pull scripts they will be explicitly NOT cached by your browser.

http://api.jquery.com/jQuery.ajax/

“cache Boolean
Default: true, false for dataType ‘script’ and ‘jsonp’
If set to false it will force the pages that you request to not be cached by the browser.”

Set {cache:true} in the above ajax settings object for an increase in performance for static scripts.

Comment by ryanday — March 18, 2010

@stoimen: The actual loading of the scripts (like $.ajax) is pretty trivial, you don’t even really need a library to do that. The real reason for using something like RequireJS is the dependency management (with async resolution), especially when you have dozens or hundreds of modules.

Comment by kriszyp — March 18, 2010

I’ve developed a similar loading system for my RIA framework. It lets you define dependencies in JavaScript files and uses a “kind of ClassLoader” to resolve them asynchronously, once the file is loaded a listener is notified. A packager is also provided to put everything in one file.

You can have a look at this tutorial to have an idea on how it works:
http://www.zeleos.org/projects/webToolkit/V0.2/fiveMinutes.html

Dependency management becomes more and more important and I really think having such thing provided by the browsers would be really cool.

Comment by jkuhn — March 18, 2010

I recently had a crack at a module manager, and the above looks fairly similar. It resolves dependencies, ensures modules are loaded and executed in sequence and prevents multiple requires. Took some doing though.

Comment by kissmyawesome — March 18, 2010

This will definitely be useful for sites for which traffic is heavy and a new instance is opened and re-opened very frequently (like google or yahoo – or ajaxian even!). And at first glance, I feel much more comfortable using require as a dependency manager than using LabJS’s wait.
However its still a lot of code soup – and as anyone who develops serious webapps knows, keeping the JavaScript clean and readable is generally a bigger struggle than initial loadtime considerations – particularly if your product is going to be run for hours at a time and so can benefit from js file caching once the initial load has happened.

Comment by AngusC — March 18, 2010

@kriszyp – why I’ll use requireJS to load an async script, when as you say this job can be done via pure js code with no help of libraries at all?!

Comment by stoimen — March 18, 2010

@stoimen: Maybe a concrete use case would help? After loading the navigation container of my app, which doesn’t require much more than the base ExtJS library, the user loads the work orders module. Before that can render, it needs the overview component, which bundles a grid with some related details and aggregates panels and toolbars, which in turn can’t render without having loaded the grid classes, various grid and panel plugins, component translations, and so on… So, because all those dependencies are needed when the module is first loaded, it just loads them all up front, by dispatching a “load with dependencies” call to the loader class.
.
Managing all that stuff without a loader class is a nightmare. Especially when you take into account that the loader can load the work orders module, help desk module, and time registration module at the same time, and those can all concurrently load the same packages, but have different package dependency chains.
.
It’s one of those things that you can’t live without when you need it, and can perfectly ignore when you don’t need it.

Comment by Joeri — March 18, 2010

Just to clarify on the topic of RequireJS vs. LABjs as general script loaders:
.
I think the two libraries are orthagonal in some respects, because they’re designed for different purposes. First, let’s look at the simplified case, where RequireJS or LABjs are loading only one file, and executing only one callback:
.
require(["some/script.js"], function() {
//This function is called after some/script.js has loaded.
});
.
$LAB.script(“some/script.js”).wait(function(){
//This function is called after some/script.js has loaded.
});
.
In this case, I think it’s fair to say these two are roughly the same. They load one file, and execute one callback when that one file finishes loading.
.
Now, what LABjs was *really* designed to shine in is loading multiple files, from remote location(s), some of which have execution-order dependencies. For instance:
.
$LAB
.script(“jquery.js”).wait()
.script(“jquery-ui.js”).wait()
.script(“plugin1.js”,”plugin2.js”,”plugin3.js”)
.wait(function(){
// initialize the whole page now that all frameworks/plugins are loaded
});
.
Can that type of logic be executed in a RequireJS syntax, without any modifications to the source files? It’s unclear to me. Certainly it doesn’t appear this is what RequireJS was built specifically to do.
.
Moreover, the reason LABjs really shines in that above use case is because it loads ALL files in parallel, but it uses a variety of tricks to maintain execution order so that dependencies don’t fail in race conditions. Simply by adding the empty .wait() calls after “jquery.js” and “jquery-ui.js”, you’ve told LABjs to wait to execute what comes .after it until those two have respectively executed.
.
Of course, the caveat is that with LABjs, it requires you to know any execution-order dependencies at declaration time. For most sites, you’re at least somewhat familiar with them because you know you have to load jquery.js before you load jquery-ui.js.
.
The primary benefits then of LABjs, beyond having convenience of on-demand loading, are performance. Parallel loading and execution deferral tricks lead LABjs to achieve some of the most optimal load-time performance possible with script loaders, especially cross-browser.
.
To be fair, then, RequireJS seems to be focused on a different (but still valid) use case, which is primarily in loading a set of files who self-express their dependencies not at page-declaration time but at page-run time. This is a very different thing, and a challenge all its own.
.
Based on what I see so far, if you are working in a “module” environment where modules all express their inter-dependencies and where you don’t already have some build-process that can just concat those local files, then RequireJS seems very well suited to the task.
.
If on the other hand you need a generalized loader that can load files from anywhere, in parallel, and maintain simple execution-order dependencies only when necessary, and performance is at least as big a concern as convenience, then I humbly think LABjs is a little more well suited.

Comment by getify — March 18, 2010

You can count on getify and me adding a comment whenever JS loading is involved. ;-)

One issue is that RequireJS does not ensure scripts are executed in the order specified. Here’s a test page: http://stevesouders.com/tests/require.php

Comment by souders — March 18, 2010

RequireJS vs LabJS

http://groups.google.com/group/requirejs/browse_thread/thread/a44263737729d8ca

Comment by Les — March 18, 2010

@getify gives a good rundown with the difference between LABjs, and @souders is correct, it does try to ensure scripts are executed in the order they are listed.
.
The goal of RequireJS is to make a loader that can handle nested dependencies for files that indicate their dependencies. It can load existing scripts, but execution order is not guaranteed, only that require() dependency callbacks are called in order.
.
The goal is to try to bridge the modular development approach of things like CommonJS and Dojo but still allow scripts of today to function.
.
RequireJS also allows specifying text file dependencies, a format for i18n bundles and an optimization tool that can combine and minify JS and CSS without forcing to change your HTML (CSS optimizations are @import inlining and comment removal).
.
For normal jQuery use, I provide a modified jQuery that integrates RequireJS, so if you are just loading that version of jQuery via a script tag, then load plugins via a require() call and wait for jQuery ready to do app initialization, it should all work out and perform well. Unless the plugins have script dependencies and call those dependencies outside the jQuery.fn registration, the script execution order will not matter. It will be as fast as the equivalent LABjs usage, with the added benefit that you can use the RequireJS optimization tool to combine your scripts for deployment.
.
You can get ordered execution of existing scripts by nesting require calls, but as getify points out, that will be slower than the LABjs approach. A mitigating factor for RequireJS: you could nest the require calls for ordered execution in development, but then use the build tool so that once you are done developing, all the JS files can be inlined together. But development loading will be slower vs. LABjs.
.
So there are cases where LABjs is a better choice: existing scripts that need a specific order of execution.
.
For very modular development (which I include well-behaved/contained jQuery plugins), code with declared nested dependencies and projects that might want to reuse some CommonJS modules, RequireJS will be a better choice.

Comment by jrburke — March 18, 2010

If you want something specifically for jQuery, I’ve got a fast async loader for plugins over here: http://github.com/aheckmann/jQuery.use

Comment by aheckmann — March 19, 2010

@jrburke: There’s no reason why developers have to choose one or the other (preserving execution order vs modular development). You could embed the LabJS logic in RequireJS.

Comment by souders — March 19, 2010

@souders I am on the fence about the script type=”script/cache” approach that LABjs uses to get the load order to work. It means two passes of script tags, relies on good cache behavior in the browser, and I feel like modular development is the way forward. However, I will give it more thought, since it appears to work.

Comment by jrburke — March 22, 2010

@jrburke- i definitely agree i don’t like the hackery of the tricks used for preloading. Originally LABjs avoided those things. But, in collaboration with Steve, I found that the performance benefits of being able to preload scripts cross-domain (like google analytics and jquery, for instance) was just too great to ignore. Keep in mind, LABjs only uses the script mime hack in some browsers, and even then only cross-domain. It by default uses XHR for preloading local scripts. The script mime hack is the only known way in some of those browsers to reliably preload but defer execution.
.
It was a reluctant tradeoff between performance and hackery. but LABjs was primarily designed to amp up performance. And so it does that well.

Comment by getify — April 6, 2010

Just as a (long-after) follow up to this thread regarding LABjs and RequireJS, James and I wrote this Script Junkie article on LABjs & RequireJS highlighting all the things you need to know to decide how to use either.

Comment by getify — August 24, 2010

Leave a comment

You must be logged in to post a comment.