Monday, June 16th, 2008

modules.js: A New Stand-alone JavaScript Module Loader

Category: Ajax, JavaScript

Kris Kowal dropped us a line pointing us towards his year-long labor of love: modules.js, a stand-alone dynamic JavaScript module loader.

As the module-loading space is quite crowded at the moment (e.g., Google, Dojo, Yahoo, JSModule, etc.), we asked Kris to explain what makes modules.js different. He had some interesting things to say.

Existing Module Loaders Fall Short of Traditional Language Import Semantics

Kris started out his notes to us with an interesting point:

Ideally, you [would be able to] use import semantics similar to Java, Python, or even Perl [in JavaScript]. All [existing JavaScript module loaders] fall short in one way or another.

He then points out some ways in which modules.js meets this objective where others fail (along with an admission that some of his information about these other loaders might be a bit out-of-date):

Compared to JSModule

JSModule is the most similar [to modules.js]. It also uses XHR, eval, singleton
module cache, and scope injection. It’s also lighter. In general,
modules.js is what JSModule could grow up to be. JSModule is a bit
messy; it pollutes the scope chain that it implicitly passes to its
modules with its own internal variables because it uses eval in local
scope instead of global scope (I have a fun way around this problem).
JSModule also adds itself to global scope; modules.js cleverly avoids
this. modules.js includes modules using their URL relative to the
module loader URL or the module URL if it’s prefixed with a dot; this
permits easy migration and refactoring. JSModule also supports an
include path; when I explored that feature for modules.js, I
determined that searching for the script over HTTP performed very
poorly even though it provides a better approximation of how other
languages work server side.

Compared to the Dojo and Google Loaders

Dojo loader (last I checked) and Google’s loader both perform strictly
asynchronous loads and piggyback one layer of dependency management on
the onLoad handler. The key difference here is that module’s loaded
with modules.js can have recursive dependencies. In Dojo and Goog,
there is an important distinction between loading scripts in global
scope and in module scope. In global scope, you simply initiate an
asynchronous load of all the modules you want and attach an onLoad
observer that will be signaled when the page has loaded AND all of the
requested modules have loaded. However, onLoad may fire before all of
the modules required by your modules are loaded. To get around this,
in module scope, you have to write complicated dependency management
code to ensure that the scripts load in order: [your dependencies,
your continuation, the onload continuation]. This can be very
complicated and detracts from time spent actually writing features.

Google loader uses document.write to do script insertion which has two
advantages over modules.js: it prevents script flicker on pages with
long DOMs, and it can load cross domain scripts, which is handy if
you’re loading standard libraries from their cache.

Compared to YUI Loader

YUI Loader uses a static dependency table to ensure that YUI modules
are loaded in the correct order. This means that third party modules
have to monkey patch this table if they have inter-module
dependencies. It’s similar to modules.js in that it uses XHR.

A Note about Global Scope

YUI, Goog, and Dojo all evaluate modules in global scope. modules.js
protects global scope with an enclosure and provides developers with
the ability to share functions without adding them to the transitive
global scope. Instead, you add them to the module object and import
them into your imported object’s scope. Nothing ever /needs/ to be
added to global scope, and developers are protected in many ways from
accidentally “sharing” variables with other modules on global scope.

In conclusion:

It’s not a solution for everybody, but it’s a good solution for many
people.

Frankly, the community sees a lot of new Ajax libraries written by folks who don’t appear to have familiarized themselves with the state of the art before diving in. While I have respect for anyone who takes the time to contribute to the community, it’s refreshing to see a craftsman at work. Nice work, Kris!

Posted by Ben Galbraith at 8:00 am
14 Comments

+++--
3.9 rating from 41 votes

14 Comments »

Comments feed TrackBack URI

How many people use a loader? What are the arguments for using one rather than just having a clump of script tags do the work?

Comment by Nosredna — June 16, 2008

Only one thing, when using YUI Loader is
loader.addModule({
name: “MyLibrary”,
type: “js”,
fullpath: “http://…/my_lib.js”,
after: [“base”] // or any other lib
});
I think this is not monkey patch … you shoud better check twice, then you write somethink like this. Also statement about using global scope is misguided.

Comment by Bredy — June 16, 2008

Only one thing, when using YUI Loader is
loader.addModule({
name: “MyLibrary”,
type: “js”,
fullpath: “http://…/my_lib.js”,
after: [“base”] // or any other lib
});
I think this is not monkey patch … you shoud better check twice, then you write somethink like this. Also statement about using global scope is misguided.

Comment by Bredy — June 16, 2008

Hrmm….

It seems the author of this system (which looks like good work) didn’t really take any care to investigate Dojo seriously before inaccurately characterizing it. Dojo’s package system DOES handle recursive dependencies but ALSO provides async module loading after pre-processing, but that mode still handles recursive dependencies. That allows the Dojo package loader to work in CDN environments without serious changes to existing code other than the addition of provide()/require() statements.

Furthermore, in addition to his inaccurate statements about the basic function of Dojo’s package loader, he’s also wrong about the way that Dojo manages onLoad handling. The Dojo system doesn’t fire handlers registered with dojo.addOnLoad() until all dependencies have been satisfied, both in the local (XHR) and x-domain cases. Dojo’s package system ALSO integrates with Dojo’s build system allowing you not only to be able to easily load modules but also to coalesce/optimize them for deployment without code changes.

It’s great to see other toolkits picking up features which have been in Dojo for years now….hopefully Kris will take another look at Dojo and talk more accurately about it in the future.

Regards

Comment by slightlyoff — June 16, 2008

Well, again something Archetype’s loader handles better, and we’re not even cited :(

Full loading process, including a final code starter, recursive configuration, etc. But in Archetype, it’s just 10% of the framework and it’s way more practical to use dependency management in components thant just registering dependencies in a files between libraries!

If you’re not convinced, check out the 0.8.0-beta we’ve released some hours ago : http://sourceforge.net/project/showfiles.php?group_id=194362

http://archetypejs.org

@ Kris : it would have been great to get you in our team ;)

Comment by temsa — June 16, 2008

Does eval work in Firebug yet? I know it doesn’t in 2.0. Not being able to use it is a serious limitation and why having asynchronous loading is very useful.

@Nosredna In a production system, you really shouldn’t have a clump of script tags anyway. Your application should be packaged, and maybe event compressed, in one file. However, the Module loader would be useful in cases where you have a VERY large JavaScript application, where much of the application’s code isn’t used immediately.

As this post seems to be filled with people promoting their own technology, I might as well throw JavaScriptMVC’s hat into the ring.

With JavaScriptMVC, you can’t do loading like this, but it’s not important because the compression is so easy. Even easier than … DOJO. You don’t have to make code changes, but you have to write extra code. Take that Alex. Bwhahaha.

Btw, I hope you can tell I’m joking. Dojo’s got it all. I couldn’t have done the new compression system without ShrinkSafe.

Comment by JustinMeyer — June 16, 2008

@Nosredna The argument to using a loader is scalability and maintainability. With a module loader, you can write more discrete and simpler scripts, so where managing a clump of includes (which, for performance reasons, you’ll need to aggregate into a single at some point anyway) as a strategy becomes untenable, with a module loader you can continue writing and requiring discrete modules for a long time before the burden of managing their dependencies becomes a problem.

@Bredy my apologies for the cursory examination of YUI loader.

@slightlyoff I read Dojo’s loader a year ago and found a lot of inspiration there. It has had a major release since then, so some things have changed and I never fully grasped the complexity of the system. My comments about the addLoadListener were rather attributed to the Goog library, which was also inspired by Dojo.

@temsa Archetype is interesting. However, it still doesn’t get you the simplicity of include(“module.js”) or require(“module.js”) (which imply dependency, are loosely coupled from each other, and mnemonically map to analogs in other languages readily). I will be giving it a more thorough examination and will probably contact you or blog about it.

@JustinMeyer You raise a valid point. I get much less useful debugging information out of Firebug with modules.js. However, the module loader includes a debugger module like Firebug-Lite that can be opened in any browser on any web page running modules.js. The debugger console also runs in a module, so you can do “include”, “require”, and logging calls from the command line. The module loader also marks up functions with module origin information and adds logging functions to the module scope so It’s easy to add trace messages to your code, like log(“Found: ” + repr(object)). I also make an attempt at rethrowing exceptions with adjusted file names and line numbers, but I’ve had mixed results and could definitely improve on these points.

Comment by Kris Kowal — June 16, 2008

@Kris: have a look at Component system dependencies, this is no more than about saying dependencies : [“foo.bar.barfoo”, “bar.foo,foobar”] in the object

And this system doesn’t stop to js loading, but handles also templates and css loading associated to the component ;)

Comment by temsa — June 16, 2008

Sorry for double posting, clicked too soon on the submit comment button :P

@Kris, I’ve forgot, you can “include” as well files in archetype using a Archetype.require with some params for it to be synchronous (it’s asynchronous by default with a callback, because synchronous will freeze your browser during the load else).

For the logger system, you should look at nitobibug which is quite amazing, we plan to introduce it in one of our default logging system for the next release.

@JustinMeyer : get at least a 1.1 firebug version and you’ll have it. However, for cross site js loading, using xhr is not a good and it can be desirable in order to load your js application faster when you’ll want to put it in production (using static1.js.mydomain.com, static2.js.mydomain.com, etc. in order to “override” the 2 request per server limit of http protocol …).

As you said, javascript compression is easy… but is very unhelpful for debugging, that’s why providing an api like Archetype provides a good to handle both development and production, using or not compression a cross site loading!

Comment by temsa — June 16, 2008

What if the clump of files is my js files, plus a few things coming in from, say, google’s ajax library servers.

Obviously, I don’t want to concat my js with what google is serving.

So is module loading better if what I’m bringing in is coming from google’s servers?

Comment by Nosredna — June 16, 2008

@Noresdna: you certainly won’t use “module loading” throught XHR on a different server, except if you find a good way to do cross server XHR easily…

Comment by temsa — June 16, 2008

OK. I’m already merging the js I serve. So I guess I ignore all this module talk.

Comment by Nosredna — June 16, 2008

Or you can use Archetype to do this elegantly, efficiently, without relying on XHR ;)

Comment by temsa — June 17, 2008

@temsa: Maybe a short presentation with simple examples?

Comment by cheba — June 18, 2008

Leave a comment

You must be logged in to post a comment.