Tuesday, June 30th, 2009

LABjs: Simple abstraction for loading dependencies correctly

Category: JavaScript, Performance

<>p>Kyle Simpson has developed LABjs, a library that lets you define your JavaScript file dependencies, and then loads them as efficiently as possible.

Kyle told us:

This project is a simple little tool (1.6k compressed!) for being able to load javascript files dynamically. It’s like a lot of similar projects where the goal is to improve the speed of page load by allowing scripts to load in parallel. The thing it does slightly differently than most others like it is it allows you to “block”, which is to say, load one or more scripts in parallel, then wait for them to finish, before going on to something else, like loading more scripts.

What I wanted was a pattern where I could load scripts in parallel, just like with script tags, but also block and wait if there was an explicit ordering dependency that required it.

What most loaders fail to do well is let you define “dependencies” simply based on loading order. With regular script tags, the browser blocks for you, so you can make sure for instance that jquery.js loads before jqueryui.js. But imagine you’ve got 3 scripts that can download in parallel (not dependent on each other), and then two more that need to wait for those 3 to load. You can’t do that with script tags, and you also can’t do that very easily with a lot of the script loaders/frameworks that I’ve found.

Most of them rely on intrusive concepts to do “dependency” management. For instance, each child script has to “signal” (callback) that it’s done loading, to the parent page. Or the parent script and child scripts have to explicitly declare dependencies using some framework or conventions. Also, some other loader libraries rely on attaching a single load callback handler for EACH script. This makes it awkward or difficult to wait for several to load at a time, before proceeding, since you as the author have to keep track of what has loaded yourself.

jsLAB lets you load pretty much any script file, whether you control it or not, with no intrusion or convention for dependencies, other than the order and blocking that you define. It keeps track of what you’ve asked for and what has downloaded, only loads a unique script filename once, and lets you only define your handler once for a set of scripts that will load together in parallel. The API style (with chaining) makes is very easy to convert a set of script tags in your page into code to load them, without having to worry that race conditions will cause issues for scripts loading in the wrong order if there are implicit dependencies involved.

Example

Old:

  1. <script src="jquery.js"></script>
  2. <script src="jquery.ui.js"></script>
  3. <script src="myplugin.jquery.js"></script>
  4. <script src="initpage.js"></script>

New:

javascript
< view plain text >
  1. $LAB
  2. .script("jquery.js")
  3. .block(function(){
  4.       $LAB
  5.       .script("jquery.ui.js")
  6.       .script("myplugin.jquery.js")
  7.       .block(function(){
  8.             $LAB.script("initpage.js");
  9.       });
  10. });

In the above example, “jquery.ui.js” and “myplugin.jquery.js” can load in parallel because there’s no dependencies, but they will wait for “jquery.js” to load first, since they depend on it, and then “initpage.js” will wait for all of them to load before it runs, to it makes sure all code it will call is in place, similar to a $document.ready(…) concept.

The page link above also shows a few other variations on the .script(…) signature. For instance, you don’t have to do a single script() call for each file (though I think it makes thing more readable). You can pass as many scripts singularly as parameters to one script() call. You can also pass an array of scripts, and it will loop through them and load them in the same way. Lastly, you can pass in an object instead of string, and the object literal can contain “src”, “type”, and “language” specifications, if you want to override the defaults of “text/javascript” and “Javascript”, for some reason.

Related Content:

Posted by Dion Almaer at 6:16 am
24 Comments

++++-
4 rating from 40 votes

24 Comments »

Comments feed TrackBack URI

I could be mistaking but I see a IE memory leak pattern in the code. You are assigning a closure function to the onload event handler of the script element. So run it though sIEve on IE 6 XP SP1. Also the ready state “loaded” is not the same thing a “complete”. I’ve been meaning to blog about this since many other similar script loaders has that same issue. The “loaded” state is normally fired when the JS is loaded but it might not be parsed if there are large scripts or if something is hogging the CPU the loaded state will be fired before the script gets parsed and executed. This could result in errors if the scripts relay on each other.

Comment by Spocke — June 30, 2009

The way I solved this in my own on-demand loader is to allow the user to pass in a function that checks whether the load is finished or not. It will listen to the load event, run the function, and if the load still didn’t complete, it will start periodically checking whether the script completed. This is very convenient for scripts that themselves in turn want to load some code on-demand before they initialize themselves.

Comment by Joeri — June 30, 2009

@Spoke — the memory leak concept is probably a good catch. Will be easy to simply de-assign the handler from the script tag after it fires. I’ll update the code accordingly.

As for the loaded vs. complete state, in my testing experience, the difference between the two states (in IE) was whether the script was coming from cache or not. The example page demo tests with 3 files (“load.js”, “load2.js”, and “load3.js”), which are all identical files, but they are each over 300kb. So the demo is testing loading 900kb worth of script in 3 chunks, and it works in all my test scenarios. I think this is pretty good evidence that the loading detection is pretty robust. If a browser misbehaves randomly based on system CPU load, I’d consider that a pretty difficult corner-case to solve for. I was mostly just concerned at getting the code to handle regular loading use cases properly, and it seems to do so.

Comment by shadedecho — June 30, 2009

@Joeri — I found that I needed a more agnostic library to do general script loading. In other words, I didn’t want to have a library that required it, or the author, or especially the loaded script, to have any special knowledge of each other. When you’re dealing with a tight framework of modules, you can impose convention and tight integration to inform of load, especially of child dependencies. But, that doesn’t work well for a general loader that should be able to handle ANY script, which was the express goal of this library.

Also, on the topic of the scripts themselves having child dependencies to load: The primary use-case for this tool is to replace your normal use of script-tags in your markup. It can be pushed to facilitate dynamic loading of resources at any time.

But in either case, especially with script tags, if an author loaded a script file that had other dependencies to load, that main script file would already have needed to handle async/delayed initialization, and ideally it would abstract that from the author ever needing to know. If the script wasn’t that smart, and the author would have needed to do something special to “wait” for initilization, the SAME would be true when using LABjs. So I consider your point to be valid but moot with respect to LABjs.

It’s certainly not a fully functional, do everything loader. But it’s a very tight, simple way to load scripts, and block on loading them before doing other things like loading more scripts. Beyond that, it tries to be very agnostic of what goes on inside the scripts and leaves that as an exercise for the author to handle.

Comment by shadedecho — June 30, 2009

If you’re going for performance, then why wouldn’t you combine all js into one file?

Comment by WillPeavy — June 30, 2009

If you’re going for performance, then why wouldn’t you combine all js into one file?

Maybe one is pulled from a CDN, others you host yourself, others are dynamically created on your server, etc?

Comment by youngestlinton — June 30, 2009

@youngestlinton exactly, you often times don’t have control over every JS file you load on a page. Take for instance my site, http://flensed.com I’ve got JS files from google-analytics, addthis, the jquery from the CDN, and then my own scripts. I combined all my own scripts into one, but I still have like several that can load in parallel. So, I installed LABjs on the site, and the load time was improved by over 200%.

Comment by shadedecho — June 30, 2009

Compiling javascript into a single js file, ok.

But the trend I see these days, is that “not so smart” developers compile frameworks like jquery into the same file. What’s that? “Best practise”??? wrong… It might work for initial deliveries of websites, but it all goes to hell in a year or so, when several third parties all delivered “compiled” libraries and you end up with the browser evaluating 3 jqueries, 5 prototypes, 3 versions of THEIR OWN framework (true story btw) and so on on a single page.

People that compile frameworks together with their own code should be shot!

Thank god some people DO get it and create libraries that modularises javascript.

Tbh. why aren’t browser vendors, w3c and ecmascript working on these kind of solutions… Or are they?

Comment by BenGerrissen — June 30, 2009

ps. My ramblings are mainly about enteprice websites where customers keep hiring third parties (designers/advertisers) and we have to install their code. Since customers don’t pay us to refactor javascript, we place it as is.

If you manage your own websites and/or rarely work with third parties creating web applications, I’m sure you will have no to little issues.

Comment by BenGerrissen — June 30, 2009

This is interesting, but I think forcing the developer to do their own management of blocking vs. non-blocking dependencies is overkill.

One of the nice things about the loader in Objective-J is that it does a look ahead phase and starts downloading any pending imports immediately, but always executes all import statements in the correct order. You get the best of both features, without needing any additional management on your own.

Comment by rboucher — June 30, 2009

@rboucher — strange, I wouldn’t have thought of it as “forcing” them to do blocking on dependencies. This tool was intended to replace use of script-tag soup in the HEAD or BODY of your document. In those scenarios, authors already have to load scripts in the right order, and rely on the browser to “block” so that they load dependencies correctly. LABjs simply lets you speed up the parts of that set of script-tags which aren’t really dependent on each other and can be parallel loaded.

It should also be noted that this approach allows the rest of the page to keep loading, as well, instead of waiting on ONLY the JS to load.

I don’t think it’s too hard for web authors to think “Oh, jquery.js needs to load before all the plugins, duh!. So, I’ll first load it, and then block() and then load the rest in parallel”. If that is too hard or forcing them unnecessarily, then I’m not sure other heavier approaches will be palatable.

Interesting thoughts on Obj-J approach. I’ll look closer at it. Again, LABjs can also be thought of as a general, simple dependency loading/management tool, though its primary intended use is just to replace script-tags in a better, more efficient way.

But that’s just it, it’s very simple, JS only, doesn’t work on any build-time steps, or require any heavy frameworks. It also is completely agnostic to what you are loading, so that you don’t have to modify or inspect your scripts to get special convention-based loading signals.

Comment by shadedecho — June 30, 2009

@rboucher : forcing the developer to manage the loading is not that bad, at least for me, being lazy , is more easy to write .script[...], than , even with copy/paste.
Anyway you are forced to set what’s blocked and what not setting the order of the scripts
On the other hand, on a project I’m working on, I load 280k, in 8 files – developer versions so no minified – in 8 seconds.
with labjs, this is done in 2.5 seconds, so I’m pretty excited.

Comment by rborn — June 30, 2009

@willmofatt — LABjs has been tested, and seems to be working, in windows, with IE6, IE7, IE8, FF2, FF3, FF3.5, Safari 4, Opera, Chrome — I’ve not done full testing on the other platforms (MAC, Linux) yet, but that’s up next. In any case, that’s pretty wide coverage so far in terms of windows browsers.

Comment by shadedecho — June 30, 2009

Awesome code! 5*

Comment by oopstudios — July 1, 2009

@shadedecho – was loading multiple files with LABjs faster than a single compiled file?

@BenGerrissen – I design enterprise apps, that have mutliple js files that are compiled into one file before they reach prod. It’s fast, and it works well. I wonder why you aren’t defining the API for 3rd parties. If you let them define their own, of course you are going to be frustrated…

Comment by WillPeavy — July 1, 2009

@willpeavy — LABjs was designed mostly for sites who use JS that they don’t control, and sometimes don’t even host. Like most sites on the internet, for instance, who link to jquery.js from Google AJAX Api hosting, and google analytics, and addthis share button JS, and so forth. In those circumstances, you often end up with half a dozen JS files loading one script-tag at a time (often horribly in the HEAD of the document), blocking the rest of the page assets from loading.

In these cases, you cannot combine files. On my sites, I’ve now put LABjs into play, by loading in parallel all these external scripts, and *also* combined my own scripts into one or a couple of files at most, to reduce the amount of loading.

Also, on the topic of combining… It’s a good approach in general, but can quickly lead to problems if the file is big and any part of it has change at any relatively medium to high frequency. In those cases, the users have to redownload an entire new file because you tweaked one line of JS. I think there needs to be a happy balance of combining files, and still utilizing (as much as is possible) the power of the browser cache to keep average viewing user experience optimal.

In any case, rare is it that a website can (or should) have ONE js file on a page, usually more. And sometimes they have dependencies. That’s where LABjs comes in, making it easy to load those, manage dependency blocking, and still let the rest of the page load quickly in parallel.

And what about your one JS file? Are you loading that with a regular blocking script tag? Odds are, if it’s “big”, and you’re so in the HEAD of your page, you’re not getting a lot of optimization benefit. A single call to LABjs for your one file would still improve things because you would let the rest of the page load while your big mammoth file loads.

Comment by shadedecho — July 1, 2009

@WillPeavy, yeh, that works, but in most java joints doing international enterprise websites at low budgets usually means frontend management gets cut off a lot and done away with *just copy paste*…

*sigh* *depressed*

Comment by BenGerrissen — July 1, 2009

SHORT: This library is nice, but could be better.

LONG: It’s sad that developers have to do so much work to get around something browsers should do automatically. Even the newer browsers that support parallel script loading (IE8, FF 3.5, Saf 4, and Chr 2) still block other, later resources from being downloaded. I’ve shared my thoughts with some folks involved in HTML5 on my proposal for the SCRIPT tag to handle DEFER, ASYNC, and POSTONLOAD attributes. (I’ll write a blog post about that soon.)

Back to LABjs, let’s revisit the goal. I totally understand that external scripts that depend on each other have to be *executed* in order, but there’s no reason they can’t be *downloaded* in parallel. We’ve seen newer browsers move in this direction, but as I mention in Chapter 4 of Even Faster Web Sites and in my blog post on Loading Scripts Without Blocking, they still block later downloads. Until browsers get better at loading scripts in parallel (and IE 6&7 disappear), developers can use advanced script loading techniques to make their pages faster.

The issue with LABjs is that, when a developer has two scripts that depend on each other (as is often the case when using frameworks like jQuery and Dojo), LABjs requires that the developer use the blocking functionality. This defeats the original goal of loading scripts in parallel.

There is a workaround to this that is compatible across all browsers. I describe it in my book, but the code sample is available without buying the book. The loadScripts Different Domains sample contains the JS implementation to do this. It allows two scripts that depend on each other to be loaded in parallel but executed sequentially. It works in all browsers except Safari 3 and Chrome 1.

Hats off to Kyle for pushing the envelope and making this library available. It would be great if he took it to the next level to support parallel loading of scripts with dependencies, avoiding the blocking behavior that is the motivation behind this work.

Comment by souders — July 1, 2009

@souders — thank you for the kind comments! I’m impressed by and still trying to learn the ins and outs of your code. If I figure out how to leverage more stuff I learn, I’ll certainly keep trying to push LABjs to be even better.

I think and hope it’s a good, simple, first step to start getting more and more regular websites to improve the performance of their script loading. IMHO, it’s badly needed. My tool is definitely intended for “the masses”. :)

Comment by shadedecho — July 1, 2009

@shadedecho – I agree with @souders… this library could really take it to the next level with paralell downloading all files and then executing based on dependency.

Comment by danroberts — July 2, 2009

@danroberts-
I have now read @souders’ second book, “Even Faster Web Sites”. It is great, and I highly recommend it to everyone.

So, now that I have done some further investigation and digging into his code (Ch4 and Ch5), I have some thoughts/comparisons with it versus the direction I took with LABjs.

Short:
Souders’ method, which he has named “EFWS” (name of his book “Even Faster Web Sites”) is really awesome, but has some admitted limitations which run contrary to some of LABjs’ motivating factors. In addition, I think depending on certain conditions (which I will discuss below) LABjs may achieve similar performance in general with EFWS. So both scripts stand on their own as good strong choices for web authors, but for different reasons.

Long:
I’ve written a more detailed evaluation comparison here: http://labjs.com/LABjs_vs_Souders.html

My Conclusions:
My guess is there are some sets of pages out there which have a couple of very proportionally large script assets compared to few numbers of other smaller page assets (images, CSS, etc), and it’s possible that in those cases Souders’ method may squeeze a little more performance in total page load.

But there are also plenty of sites which the balance would be more swayed the other way, with more (or larger) page assets, and compartively smaller (or fewer) numbers of script assets, in which case I think LABjs might perform better.

So, I highly respect the work that Souders has done, and I encourage others to take a look at his code, even try it out on your own pages.

But I think LABjs serves a slightly different audience and use case. The code is smaller and simpler, and is also more flexible to be used at any time during the life of a page. This gives page authors a lot of great choices they didn’t have before.

I hope this helps clarify to those reading the choices they have available to choose from, and why they might look to LABjs and/or Souders’ EFWS method for loading JS into a page.

Comment by shadedecho — July 2, 2009

New 0.6 version of LABjs posted, fixes several bugs, thanks to community input!

http://labjs.com/LABjs-0.6.zip

Comment by shadedecho — July 6, 2009

Thanks to questions and input from Austin.Javascript members, I’ve released LABjs v0.7 now, which is a significant reworking of the internals of LABjs, providing a much simpler and yet more powerful API for usage.

No more need to nest $LAB calls for loading scripts after blocking.. simply continue the main chain of calls and LABjs will manage the loading order/execution for you.

If you haven’t checked out LABjs yet, you really should give it a try. It’s simply the best way to load JS onto your pages.

Comment by shadedecho — July 28, 2009

it is hiding controls in IE. check below code

Calendar Control

function LoadJS(obj) {
$LAB.script(‘JS/jscal2.js’).script(‘JS/en.js’).block();
alert(‘hi’);
}

<!–
–>

Comment by celalit — October 10, 2009

Leave a comment

You must be logged in to post a comment.