Friday, June 6th, 2008

A Technique For Lazy Script Loading

Category: JavaScript, Performance

Bob Matsuoka has written a guest article on the topic of lazy script loading. Thanks so much Bob!

A recent article “Lazily load functionality via Unobtrusive Scripts” discussed how to lazily load Javascript script files by appending script elements to the HEAD tag.

While this works as expected, I’ve found that for best results, you should also consider tracking which scripts have been loaded in order to prevent re-loading an already loaded script, and more importantly supporting callbacks so that you can guarantee loading of scripts prior to calling functions that depend on that code.

NOTE: The example loader script, which has been tested in FF, IE, Safari, and Opera, uses prototype.js for DOM and array routines. I developed this originally for a project that already had prototype.js available, but it uses it only superficially. It should be simple to remove these references if you’re not using prototype.js.

javascript

  1. /**  
  2.  *  Script lazy loader 0.5
  3.  *  Copyright (c) 2008 Bob Matsuoka
  4.  *
  5.  *  This program is free software; you can redistribute it and/or
  6.  *  modify it under the terms of the GNU General Public License
  7.  *  as published by the Free Software Foundation; either version 2
  8.  *  of the License, or (at your option) any later version.
  9.  */
  10.  
  11. var LazyLoader = {}; //namespace
  12. LazyLoader.timer = {};  // contains timers for scripts
  13. LazyLoader.scripts = [];  // contains called script references
  14. LazyLoader.load = function(url, callback) {
  15.     // handle object or path
  16.     var classname = null;
  17.     var properties = null;
  18.     try {
  19.         // make sure we only load once
  20.         if ($A(LazyLoader.scripts).indexOf(url) == -1) {
  21.             // note that we loaded already
  22.             LazyLoader.scripts.push(url);
  23.             var script = document.createElement("script");
  24.             script.src = url;
  25.             script.type = "text/javascript";
  26.             $$("head")[0].appendChild(script);  // add script tag to head element
  27.            
  28.             // was a callback requested
  29.             if (callback) {            
  30.                 // test for onreadystatechange to trigger callback
  31.                 script.onreadystatechange = function () {
  32.                     if (script.readyState == 'loaded' || script.readyState == 'complete') {
  33.                         callback();
  34.                     }
  35.                 }              
  36.                 // test for onload to trigger callback
  37.                 script.onload = function () {
  38.                     callback();
  39.                     return;
  40.                 }
  41.                 // safari doesn't support either onload or readystate, create a timer
  42.                 // only way to do this in safari
  43.                 if ((Prototype.Browser.WebKit && !navigator.userAgent.match(/Version\/3/)) || Prototype.Browser.Opera) { // sniff
  44.                     LazyLoader.timer[url] = setInterval(function() {
  45.                         if (/loaded|complete/.test(document.readyState)) {
  46.                             clearInterval(LazyLoader.timer[url]);
  47.                             callback(); // call the callback handler
  48.                         }
  49.                     }, 10);
  50.                 }
  51.             }
  52.         } else {
  53.             if (callback) { callback(); }
  54.         }
  55.     } catch (e) {
  56.         alert(e);
  57.     }
  58. }

Download the full source and example project.

Tracking Loaded Scripts

A common use of a lazy loader is in conjunction with “require” type function that allows you to specify which scripts are needed for a particular script to execute. Since library scripts are often called by more than one script, I thought it important to allow my lazy loader to track which scripts have already been loaded on a page to prevent unnecessary re-loading.

In this example, I’ve created the LazyLoader.scripts array. As each script is called, this script src is tested against the array of scripts already called, and if it exists, the script is not re-loaded, but its callback is executed. This allows you to call any required script as often as needed with impunity.

Supporting Callbacks

A more important addition is support for callbacks. It is my experience that with anything more than the simplest scripts, you cannot guarantee that a script is available for use unless it is called as part of a callback tied to the loading process. Unfortunately most of the browsers handle script onload events slightly differently. The examples I’ve provided should work for Firefox, Safari, and IE.

The basic logic for supporting callbacks is to allow a closure to be passed to the lazy loader. The function is then bound to the script event triggered by its loading. FF and Safari 3 support the “onload” event. IE supports the onreadystatechange event, and requires further testing of the state for either ‘loaded’ or ‘complete’ (depending on whether the script is cached).

Supporting callbacks in Safari 2 and Opera require a slight wrinkle, since neither support “onload” or “onreadystatechange” script events. For these browsers, you need to set a quick interval script to test document.readyState for “loaded” or “complete”. Once ready, the interval can be cleared and the callback executed (I use a different interval for each script loaded, but I’m not sure that’s necessary, since we are only testing the document object, not the state of the individual scripts).

Calling The Loader

Calling the loader is straightforward. This implementation is setup as a static function using a namespace. Here is an example without a callback:

javascript

  1. LazyLoader.load('js/myscript1.js');

Here is an example with a callback:

javascript

  1. LazyLoader.load('js/myscript2.js', function(){
  2.     var myobj = new MyObject('myobj');
  3. });

In the second example, the “myobj” instance of MyObject is only created after js/myscript2.js is loaded. You can also daisychain loaders by including them in the callback function.

Conclusion

I’ve used this technique for nearly a year with good results. We have a very large library of objects and functions that are only needed for specific pages that share the same “layout”, and this allows us to call the scripts properly (in the HEAD tag, as opposed to including script references within the body of the page) and cleanly. We use this function in conjunction with a “require” function to script dependencies can be shown cleanly.

A script loader also works very nicely when used in conjunction with form-based script loading, which is a technique we use to declaratively reference script objects and bind them to HTML forms, as well as pass in server-side variables. I will discuss this in a follow-up article.

Posted by Dion Almaer at 10:00 am
33 Comments

++++-
4.2 rating from 51 votes

33 Comments »

Comments feed TrackBack URI

Its nice to see people thinking in the same way – keep up the good work Bob :-) … and give this prototype system a look, it handles js, css and html file requirements recursively and asynchronous:
http://www.two-birds.de

Comment by FrankThuerigen — June 6, 2008

Yeah, thanks for claiming this as *your* lazy loader script. Code to do just this already exists in the open source UIZE Framework. The Uize.module declaration allows a *required* parameter to specify all the modules that are directly required (ie. won’t themselves be deferred loaded) by the module you’re defining. The *builder* parameter, which lets you specify the code that will build the module, only gets executed when a module’s dependencies have been satisfied. A *Uize.moduleLoader* static property lets you define a custom module loader function – the default is to load via adding script tags to the document.
 
http://www.uize.com/explainers/all-about-modules.html
 
PS: It’s good that you’re discovering things that others have already discovered. Sorry to be a bitch, but it would be nice if some people recognized some of the goodies that await them in the UIZE Framework, with the help of engineers at Zazzle (although, everyone’s welcome to join in).

Comment by uize — June 6, 2008

*developed* with the help of engineers at Zazzle, that is. D’OH!

Comment by uize — June 6, 2008

Supporting callbacks in Safari 2 and Opera require a slight wrinkle, since neither support “onload” or “onreadystatechange” script events

Really? Try this in Opera:
data:text/html;charset=utf-8,

Comment by dbloom — June 6, 2008

Erm, trying this again…
data:text/html;charset=utf-8,<script onload=”alert(‘onload!’)” src=”data:text/javascript;charset=utf-8,alert(‘this is the script.’)”></script>

Comment by dbloom — June 6, 2008

Bummer that it requires Prototype. Anyone strip out the Prototype requirements or convert to Dojo or jQuery yet?

Comment by Nosredna — June 6, 2008

I don’t know what it is, but ajaxian.com is super-slow to scroll in FF… could be the flash, I don’t know.

Comment by Anonymouse — June 6, 2008

@uize, I didn´t mean to pick on Bob. Its just that I like it if people seem to come to the same conclusions that came to. The more people work on this the more ideas will evolve from it.
BTW I started this lazy loading stuff about 3 years ago when there was no firebug around – I have no idea whether I did it first neither do I care at all. I simply like that people work on this too.

Comment by FrankThuerigen — June 6, 2008

If anyone has interest, I would be willing to post a version of the module declaration mechanism that is factored out of the Uize base class. Just need the Ajaxians to let me contribute a posting on the subject.

Comment by uize — June 6, 2008

Mh, looks like the MooTools Asset.javascript(url, {onload: fn}) … including helpers for loading images and stylesheets. But it has no check, if its loaded already.

Comment by Harald — June 6, 2008

The YUI also has a Utility like this which goes further by also allowing for CSS files and purging old files (say when you get a json feed):

http://developer.yahoo.com/yui/get/

You can get that from the CDN hosted versions and not have to include the 124KB of prototype (or use the google hosted one) which somehow defeats the purpose in terms of performance.

Comment by Chris Heilmann — June 6, 2008

The approach used here, createElement(‘script’), is a good approach for lazy loading scripts, except that in FF it triggers the browser’s busy indicators (status bar, progress bar, hourglass cursor, etc.). If someone truly wants to “lazy load” a script it should be done in a way that doesn’t hinder the user’s perception of the page being ready. Another aspect of this approach is that in FF it preserves the execution order of the scripts added. If you want to lazy load multiple scripts that don’t have code dependencies, it would be faster to use a technique that executes scripts as soon as they return.

I presented six techniques for lazy loading scripts at Web 2.0 Expo SF and Google I/O. The slides can be found on my web site ( http://stevesouders.com/ ). There’s a great table in the slides that summarizes the behavior of the six techniques. If you want to load a script in the background the createElement(‘script’) approach is good for IE, but for FF it would be better to use XHR Eval or XHR Injection, as long as the script is on the same domain as the main page. If the script is on a different domain, the createElement(‘script’) is the right choice.

In that slide deck I show a decision tree that recommends the best technique given a few input parameters: is the script on the same domain as the main page, do you want to avoid triggering the browser busy indicators, and if loading multiple scripts is it necessary to preserve execution order. It would be great if this logic could be incorporated into the lazy loader. Also, I think the most useful implementation of this is on the backend (PHP, Python, etc.). The web developer could call a PHP function like loadScript(url, bTriggerBusy, bEnsureOrder) and the backend framework would use the appropriate technique in the HTML page for loading the script. This would reduce the amount of lazy-loading JS code needed in the browser.

I analyze the performance of a lot of web sites (both inside and outside of Google) and the way scripts block all other downloads and rendering is one of the biggest issues that slow down web pages today. It’s great that Bob has published this. I’m going to contact him now about the possibility of incorporating some of this additional logic.

Comment by souders — June 6, 2008

my synchronous version that allows you to call functions in scripts that haven’t been loaded: http://www.webreference.com/programming/javascript/mk/

Comment by cwolves — June 6, 2008

Great work, we’ve had this natively in Gaia for more than a year though. It works completely abstracted away too so when you show or create a widget on the Server Side the widget itself will track which JS files to include and every piece of code that references that widget in the return value back to the client will wait until the JS files included in that callback are finished loading. But I guess this is good news for those not using Gaia ;)

Comment by polterguy — June 6, 2008

@uize, and @polterguy you guys are constantly plugging your frameworks, it is ridiculously obvious.

@uize Unless you know it is ripped and can provide a link other than plugging your own framework, say so… otherwise don’t even hint at the fact that it isn’t their code, because you don’t even know.

Also, “LazyLoading” has been around forever, so claiming that you were the first is silly. I’d bet the first implementation was even before your framework was even written.

Comment by Tr0y — June 6, 2008

Also, dojo.require can load a Javascript module from the appropriate URI. They had this feature available for a long time.

Comment by Les — June 6, 2008

First version was written by Dan Pupius (now of Gmail) back in 2001, adopted by me later on in the f(m) toolkit (2002-ish), and has been a staple of Dojo since the get-go.

Before that, library loading was done in JS by Dan Steinmann with the DynAPI, which was aimed at Netscape layers and IE divs.

I have to agree with Tr0y: it gets a little old when commenters see a decent implementation of a particular technique, jump on it with a less-than-friendly-tone, and then attempt to plug their own work in the process. It makes it so that its almost not worth reading the comments here anymore.

Almost.

Comment by ttrenka — June 6, 2008

I should add that reading the articles can be an invaluable resource for anyone interested in JS techniques, such as this one.

Comment by ttrenka — June 6, 2008

@Tr0y
Sorry… I wasn’t suggesting for a second that any code or idea was *ripped*. There’s obviously been a ton of great thought on this subject by a number of smart people, such that it’s futile trying to claim, suggest, or even hint at ownership. This is a great forum for us all to learn from one another’s advances.

Comment by uize — June 6, 2008

@ttrenka
Thanks for the historical info.

Comment by uize — June 6, 2008

@uize
The logic in this is entirely my own, though I’ve obviously built on the work of others. In fact, this article was meant to extend a previous one that only discussed script injection by adding callbacks, intervals, and script tracking. I assumed others have done the same, but I hadn’t seen them.

One other addendum. While Safari 3 does support DOM tracking of script injection, it seems a bit flakey. I went back to internal testing and it seems more reliable.

I should also point out that I use lazy loading in conjunction with declarative binding for best results (add custom attributes to a form and look for those attributes on page load to inject script and bind to that form. If Dion is gracious enough, I’ll post a follow-up describing that technique).

Comment by bobmatnyc — June 6, 2008

@Harald

I’m not as big a fan of CSS loading, particularly because I don’t think CSS lends itself as well to this type of functionally-specific segmentation. I think a better approach with CSS is to combine GZIP w/ content expiration.

Comment by bobmatnyc — June 6, 2008

@bobmatnyc
Thanks for chiming in. It’s great that you’re investing in this exploration and keeping us updated with your findings.

Comment by uize — June 6, 2008

He said, she said, I did it first, who cares. Let’s just learn from each other shall we?

Comment by aheckmann — June 6, 2008

Browser sniffing is bad! It is not necessary to user a browser sniff to load some script into a browser.

Comment by PeterMichaux — June 7, 2008

@PeterMichaux I second that… I´d advise to get rid of browser sniffing as much as possible, because it causes a constant need of redesign as new browsers come up – especially if large portions of library code rely on it.

Comment by FrankThuerigen — June 7, 2008

Archetype JS provides this kind of loading, a logging interface and a configuration that handles the script and file in a java package manner with it’s dependencies, giving you the ability to alias some file and load whatever it needs to work.

http://archetypejs.org

Comment by temsa — June 7, 2008

Nice article…

I’m thinking… a lazy Java class loader along with the JVM that was made in JS… or something…

Comment by BjornGoransson — June 10, 2008

I’m researching a solution that does all of this (dependencies, cross domain loading, callbacks) and one more thing: it must allow multiple versions of the same JavaScript modules to be loaded on the same page without overwriting each other. Any one know of an existing application?

Comment by micmath — June 13, 2008

@micmath: twoBirds can do it, but this is rather a js file naming issue for application code. I don´t think there is a general out-of-the-box solution for system library code like jQuery and EXT, unless functionality is provided by the authors of these libs. BTW, jQuery can handle it, about the others I don´t know.

Comment by FrankThuerigen — July 30, 2008

jQuery can dynamically load scripts into a page with $.getScript(url, callback)

Comment by cancelbubble — April 2, 2009

I’ve been using your browser sniffing code to lazily load scripts on my site. I’ve just discovered one change that should be made…

Prototype.Browser.WebKit && !navigator.userAgent.match(/Version\/3/)

should be replaced with…

Prototype.Browser.WebKit && navigator.userAgent.match(/Version\/[12]/)

This will ensure that Safari 4 is treated the same way as Safari 3.

Comment by davidchambers — July 17, 2009

Methinks prototype should incorporate this natively, ala jQuery’s jQuery.getScript(…), as @cancelbubble pointed out above. I haven’t poked around with the source of it, but I assume it handles re-load checking, etc.

Comment by herringtown — January 12, 2010

Leave a comment

You must be logged in to post a comment.