Tuesday, November 20th, 2007

CacheFile.net: Central JavaScript Library URLs

Category: JavaScript, Library

Jon Davis has created CacheFile, a place to store versions of common libraries such as Dojo, jQuery, Ext, Prototype and more. Dojo has a nice CDN already thanks to AOL, and YUI thanks to Yahoo! The other libraries don’t have the same, so this could be the solution:

CacheFile.net is an HTTP web server that contains common Internet resources that are frequently reused on other web sites. It exists to alleviate the need for a common root URL for web resources that are otherwise not directly linkable at their primary URLs.

For example, popular Javascript libraries may have a direct download link for a specific version of their script files, but may discourage the direct linking of these files on their site. People are instead encouraged to download the scripts and manage them on their own servers. But the problem with each web site retaining its own copy of the same resources is that web users must re-download them all over again as they navigate from site to site. Over browsing five different jQuery-driven web sites using the same version of jQuery, there may be five different copies of the script being seperately downloaded.

You can take a peek at the listing of scripts and you can even grab a versioned file while setting the expires tag that you want:

  1. <script type="text/javascript" src="http://cachefile.net/scripts/yui/2.3.1/build/yahoo/yahoo.js?expires=Thu,+01+Jan+2009+00:00:00+GMT"></script>

I am all for this idea. I want a central set of resources to host JavaScript files so browsers can do smart things and have special versions for browsers cached etc. The only issue with this particular site is:

  • They could turn it off at any time (and mention this on the home page)
  • Trust. We need some kind of trust relationship. I am sure Jon is a good guy, but if he wasn’t he could switch out the libraries. Or, not in a malicious way, the files could be incorrect
  • CDN. AOL and Yahoo! use CDN’s to serve their files

I applaud Jon for doing this though!

Posted by Dion Almaer at 5:19 am
28 Comments

++++-
4 rating from 25 votes

28 Comments »

Comments feed TrackBack URI

I like the idea, but another downside is that there is no compression being done over the wire.

Comment by Claude — November 20, 2007

For point 2 about “trust” try googling for “link fingerprinting”:
http://www.gerv.net/security/link-fingerprints/

Comment by Jordan — November 20, 2007

hmm, the next step could be:
[script src=prototypejs.org/public/prototype.js type=text/javascript /]
or
[script src=jquery.com/public/jquery.js type=text/javascript /]
To overcome the trust problem, provided library sites care to provide the bandwidth.

Comment by Ben — November 20, 2007

Sooo, who’s paying the bills?

Comment by Mike — November 20, 2007

This is pretty cool and exactly what I was doing with http://www.jsloader.com... What would really take this to the next level is leveraging the fact that you know where things are located (you have a predictable path structure) and therefore can make a common loader.js which would, if loaded on the page, allow you to call something like require(product,version [,options[]]) or load(product,version) and dynamically write the script tags onto the page. I do this on jsloader by having a “recipe” file per version per library, and those tell JSLoader what else needs to be loaded.

Perhaps add this type of code to your site so you can allow people with no knowledge of the libraries to “bootstrap” their pages by loading the libraries by only specifying a version number…

The other thing I’d want is a downloadeable version of the full repository, so I can localize it for enterprise deployment.

I did this for JSLoader (JSLoader is an open source port of what we did at my office) and if deployed on a robust infrastructure, it will make it very easy for developers to leverage the shared versions of libs and keep these assets out of their web applications’ revision control libraries….

This is in no way a plug for jsloader, but I’m just saying – I see value in slapping a common loader tool on top of any well organized repository, and this is a well organized repository

Comment by Dov Katz — November 20, 2007

Nice idea, though how does work with the security sandbox? Are there cross site scripting issues with JS downloaded from another URL? And if SSL is required are there any issues?

It is a bit mad that each site downloads their own version of a scripting framework. I’d like to see the browser applications take care of this, in the same way they do for HTML DTDs

Comment by Howie — November 20, 2007

This is idealist more then anything, basic problems are trust and bandwidth / reliability. Ajax libraries just need a better / standard way to be packaged. Something should emerge in the upcoming months.

Comment by Jonathan Bond-Caron — November 20, 2007

Im sure Jon has the best intentions

BUT

would you trust your site to rely on a third party site thats hosted on hostgator.com shared accounts?

http://private.dnsstuff.com/tools/tracert.ch?ip=cachefile.net

i think i will stick with yahoo’s CDN for YUI and own servers for jquery

I hope all the best in this project, the data that can be mined from the access logs can be sold onto advertisers like google or yahoo who love to get their hands on even more data

Comment by mihd.net admin — November 20, 2007

Ben: that prototypejs.org link points to Mephisto’s copy of prototype and isn’t guaranteed to be current, so I would not rely on it.

Comment by rick — November 20, 2007

Hi guys. Thanks for the comments. The reservations and concerns mentioned here are quite valid. I wanted to comment on a couple of things: 1) pay the bills? what bills? seriously, though, this is donationware, if it’s hardly used then I can certainly afford it myself, if it’s moderately used then ten people contributing a buck or two shouldn’t be hard to come by, and if it’s heavily used then a hundred people contributing a buck or two is no biggie either. These are small files with low processing overhead (logging, mostly, and coming gzip compression). 2) Issues regarding bandwidth and reliability, such is certainly a resonable concern, but to this I’ll just say that reliability will go up in accordance with how much people use it and/or how much people donate. Right now it’s on a HostGator account to get it started and HostGator is a fine host. However, I, too, have my doubts about it being the best platform for a heavily depended upon high uptime web site, so I’d like to migrate ASAP. Send me your money. :P :) 3) On trust, my hope is that the community will, in the process of adopting the platform, develop some leadership and accountability structure so that we can go beyond “Jon Davis’ nice idea” to a community-driven project. Isn’t there a model out there for shared responsibility in web hosting like there is for open-source software? We could build a community-driven CDN out of this. Otherwise, Akamai and other CDNs are certainly on the table. My goal really is to make this whole thing reliable. On the other hand, if people don’t show their support for the idea by either settling for the current system now or by donating, a CDN will be too difficult to come by. 4) Gzip compression (when the browser says it supports it) is coming. 5) Downloadable archive of complete library: that’s coming. I’m also looking at adding support for a virtual “current” version for each library that is dynamic rather than statically versionized.

Finally, “they could turn it off at any time”, hm, so I could. But I won’t. I’ve already committed that if I shut the site down, I’ll give at least a month’s notice if anytime very soon, at least a year if it’s been online for the next couple months. In either case, I will add to the forums a way to subscribe to a mailing list for such notifications.

@jordan, I’ll take a look, thanks for the link!

@Dov, *sob* if only I’d seen your site before I started this project, I’d send money your way instead!! I’m sorry. However, I’m also going to go beyond JavaScript and will be keeping an eye out for commonly used DTDs, schamas, microformat css’s, and more. I’ve already included syndication graphics.

@Howie, you make important points, if a site is using SSL, cachefile.net won’t do https right now. This will result in a browser-side notification / error. This is something that needs to be fixed; on the other hand, best practices are such that you should never reference external resources on an SSL-secured site anyway. As for cross-site script attacks, these are standard popular libraries only, no one will be uploading Javascript scripts to the repository (except me, or other admin people).

Comment by Jon Davis — November 20, 2007

Amazon S3 would probably be a good solution for a host. Its always going to be fast, reliable, and cheap.

Comment by Andy Kant — November 20, 2007

> Send me your money. :P :)

.. hm. While the site is “donationware” in the sense that there’s a donation link, I do want to emphasize that this was in fact stated tongue-in-cheek and I’m not doing this to try to clean out people’s wallets but to try to help meet a serious need that the web has. Please don’t be offended by the above silliness.

Comment by Jon Davis — November 20, 2007

It would be great if browsers would support major js libraries so… they would be incorporated in the browsers and browser’s update would update the recent libraries too

the second thing the trust issue could be sovled with server mirroring or more servers offering the same thing.. and yo would only have to upadte you server list once in a while (like emule works)

Comment by Damir Secki — November 20, 2007

Jon, are there any copyright implications with any of this? I know that EXT has one of the more conservative license agreements, I’d be curious if any of this falls into reseller/OEM realm.

Also, the Dojo AOL CDN is more than just a link to the Dojo source, it’s a CDN cross domain build. Are the ones you serve the same, and have you done the same thing for the other toolkits?

Comment by JP — November 20, 2007

Maybe better to use Coral Content Distribution Network ?

Just add .nyud.net to your domain name, and you’re done, also you can mirror any file, including your logo, css..etc

Atm it seem to have a network of 260 servers (!) world-wide, not sure about the “speed” and of guaranties what the file will be sure delivered..

I’ll better go for Amazon S3.. But a “foreign” url showing all the time in the status when you browse another page is not very handsome..

Comment by Nicolae Namolovan — November 20, 2007

There are lots of CDNs I could plug cachefile.net into, as well as very reliable distribution services like Amazon S3 (which, by the way, is obviously not free).

The issue is that a) I’m certainly not going to be changing any URIs, the primary objective is to keep the URIs static, not just to host the content reliably. And b) if I do go with a CDN or Amazon S3 or something, it will involve a 302 redirect. I’m trying my hardest to avoid 302’s, as that can be almost as expensive as a bad web host to begin with.

I am definitely not interested in going long-term with HostGator.com. This is just an incubation period while static URIs are defined and these logistics are being initially discussed with you, the Internet community.

Comment by Jon Davis — November 20, 2007

> Jon, are there any copyright implications with any of this?

Check the licenses, which are included in the browsable directories. They should all be redistributable. I’ll triple-check myself and pull anything I put up if I realized I goofed up on any particular non-redistributable library.

Comment by Jon Davis — November 20, 2007

Regarding my concerns about 301/302 redirects and CDNs, I describe my issues here: http://cachefile.net/forum/viewtopic.php?t=15

Comment by Jon Davis — November 20, 2007

Jon,

You don’t have to change the URLs to use Amazon S3, at least if you plan ahead.

“Over browsing five different jQuery-driven web sites using the same version of jQuery, there may be five different copies of the script being seperately downloaded.”

Uh, as the cachefile server is configured that’ll happen too using cachefile — and as someone else pointed out, the files are not compressed.

The cachefile server isn’t just missing essential cache headers; it’s actually explicitly telling the browser NOT to cache. Very Clever (not).

– ask

Comment by Ask Bjørn Hansen — November 20, 2007

ps. just to add: I do realize you can “set the expiration date”, but that completely defeats the stated purpose of having the files cached universally from site to site

Comment by Ask Bjørn Hansen — November 20, 2007

I read some time back that jQuery were providing a hosted version of their script (though not on a CDN, but hosted by Google).

The (old) url I’ve got is: http://jquery.com/src/jquery-latest.js (which redirects – so the new URL may be better).

It’s worked consistently for me for the last year and is great for bookmarklets or just pulling it quickly.

Comment by Remy Sharp — November 21, 2007

It has to be a very stable server to host libraries for others. I understand what gives that trick for the idea’s author – an enormous links. But if the server fails people’s web sites can fail too.

Comment by Nick — November 21, 2007

Nick: that’s why “I gave” the idea is to have multiple servers

Comment by Damir Secki — November 21, 2007

@ask: I don’t know what you’re seeing but as it’s configured right now they will be cached for one full year. As for compression, I already explained that gzip compression is coming. I enabled gzip compression last night but ran into technical issues so I had to turn it back off temporarily until I fix the problem.

@Nick: I agree, the server should be stable. The current host has uptime of 99.9%. Really the only thing that will throw it off will be the scalability of the script code that manages the traffic and adds features like querystring-requested expire dates (etc). However, again, the objective is to cache the data with static URIs, not pump heavy traffic continuously. If it’s server stability you want, help me couple your donation with my own donation.

Comment by Jon Davis — November 21, 2007

@ask: “ps. just to add: I do realize you can “set the expiration date”, but that completely defeats the stated purpose of having the files cached universally from site to site”

… The browser will view the URL with the specified querystring as a seperate entity than the URL without the querystring.

Comment by Jon Davis — November 21, 2007

Re the querystring, added to the description: This is not typically the best approach as it creates a seperate cache entity for web browsers because querystrings are treated as different URI, but the feature was added for flexibility. You should not use this feature with a dynamic date value or the resource will never be reused in the browser’s cache, but for a static future-dated item this feature may be useful to some web sites.

Comment by Jon Davis — November 21, 2007

I made 3 demos (prototype, ext, jquery) which can load files from cachefile.net.nyud.net (the coral CDN proxy) via JSLoader…

Take a look
http://www.jsloader.com/demos/

Comment by Dov Katz — November 21, 2007

Meanwhile I added Coral CDN as a recommended URI alternative directly on the CacheFile web site.

Comment by Jon Davis — November 21, 2007

Leave a comment

You must be logged in to post a comment.