Thursday, August 31st, 2006

JS Linker in Dojo Toolkit

Category: Dojo, JavaScript, Library

<>p>AOL has been very generous by donating code for the JS Linker for Dojo:

The JavaScript source code can be represented in different levels of granularity. The JavaScript Linker uses the Abstract Syntax Trees (ASTs) representation, which represents the lowest level of detail, to model the source code. One of the main task for this project was to write a JavaCC compatible grammar that strictly follows the ECMA Specification. JavaCC uses this grammar to build a custom parser than can read and analyze the JavaScript source, which in turn, is used to build the JavaScript Linker.

What this means to Alex Russell:

This is the holy grail of JavaScript optimization: removing “dead” code. Dojo already provides a package system to help prevent including too much and a build-time compressor to help reduce the size on the wire of what you do need, but the linker does all of this one better by analyzing the application and figuring out what functions are entirely unused.

JavaScript has some unique issues with packaging, and this will help Dojo coders out a lot.

Related Content:

Posted by Dion Almaer at 4:51 pm
26 Comments

+++--
3.6 rating from 37 votes

26 Comments »

Comments feed TrackBack URI

I have my doubts about this. How is this suppose to deal with the dynamic nature of JS?

Like


if ( this["show" + this.view] ){
this["show" + this.view]();
} else {
this.show();
}

or suchsome.

Comment by Martin — August 31, 2006

Well, it certainly passed my 5 mins test… and the size reductions are incredible…

Comment by andyhot — August 31, 2006

Hey Martin,

Obviously, we can’t handle everything. The linker will try to be conservative about some of these cases, but you can always trip it up by obsfucating your code. If it misses something you need, you should be able to get it back in with a global block like:

if(false){
thinger.foo();
}

or some such.

Regards

Comment by Alex Russell — August 31, 2006

You had time to run it already Andy!? I’m jealous…You should be closing more bugs ;)

Comment by Jesse Kuhnert — September 1, 2006

OMFG!! Are you kidding me Andy? Down to 2.3kb for dojo and 6.kb tacos? That’s nuts. Now I really am excited..:)

Comment by Jesse Kuhnert — September 1, 2006

Hi Alex,

I will definitely free some time to give this tool a try and see how it performs, which style of programming it cannot cope with.

E.g. I consider my example above no obfucation. I use the dynamic features of JS where helpful, and this is one of these cases. Do hints like Thinger.prototype.foo(); work as well?

Speaking of objects, I wonder what it does with OOP style like http://dean.edwards.name/weblog/2006/03/base/ , amending Thingy.prototype through functions etc.

Comment by Martin — September 1, 2006

i hate DOJO, ,RICO, backbase, prototype, scriptacolous and all the bloody fat and not crossbrowsersave libs! throw this unperformant and fucking big overloaded crap to trash!

boycott this stupid shit!

Comment by xxx — September 1, 2006

remove DOJO…coz its DEAD by itself!

Comment by serius — September 1, 2006

Hey, if you didn’t have a billion lines of source code, you wouldn’t need to optimize it. Why in the world are people so excited about DOJO? It’s taking all the crappy things about Java and putting it into javascript. THERE ARE WAYS TO DO EVERYTHING DOJO DOES WITHOUT A MONSTER CODE BASE. Bump off the Jandwagon people.

Comment by Chris Hamilton — September 1, 2006

Don’t you think we’ve already tried the other libraries? =p

Comment by Jesse Kuhnert — September 1, 2006

Do you know how to do it without a library?

Comment by Chris Hamilton — September 1, 2006

Hey xxx, serius, and Chris:

So fundamentally we would like to be *out* of the business of providing these kinds of abstractions. That we have to ship around functions that make up for the stagnant and back-asswards DOM APIs and browser implementations makes me none too happy. And don’t you think that packaging is something the language should provide? The widget system, likewise, is architecturally constrained by the implementation decisions of IE. It’d probably be about a third the code were it not for the various browser issues. The things we’re going to need to do for data binding because the JScript team failed to implement getters/setters just makes me want to cry.

Dojo is big because it tries to accomplish a lot of things and because we try to do them everywhere. The build system is there to help make it as light as you need it to be for your app, but fundamentally, until the browsers start to fix some of the things they’ve left neglected, every major toolkit will asymptotically approach the 100K+ mark for any non-trivial use case. MochiKit and Scriptaculous are there today. Prototype alone is 54K, and all of these are tools that advertise how “lightweight” they are. Dojo builds that do similar things are usually in the same ballpark. We spend a *lot* of time trying to ensure that builds that need just one thing are as lightweight as possible. Jquery and MooFX take a different approach to solving the problem: simply promising to do less. It’s a good solution too.

Dojo is up-front about what you “pay for”, when you pay for it, and how to make your build smaller without rewriting your non-trivial app. If writing, debugging, and testing this stuff yourself is your idea of a good time, well, then no one will dissuade you from doing exactly that. Best of luck. If you want to be part of a community that has hit these problems before, has some solutions, and is constantly looking for better ones, we welcome you to join the mailing lists and help us make Dojo better for whatever your needs may be.

Regards

Comment by Alex Russell — September 1, 2006

Hi Alex; what if we make some specialized Dojo builds that we call the “Prototype” build or the “Scriptaculous” build, and show how to use Dojo for those use cases that people usually use those two libraries for? It might help to show how the build system slices off just what you need, and how the size is small (the dojo.event build for a Prototype slice, for example, might not include the dojo IO provider for uploading, for example).

Best,
Brad

Comment by Brad Neuberg — September 1, 2006

Alex,
Please don’t lump me in with “xxx” and his h4x0r friend “serius”. I apologize if my post sounded like a thoughtless attack.
I’m a enterprise software engineer working exclusively with Ajax in a logistics company. We started applying ajax to an existing application as an experiment. But we quickly realized how powerful this paradigm is to internal business applications. Our guys in the warehouses can now enter data at blazing rates using autocompletes, behind the scenes database updates and smoother UI’s. And it beats the hell out of desktop apps because we don’t have to install it on every station in every warehouse. It’s a webpage.
So I know where you’re coming from, you’re constantly frustrated with the inconsistencies and lack of structure that exist in most ajax apps. Before I came here, most of their “screens” had 700 lines of javascript in the headers.

When we started all of this, Dojo and most of the other “frameworks” didn’t exist. So we rolled our own. After a while we had similar sized .js files like you described.
Then I came into contact with two very very experienced enterprise software engineers with 20+ years of experience designing, building and implementing all kinds of client/server and messaging systems. They described to me how to unify GUI and data into messages and message frameworks. They were doing this when the Internet was still called DARPA NET on dumb terminals – instead of Javascript and DIVs they used VT220 terminals codes and C. It’s the same thing all over again.

What’s my point? We married the two. Our client side code is a few hundred lines and it’s sole purpose is to dispatch and render messages. The result is a very very light system. Light in the sense that the source code, or scaffolding as we call it, has no moving parts and in no way interferes with your changing business requirements or even changing environments.

So going back to my original point. You’ve come across a platform that allows you to break out of the confines of source code and J2EE mayhem. And what did you do? You rewrote Java in Javascript and called it Dojo.

Comment by Chris Hamilton — September 1, 2006

Hey Chris,

I take serious exception at your conclusion that we “rewrote Java in Javascript and called it Dojo”. We embrace and build on the functional nature of JS. I personally prefer Python to most other languages, and only grudgingly use Java. What we’ve done with Dojo is to build a set of layered libraries that you can pick and choose from. Our plan is, and always has been, to build a set of common APIs that we can implement everywhere. A standard library, if you will. We didn’t just invent features because they were fun to hack up, we built them because we needed them in apps that we’re building. The point of Dojo is that you can take or leave them as you see fit. Obviously, you’ve taken a different route. Fair enough. All I ask is that you not jump to the the conclusion that we’re over-engineering or spending effort in places where it isn’t necessary when you aren’t even using the tool.

Regards

Comment by Alex Russell — September 1, 2006

It’s pretty difficult to mount an argument against the existence of reusable software libraries.

My main hope now is that the Dojo project focuses on making these tools easy and accessible to the general Ajax community, because I fear that at present many still consider it a black art (though I realise the Dojo team may have different objectives to the project’s end-developers). Dojo, and especially package management, has already solved many of the problems that hurt developers, but I don’t think many developers are yet aware of how to use the solution. Though some docs talk about running ant and Java, many potential beneficiaries of Dojo have never used either.

Comment by Michael Mahemoff — September 1, 2006

Dojo will be fine, however people working on it may start to get those same migraines they used to get from large J2EE projects.
However, I’m not saying it’s a bad thing in general. Hopefully DOJO brings Ajax to the masses of stalwart J2EE and .NET programmers who live in worlds of strict corporate pressure and beuracracy (those “ajax is a fad” people). I just feel that Dojo is taking some dangerous concepts to the world of client side messaging. I’m warning against putting too much “scaffolding” into the client because it could make applications written in it rigid and inflexible. This may be just what the doctor ordered for public facing websites that dictate their own updates. But when you’re writing applications that run someone’s business (inventory, point of sale, manufacturing, etc..), you can’t rely heavily on source code or your project will be a nightmare. I’m actually very surprised that no MQ Series/Tibco folks haven’t stepped up to the plate in a discussion such as this. Just remember that Ajax’s core functionality (that which is included in the XHR and it’s variants) is to allow the server and client to send messages to one another. So it makes sense that a messaging/pub sub architecture could result in smoother and more flexible Ajax applications than RPC, client side OO and compile->deploy development. I won’t say anymore because I feel like I’m being labeled as a heretic, but I just don’t want everyone out there to think that in order to accomplish ajax on their site, they must download and adopt one of the monolithic frameworks like Dojo or YUI. Have you ever thought about client side XSLT? Or just sending HTML or XHTML snippets combined with simple effects libraries like Scriptaculous and MooFX?

Comment by Chris Hamilton — September 2, 2006

Hey Chris,

It seems pretty clear that you’re not spending much time with Dojo. What “scaffolding” exists is entirely in the widget system, which is entirely optiona, and even that is over-rideable at every point. Anyway, I’d like to know more about what “dangerous concepts” you think we’re exposing. You’ve said as much a couple of times without outlining what they are. Is is that we provide an (entirely optional) topic-based event system? Or is it something else?

As for XSLT, yes, we’ve considered it, but we already have a system that allows you to send down just HTML snippets (widget templates) and instantiate them at-will. Perhaps providing XSLT for those comfortable with it would be a win, but that’s a pretty small group of people compared with those who can write HTML and CSS.

Regards

Comment by Alex Russell — September 3, 2006

Chris, Dojo is not one monolithic toolkit; it’s many small slices that you can choose to use or not. You mention client-side XSLT; I am actually using this for a consulting client, where we get OPML on the client, then use XSLT to transform it into something that can be displayed to the end user. We still use Dojo. In particular, we use the following ‘slices’ of Dojo:

dojo.event – Dojo’s event system, which allows me to do one line to subscribe to events: dojo.event.connect(myElem, “onclick”, this, this._myElemClicked)

Dojo internally normalizes the event object cross browser, handles memory leak issues on IE, and more – this one line vastly simplifies my code and makes it more robust

dojo.string – simple string handling methods that JavaScript forgot or that you have to roll yourself, such as dojo.string.trim(), dojo.string.startsWith(), etc. I usually have to hand roll these myself, so why not just use Dojo’s battle-tested and performant versions?

dojo.html – methods when working with HTML and style classes, such as dojo.html.hasClass(), dojo.html.addClass(), etc. Most developers forget that elements can have multiple classes, and create code like this:

if(someElem.className == “foobar”)

then, they wonder why their code doesn’t work, because someElem might also have the class “important”. Dojo’s class methods solve this:

if(dojo.html.hasClass(someElem, “foobar”))

again, this is not bloated; if you didn’t have these you’d have to roll them yourself, which I’ve done countless times. And if you don’t have them, then your producing incorrect code that will fall over…

dojo.io – A layer over XHR to account for it’s browser differences; again, this is very easy to use:

dojo.io.bind({ url: “http://foobar.com”,
mimetype: “text/xml”,
error: function(type, errObj){ /* error function here */ },
load: function(type, data, evt){ /* load function here */ });

Now, Dojo IO takes care of papering over important browser bugs (like how IE doesn’t actually displatch your GET request if it has the same URL as one it’s done before, handing it from the cache incorrectly instead); handing you back XML if you have text/XML as your MIME type in a cross browser way, etc.

None of this looks bloated to me; in fact, it’s thing I have to create myself usually. What it DOES do is make your code cleaner and more focused on the application rather than browser hacks. My code now actually reads like the processes it’s trying to do, rather than boiler plate junk. Plus, using the above is pretty easy. I’m not sure why people say Dojo is hard; I think what it is is you have to know basic JavaScript. Folks don’t want to take 20 minutes to read an intro to JavaScript article before hacking on Ajax.

Dojo does have more experimental and interesting ‘slices’, such as my Dojo.Storage and the Dojo Widgets, but you DON’T HAVE TO USE THEM. That’s the most important thing that seems to get lost on people; each slice is a seperate package, just like a Java Package, and if you don’t want it DON’T IMPORT IT. Sorry for the caps but I’m getting tired of this same argument…. :)

Best,
Brad

Comment by Brad Neuberg — September 3, 2006

I actually think Alex went with the right choice not to use XSLT for our widget system; he’s right that it’s a bit esoteric. Now, I do think the performance advantages of it are strong, and it might make an interesting alternative.

Best,
Brad

Comment by Brad Neuberg — September 3, 2006

If dojo required the use of tools during development then I’d say you were on to something..

Because I know I can use it without any tooling at all during development quite happily it doesn’t raise any obvious flags for me…Everything beyond that point is just “icing”.

As for being opposed to the use of java in tooling, I guess I sort of understand the current trend in not liking it….People started turning against c++ a long time ago too, but more than likely it’s still running the browser you are reading this thread in right now.,.

Comment by Jesse Kuhnert — September 4, 2006

Alex and Jesse, I think you guys handled this argument like zealots more than engineers. I am working on a response to your points and I will email it to Alex to save people reading this site a prolonged and agonizing public argument. No side of this is necessarily right, but I feel like you guys are ignoring the existence of a different side completely. It basically comes down to this, despite what you might think, you are arguing for Ajax following the disciplines of RPC/OO/Compile->Deploy and I’m coming from the side of Ajax as messaging/Functional Programming/Pub/Sub etc.. Just because our programming methodologies differ from yours, they aren’t invalid.

Comment by Chris Hamilton — September 5, 2006

Chris: looking forward to your mail. It seems there’s still a wide gulf between what you think Dojo is and what we actually do with it. Hopefully we can clear that up.

Regards

Comment by Alex Russell — September 5, 2006

chris: I don’t think it’s zealotry as much as you hinting at this “better” way of doing things that you still haven’t explained. Whether I like it or not my natural tendency is to question everything I’m told/read/know on a constant basis.

If you have a better way of doing things that I actually believe is better then I’d certainly heavily debate either dropping everything I’m doing right now or trying to find a way to make it happen in projects I already work on. That’s what I did while using prototype and found dojo. I hope that clears up any thoughts you had about me having “fan syndrome” ;)

Comment by Jesse Kuhnert — September 5, 2006

이런뒌장

Comment by ë­˜.. — September 12, 2006

is there a single library that does all that?
createDocument()
transformXml(xmlUrl, xslUrl)
getNodesBYXPath(xpath)
cacheNode(id, node)
getCachedNode(id)
addEvent(target, eventType, handlerFunction)

Comment by sim — October 23, 2006

Leave a comment

You must be logged in to post a comment.