Tuesday, August 14th, 2007

YUI Compressor: The latest minification tool

Category: Java, JavaScript, Utility

Julien Lecomte has created a new tool that aims to minify better than JSMin, and compress better than the Dojo compressor:

The What

The YUI Compressor is a new JavaScript minifier. Its level of compaction is higher than the Dojo compressor, and it is as safe as JSMin. Tests on the YUI library have shown savings of about 18% compared to JSMin and 10% compared to the Dojo compressor (these respectively become 10% and 5% after HTTP compression).

The How

The YUI Compressor is written in Java (requires Java >= 1.4) and relies on Rhino to tokenize the source JavaScript file. It starts by analyzing the source JavaScript file to understand how it is structured. It then prints out the token stream, replacing all local symbols by a 1 (or 2, or 3) letter symbol wherever such a substitution is appropriate (in the face of evil features such as eval or with, the YUI Compressor takes a defensive approach by not obfuscating any of the scopes containing the evil statement). The YUI Compressor is open-source, so don’t hesitate to look at the code to understand exactly how it works.

The Limitations

Unlike JSMin, the YUI Compressor is slow and cannot be used for on-the-fly code minification (see minify for a PHP implementation of JSMin by Ryan Grove, another Yahoo! engineer, that does on-the-fly JavaScript minification among other things).

Download YUI Compressor 1.0 to start compressing your files.

Posted by Dion Almaer at 8:00 am
28 Comments

++++-
4.5 rating from 107 votes

28 Comments »

Comments feed TrackBack URI

This is a very nice tool, works fast an so far hasn’t produced any errors in code that I have run through it.

Comment by digitalspaghetti — August 14, 2007

Is this superior to Dean Edward’s packer?

Comment by Peter Goodman — August 14, 2007

I can’t believe anyone can write a compression article without mentioning Packer. Everyone I know uses Packer with JSMin only being a second option. Packer offers the highest compression ratio that I’ve ever seen for large files and there is even a Javascript version of Packer so you can pack your scripts right inside the browser.

And the article touts the use of Java and the Rhino engine without mentioning that Dojo’s ShrinkSafe also uses the exact same thing (Java and Rhino).

Comment by Jordan — August 14, 2007

@Jordan: This isn’t an article about a packer/compressor competition. Its about a new option that people can use to make their JS smaller. Why does everything have to be some form of competition?

Comment by Rey Bango — August 14, 2007

I always use Packer, its especially useful as it doesn’t have a dependency on Java. There are implementations in 3 or 4 languages I think, and you could just as easily port it to more languages. Also, it was created by Dean Edwards, and he sure knows his JavaScript.

That said, I’ve used Dojo ShrinkSafe as well and its pretty nice.

Comment by Andy Kant — August 14, 2007

@Rey
@Jordan

Rey is right. This is not about competition. Dean Edwards’ Packer is a great tool (so are the Dojo compressor and JSMin) and I should have mentioned it in my article. However, Packer is not 100% safe with regard to ‘eval’ and ‘with’ statements. If you don’t use those (evil features), Packer is a very good alternative.

Comment by Julien Lecomte — August 14, 2007

JSMin is definitely not safe. Dojo shrinksafe is.

Comment by Alex Egg — August 14, 2007

@Julien, I’m working on Packer 3.1 at the moment. I think I’ll steal your idea of not compressing functions that contain eval/with, although I’ll probably make it optional. Good work on your compressor.

Comment by Dean Edwards — August 14, 2007

@dean

Thanks for the kind words. Another area where the YUI Compressor *seems* to do a better job than Packer is when parsing complex variable declarations. For example:

var foo = function () {…}, bar;
// do something with foo and bar…

Packer will not obfuscate ‘bar’ while the YUI Compressor will.

Regards

Comment by Julien Lecomte — August 14, 2007

You can’t seem to have a post on a product in this field without people jumping out of nowhere to say “OMIGOD, you must know about this product I use!”

I think the funniest example was when I saw someone jumping in to ask the poster if he’d looked at MooTools…on the blog of someone who actually works on Prototype.

Comment by Eric — August 14, 2007

@Alex Egg,

Dojo compressor is NOT safe. Try e.g. compressing the following, which is valid JS:

if (document.createEvent) {
var event = document.createEvent(“HTMLEvents”);
event.initEvent(“dataavailable”, true, true);
} else {
var event = document.createEventObject();
event.eventType = “ondataavailable”;
}

Dojo compressor doesn’t like that the variable event is redeclared, and the compressed code will produce errors.

Comment by Les — August 14, 2007

@Julien, I’ve already fixed that in the development version:

http://base2.googlecode.com/svn/trunk/src/apps/packer/packer.html

Comment by Dean Edwards — August 14, 2007

@Dean & Julien: I’m really impressed by the fact that both of you are collaborating on improving each other’s work. Great job guys! This is what these posts should be all about.

Comment by Rey Bango — August 14, 2007

Julien,
Did you modify Rhino or Jargs at all, or does this work just with the addition of the five java classes that you wrote? Is there any reason why the classes are packaged in the root package? Why is not packaged in com.yahoo…? Do you have any API for calling it from other Java classes. I would like to integrate this into Resource Accelerate. I really like the fact that it doesn’t appear to have any Rhino mods, like shrinksafe did.

Comment by Kris Zyp — August 14, 2007

@Les: that seems like it’s wrong; there shouldn’t be any issues with variable redeclaration. I’ll point Alex at this article (if he hasn’t seen it already, he probably has) and let him comment about it.
@Julien: nice job!
@Dean: you go!
@Eric: http://dylanschiemann.com/2007/08/02/oh-snap/

Comment by Tom Trenka — August 14, 2007

@kris

– Indeed, there is no modification to both Rhino and Jargs (which makes updating them a trivial task)
– There is no reason why the classes are packaged in the root package. I’m just too lazy to write com.yahoo…;-) Seriously, I don’t know of any Java code publicly distributed by Yahoo, so I am not aware of any naming convention. However, this might change in a future version.
– If you want to integrate it in your own tool, just look at the code and start hacking it ;-)

Comment by Julien Lecomte — August 14, 2007

Packer is very nice, but requires the code to be very clean. I found it break on missing semicolons in some places (was some time ago, so I can not give examples). Also, when you encounter a JS error in a live version of your app where you use Packer, checking out the spot is impossible in Firebug.

As for the YUI Compressor, I tested it and found that my prefered tool, JSJuicer, produces smaller output files. And without rewriting the names of local variables, strangely enough. Maybe the Compressor can’t grasp my code (Prototype.js style classes mostly).

Comment by Martin — August 14, 2007

@martin

Don’t hesitate to use the –warn option which will tell you about anything that seems abnormal with your code, or that might reduce the level of compression (typically, the use of ‘eval’ and ‘with’)

Comment by Julien Lecomte — August 14, 2007

Thanks for the tip, Julien. The warn flag is very nice. If it could show line numbers of the errors it would be even better.

Looks like an eval() in the global scope prevents variable name substitution for the rest of the file. Should not be like that, should it?

Comment by Martin — August 14, 2007

@Re Bango,
I think it’s implied that it’s a competition when the product description mentions the competitor’s features. It’s right there in the first line of the article: “and compress better than the Dojo compressor”.

Comment by Jordan — August 14, 2007

@martin

I uploaded this morning an update that fixes the problem you are mentioning (the link being the same, make sure you empty your browser cache before downloading the archive) An ‘eval’ in the global scope should not prevent sub-scopes from being obfuscated.

Make sure you check my blog regularly as I will post updates and patches.

Comment by Julien Lecomte — August 14, 2007

Tom,

See the Dojo compressed function test() below. See the problem?

function test() {
if (true) {
var test = {};
test.name = ‘Joe’;
}
else {
var test = {};
test.name = ‘Tom’;
}
}

****

function test(){
if(true){
var _1={};
_1.name=”Joe”;
}else{
var _2={};
_2.name=”Tom”;
}
}

Comment by Les — August 15, 2007

Defining a variable with var in a scope as opposed to a closure has no effect in javascript, all var declarations are defined within the executing scope – even if the var statement is never reached!

var a ="assigned globally";
(function() {
if (false) {
var a;
}
a = "assigned in function";
}());
alert(a);

var a in the function closure is assigned locally to a in that scope even though the var statement is never hit.

var a ="assigned globally";
(function() {
a = "assigned in function";
var a;
}());
alert(a);

also has the same affect as making a local to the scope.

Sorry to go of on a tangent but can’t help but bring it up when people define var’s in different scopes a they might declare variables in Java etc… as doing so always leads to possibly error prone results, as the way it works if far from expected.

Comment by Mark Brennan — August 15, 2007

can anyone please tell me how to implement this YUI compressor.

Comment by raghavendra — August 16, 2007

I’m working on a tool to comprehensively compare all the various different JS compression utilities. Stay tuned for the results!

Comment by Arthur Blake — August 18, 2007

@Arthur

I’m looking forward to seeing the results of your tool. However, don’t forget that the quality of a JavaScript compression utilitiy cannot be summarized by its level of compression (most often, such tools yield different savings on different files anyway) The most important factor must be the safety and reliability of such a tool (how good is a JavaScript compression tool if it introduces new bugs? These bugs have a cost that must taken into account)

I would like to see a comparison of the JavaScript compression utilities in terms of their cost efficiency i.e. their benefits vs. their cost. However, this may be hard to measure.

Regards

Comment by Julien Lecomte — August 18, 2007

@Julien
You are absolutely right, and I will take all those variables into account (as best I can.) One thing I’m interested in, is being able to visually compare the compression percentage results of multiple compressors, side by side, against the same code base and then also quickly see the additional savings gained with gzip compression.

I think that gzip compression can sometimes really reduce the actual difference between the compression tools to almost negligible in many cases. If this could be shown to be true the majority of the time, then a possible conclusion might be that a best practice would be to lean towards the more conservative compressors. I know that some people have already come to this conclusion and are already saying this, but it’s nice to be able to back it up with some pretty graphs and numbers :)

Comment by Arthur Blake — August 19, 2007

A simple online javascript minifier can be found at http://netspurt.com
It is based on dojo.

Comment by magnoli8575 — November 21, 2008

Leave a comment

You must be logged in to post a comment.