Wednesday, August 19th, 2009

Details on JS compression; Squeezing every last byte on the wire

Category: JavaScript

<>p>Ray Cromwell has a great article on techniques he has used with JavaScript compression to bring down the payload of your Ajax application.

There are some fantastic advantages to JavaScript being “binary as source” but there is also a real issue with it. We have to make a trade-off on verbose code…. even with minifiers and compressors. GWT has a compile step which allows it to do a little more, and Ray has done some work to get smart compression in there:

Combining base-54/base-64 obfuscated identifier encoding, stable sort-order for identifier allocation, my greedy clustering-by-edit-distance sort algorithm, and 7-zip as a gzip-compatible compressor, yields an incredible 21% reduction of the Showcase application. On a large 500k Javascript application, this means an additional 100k bandwidth is saved, with no performance penalty!

The post itself walks you through some of the basics of compression, and techniques such as sliding windows:

LZ77 on the other hand, is a sliding window compression algorithm based on replacing strings with backwards references to previous strings in the input. For example, the string “this is a test” contains the substring ‘is’ repeated twice in a row, separated by a space, so that the second occurance of ‘is’ can be replaced with a length (2 characters, and a backwards distance (-3 positions), called the length-distance pair. The compressor typically scans backwards in the input within a certain window (e.g. 8,192 characters or 32,768 characters) looking for matches and then encoding them as length-distance pairs. The compressor has some freedom as to how hard it will search for a match before giving up (something I’ll get to later).

One important effect of the sliding window limit is that if two Javascript functions with common substrings are separated by more than this distance, they cannot be matched.

Really interesting stuff, Ray. I hope to see some of this in the minification libraries!

Related Content:

  • Squeeze on server sales
    For years the x86 server market has been the reliable cash cow of the reseller business. Over a three-year cycle most resellers could bank on their...
  • Compression bandages for enterprise data
    Business make more data every day, but the amount of bandwidth available to move it around is harder to acquire. This feature from Voice & Data...
  • Compress, then encrypt tapes
    This tip explains when your data should be encrypted, and details some programs that can help you with this...
  • The SOA talent squeeze
    ZapThink predicts a dramatic surge in the demand for SOA consulting in...
  • The SOA talent squeeze
    ZapThink predicts a dramatic surge in the demand for SOA consulting in...

Posted by Dion Almaer at 6:10 am
4 Comments

++++-
4.3 rating from 30 votes

4 Comments »

Comments feed TrackBack URI

I would like to see a apache module OR XYZ Framework plugin which is able to compress inline javascript and/or css. Compressing external files is the first step, but inline should be the next imho.

Comment by Aimos — August 19, 2009

I’m woundering when we will see native support for Delta encoding in web browsers?

In other words: RFC 3229 – Delta encoding in HTTP

That would make great impact on performance and save a lot of bandwidth in many cases.

Comment by Trygve — August 19, 2009

The clustering stuff is definitely not GWT specific, the first implementation I wrote was a Perl script.

Comment by cromwellian — August 19, 2009

Outstanding research! Nicely done, Ray.

Comment by slightlyoff — August 19, 2009

Leave a comment

You must be logged in to post a comment.