Monday, June 23rd, 2008

Jiffy Firebug Plugin: Fine grained calculation of performance timings

Category: Performance

<p>Today is the kick off of the Velocity performance conference, and we are going to see a fair share of performance news over the next day or two.

To start out, Bill Scott (Rico/ex-Yahoo/now Netflix) has announced a new Firebug plugin, Jiffy that adds a new tab showing fine grained performance data. You want to know the time between the onunload of the previous page, the first rendering, time until onload, time after, and more.

This is where Jiffy-Web comes in. Jiffy-Web is a fine-grained and flexible website performance tracking and analysis suite written by Scott Ruthfield and the team at Whitepages.com.

The Firebug plugin uses that data, which it gets from the DOM JSON object, to do the visualization.

Jiffy Firebug Plugin

Bill wrote a detailed post on Measuring User Experience Performance that goes into the details behind this tool.

He goes into detail on how to measure things, and what can get in the way. For example, onunload:

The most logical place to measure the start of a request (“from Click”) is on the originating page (see A in figure above). The straighforward approach is to add a timing capture to the unload event (or onbeforeunload). More than one technique exist for persisting this measurement, but the most common way is to write the timing information (like URL, user agent, start time, etc.) to a cookie.

However, there is a downside to this methodology. If the user navigates to your home page from elsewhere (e.g., from a google search), then there will be no “start time” captured since the unload event never happened on your site. So we need a more consistent “start time”.

We address this by providing an alternate start time. We instrument a time capture at the very earliest point in the servlet that handles the request at the beginning of the response (see B in figure above). This guarantees that we will always have a start time. While it does miss the time it takes to handle the request, it ends up capturing the important part of the round trip time — from response generation outward.

There are a number of ways to save this information so that it can be passed along through the response cycle to finally be logged. You can write out a server-side cookie. You can generate JSON objects that get embedded in the page. You could even pass along parameters in the URL (though this would not be desirable for a number of reasons). The point is you will need a way to persist the data until it gets out to the generated page for logging.

Note that the absolute time captured here is in server clock time and not client clock time. There is no guarantee these values will be in sync. We will discuss how we handle this later.

He also talks about practical issues that he has found implementing this at Netflix, and when the data shows you the real truth:

Recently we fielded a different variation of our star ratings widget. While it cut the number of HTTP requests in half for large Queue pages (a good thing) it actually degraded performance. Having real time performance data let us narrow down on the culprit. This feedback loops is an excellent learning tool for performance. With our significant customer base, large number of daily page hits we can get a really reliable read on the performance our users are experiencing. As a side note, the median is the best way to summarize our measurements as it nicely takes care of the outliers (think of the widely varying bandwidths, different browser performance profiles that can all affect measurements.)

name lookup timed out

Posted by Dion Almaer at 9:15 am
8 Comments

+++--
3.8 rating from 34 votes

8 Comments »

Comments feed TrackBack URI

At first glance this looks like it could be a very useful tool, however with such things I am concerned that there is the potential to be a slight “uncertainty principle” effect, by which the tools used to measure performance, conversely affect performance.

I’ve certainly had issues with Firebug causing performance issues with animations, and anyone using gmail will have seen the warning displayed when you try and access the page with FireBug enabled.

My point basically is that quite how ‘fine-grained’-ly can we measure performance without making the results meaningless?

Comment by lennym — June 23, 2008

Thanks Dion. A few points:
*) At Netflix we are using a full cycle tracing mechanism that at the time does not include Jiffy-Web. This gives us backend to end of rendering timings. Jiffy-Web measures only on the client.

*) Jiffy-Web provides a simple API to do a common task (time stamp) as well as provide a way to log to a logging service

*) As Dion mentioned, this firebug panel accessing the logging information that is contained in a JSON object in the measured page

*) The firebug panel can actually work with any tool like this by setting three firefox configuration parameters (see the documentation). I am interested in other ways to work with external measuring products (or even extending firebug’s console method to write to this panel).

*) I have a separate firebug tool that is internal to netflix that visualizes this full cycle information. I will be merging the two firebug tools into one… but need a standard definition for measurements so the tool can graph anyone’s measures.

*) Jiffy-Web will be “released” sometime today in concert with Scott Ruthfield’s talk at Velocity.

*) As far as fine-grained, with any measuring there is some milliseconds wasted by having to do this in javascript. You can minimize this of course. But I have found the best measurement happens in the aggregate with statistically applied measures. But the purpose of tools like this is to give you an indication of whether you are screwing something up and possibly where to focus your tuning efforts.

Comment by billwscott — June 23, 2008

I’ll have to play around with this a bit more. I installed it at work, added the jiffy.js script to my development site, and then checked it out and wasn’t getting much data, though I suspect that could be due to the fact that my sites are all cached already.

I have a different question for you though, Dion: what are the “Round Trip” and “Coder” tabs in your Firebug display? The others (YSlow and Firecookies) I recognize, but those two are unfamiliar to me.

Comment by Dashifen — June 23, 2008

Just to continue Dashifen’s comment (since I don’t even know Firecookie), is there a repository for firebug-1.2 plugins?

Comment by urandom — June 24, 2008

Just to let others know. I had to register at Firefox Add-ons to download this plugin.

Comment by GalloNero — June 24, 2008

Very nice though the results were sometime questionable. It’s always good when new developer tools appear.

Comment by sandrejev — June 24, 2008

The screen shot was mine (Bill Scott).

The Round Trip is something that I wrote internal to Netflix. You can see an example of it here: http://farm4.static.flickr.com/3256/2597183452_c3def20fcb_o.png. I plan on incorporating this visualization into the jiffy extension.

The Coder extension can be found here: http://getlivecoder.com/. It is basically a debugger for Firebug or Firefox Extensions. Lets you poke around in the chrome, modify js on the fly.

The jiffy extension is in the “experimental” sandbox area. Until it finishes the review process you have to login to get it unfortunately. Hopefully it can move out into the regular area. (all extensions go through this process)

Comment by billwscott — June 24, 2008

This sucks.
Just to download, i had to create an account on addons.mozilla.org
Then after downloading it gave an error that the hash didn’t match.
Fine. Whatever.

Comment by Jaaap — June 26, 2008

Leave a comment

You must be logged in to post a comment.