Friday, May 23rd, 2008

GUIMark: Another Web application benchmark

Category: Performance

Sean Christmann has introduced GUIMark, a benchmark for HTML, Flex, Silverlight, Swing, and anything else you port it too.

Sean saw Bubblemark and thought that there was room for a different type of benchmark:

GUIMark takes a different approach by trying to benchmark the types of UI elements common in our Web 2.0 world. This includes things like vector redraws, alpha transparencies, text reflow, bitmap motion, and 9 scale slicing rules. From there I just fill up the render pipeline until it becomes so over-saturated that it becomes easy to visually distinguish which rendering engines are more efficient then others. As a result, the benchmark is more complicated on a visual level and requires a bit more time then Bubblemark to understand the implementation rules. Lastly with GUIMark I’ve tried to get into some of the lower level details behind how rendering engines work and how that’s affected the creation of this project.

You can run the HTML version to see it in action.

The results have been a little strange, with huge differences between Mac OS X and Windows. The Ajax application wins on the Mac!

Sean concluded:

I’ve been surprised with the results so far between WinXP and OS X. On the same machine its very clear which vendors take more advantage of the underlying hardware. The results for the different plugin technologies aren’t too surprising since it’s regularly admitted that most companies spend their optimization time on Windows due to its larger install base. This argument doesn’t hold any water though when comparing html rendering on Safari/Mac against IE /Windows where there’s roughly a 1.6 : 1 advantage to the IE team. I can’t help but wonder if the core apis on the Mac platform are creating any unnecessary roadblocks. I’m also extremely surprised at the rendering speed that Flash is able to pull off on Windows. I developed this benchmark under OS X and after compiling the results I’m considered making the testcase more intensive since Flash is running so fast, but for now maybe the really poor Mac performance will give Adobe something to work on.

It is always nice to see benchmarks, as long as we don’t get too carried away. For one, when you take a look at the application, you will quickly see that it isn’t exactly a “real world application” unless you are building a crazy scaling game ;)

Posted by Dion Almaer at 7:43 am

3.7 rating from 18 votes


Comments feed TrackBack URI

My results for the HTML benchmark:

FF3 RC1: 5fps
Opera 9.50b2: 14fps
Safari 3.1.1: 13fps (varies)
IE7: 17.5fps (wasn’t expecting that)
IE6: 12fps

Comment by Anonymous — May 23, 2008

Sorry, I was running Opera 9.27.

Opera 9.50b2: 22fps

Which means Opera is the winner (for now)!

Comment by Anonymous — May 23, 2008

Ahem, I also forgot I had Firebug turned on.

Unfortunately disabling Firebug only improved FF’s score by 1fps.

Comment by Anonymous — May 23, 2008

Thanks for the writeup Dion. I actually wrote this benchmark while I was developing eBay Desktop which has transitional effects that are comparable to whats being stressed in the test. Technically only the stars are being scaled in by the renderer, the rest of the benchmark is actually testing resizing, relayout and x y motion. Many popular sites are using resizing transitions which is why I thought it would be relevant. Both Digg and Slashdots commenting system implement resizing div containers, richer apps like Sliderocket and Buzzword have to reflow text and animate in panels. Sites like Twitter and Facebook will alpha fade in new user activity. Of course most people aren’t going to add all these things in at once, but then I wouldn’t be left with much of a benchmark :)

Comment by Craftymind — May 23, 2008

Although in my tests FF had the lowest score, I would also like to add that I also did some custom benchmarks on first-time rendering of a bunch of generated elements. The first time they are rendered FF is many times faster than IE. That being said FF is still a bit too slow for my liking for dynamically resizing already existing elements. Maybe the final build won’t have this problem, I can only hope.

Comment by Anonymous — May 24, 2008

Leave a comment

You must be logged in to post a comment.