Wednesday, April 18th, 2007

Performance test results show strong WebKit outcome

Category: Dojo, Editorial

Ryan Breen put out a call for help on a distributed performance test that had many people hitting a Dojo charting system.

The results are now in and the data seems to show:

At 37ms, Safari is easily the fastest implementation, over 3 times faster than Firefox. Internet Explorer is by far the slowest, but this may be a generic statement for the speed of VML operations. Opera was the 2nd fastest browser, narrowly losing out to Safari and turning in a time over twice as fast as Firefox.

There may be some bias in the data given that the majority of people running WebKit Nightlies are likely doing so on sexy multicore Mac hardware.

Firefox on Mac is significantly slower than Firefox on Windows. OK, so either all Firefox Mac users are on boring old PowerPC tech, the Firefox SVG implementation on Mac is slower than Windows, or my Safari hardware bias theory is totally busted. Perhaps a combination of the three. Any way you slice it, Safari’s dominating performance can’t be rationalized away — the WebKit guys have done some seriously good work on the SVG front.

Looks like Microsoft made some improvements to the VML engine in IE7 — there is a significant improvement from IE6. In the SVG space, Firefox appears to have made some marginal improvements in 2.0. Safari SVG support is new, and the per-version sample size on Opera is too low to draw any trend conclusions.

Thanks a lot to the community for helping with the test, and thanks to Ryan for compiling the data.

Posted by Dion Almaer at 6:06 am
14 Comments

+++--
3.9 rating from 29 votes

14 Comments »

Comments feed TrackBack URI

Garbage! At least the author says “There may be some bias in the data”, come back with a more serious test that measures browser performance across the same machines and it will be worth a read.

Comment by Jonathan — April 18, 2007

Also, webkit has sooo many bugs and unimplemented features that some things can’t even be compared. As soon as you get a little deeper into Javascript with the Webkit, it will fail, a good example is support for (any) of the ways WYSIWYG editors work. The contrast between Opera and Webkit is huge, Opera is making a good effort and have come so very far with their implementation, Webkit is a disaster in that area.

Comment by Mark — April 18, 2007

We wanted to thank Ryan publicly for all the help he’s done with this; he’s been great and *really* helpful. On to the peanut gallery…
@Jonathan: it’s always SO helpful to trash something without offering ANYTHING to back up what you’re saying! Thank you so much for that! How about next time you make, I don’t know, some actual suggestions as to *why* it’s garbage?
@Mark: you missed the point of the test. The goal was not to extoll the virtues of WebKit as a superior browser over everything else; it was to point out that it is the best performing in terms of SVG. (And actually the real goal is to be able to come back to the Firefox SVG teams with some real performance data; apparently some of the code of the engines is not, let’s say, the most optimized for performance). At Dojo we’ve had some suspicions for a long time, and were looking for unbiased (as in “test results gathered by someone other than at Dojo) results. Ryan was great for this, and (again) I can’t thank him enough!

Comment by Tom Trenka — April 18, 2007

3x is too big to ignore, even if the data are skewed due to hardware differences, or some other bias. Absent outright deception the likelihood that webkit is not the best performing implementation is vanishingly small.

Comment by George Bailey — April 18, 2007

IMHO the perf boost in IE7 is due to the JavaScript improvements. I’d be surprised if any change had been done on the VML engine but we’ve seen big progress on the JavaScript front.

Comment by Bertrand Le Roy — April 18, 2007

@Mark,
A lot of your complaints are addressed in the particular nightly WebKit builds that make up all of the WebKit participants on the test (all, because the current Safari release also doesn’t have SVG support).

Everything I’ve read says that the WYSIWYG editors generally work in WebKit nightlies (probably due to the fact that they have made it a major focus), and that a lot of the JavaScript problems have been addressed.

That said, why there hasn’t been a new Safari 2.5 release to bring these fixes to users is beyond me. It’s sort of infuriating, actually.

Comment by Trevor — April 18, 2007

yep the webkit SVN build works pretty well on linux – if someone can add alt-right/left and tabs to that basic QT wrapper id love to ditch firepig entirely..

Comment by cdr — April 18, 2007

safari is proably benefiting from proper use of coregraphics more than anything – even tho OSuX was unbearably slow on my 292 mhz mac, you could rotate and zoom a fullpage PDF very fluidly, and this was in 1998.

run this test again w/ firefox using cairo+glitz on linux with hardware OpenGL accel

Comment by cdr — April 18, 2007

Also, webkit has sooo many bugs and unimplemented features that some things can’t even be compared.

I very strongly disagree with that statement. Safari is a definite piece of crap both performance and feature wise as far as Javascript is confirmed, but Webkit (the currently released nightlies from webkit.org), which the test seems to have tracked, is a downright amazing piece of software, and as fantastic a jump in the quality of the JS implementation as Safari 2.0 was in CSS.

In fact, I’d really really like it if Safari 2.1 (the backport of the Safari 3.0 features) was released in June at the former release date of Leopard instead of being released in sync with leopard.

The Safari 2.0 -> Current Webkit is not an evolution of the javascript support of the browser, it’s a downright revolution (they’ve even implemented Javascript 1.6 natively for god’s sake)

Comment by Masklinn — April 18, 2007

I do agree that the Webkit is significantly improved over the current Safari version (Why the hell don’t they release new versions of Safari? Have to wait for next MacOSX version everytime?!). I mean the nightly builds can be top notch, it doesn’t matter if Safari doesn’t get upgraded.

The nightly builds aren’t exactly stable, to me they fix one problem, and break two other fixes. I seems those who develop focus on fixing THEIR problem, and ignore what the code was originally for. We can’t exactly tell everyone to use nightly builds?

Comment by Mark — April 19, 2007

they’ve even implemented Javascript 1.6 natively for god’s sake

Masklinn: so, honestly, implementing JavaScript 1.6 (presumably minus E4X, since you didn’t note that — and I actually think that’s a good decision) natively should not be hard at all. After all, if every single new method on Array can be implemented in JavaScript already, it shouldn’t take that many more lines of code to implement them natively (and it doesn’t, from someone who’s looked at the array extras code in JavaScriptCore and hacked a patch to it).

I can’t say anything about other JS changes they’ve made, but I want to make a point of noting that supporting the new methods of JS1.6 (E4X would be a very different story) is very much not hard to do.

Comment by Jeff Walden — April 19, 2007

The anti-WebKit/anti-Safari banter on this thread just confirms my belief that no matter what data you trot out to prove one thing or another, some people will continue to believe whatever they want. The only noticeable bias in the real world is all the people raised on IE coding who hate Safari. It wasn’t long ago that they hated Firefox, too, but the Mozilla team is at least working cross-platform, and it became pretty hard to hate Firefox on Windows last year.

But aside from just adding noise to the banter, I want the folks here who don’t use Safari to know that many of the point releases of Tiger have included updates to WebKit. Safari in 10.4.9 isn’t the same as it was in 10.4.0. I don’t have a scorecard in front of me showing which Safari releases correspond to which WebKit builds, but I know that there is ongoing transfer of code. In the meantime, any Mac user who really cares about having the latest, greatest browser can download the WebKit nightly and have it all. It’s just too bad the word hasn’t gotten out to a lot of them. Certainly, you can’t blame me for that. :-)

One last point… There’s a good article on the WebKit blog home page at the moment that explains very clearly where WebKit begins and Safari ends. I suggest that everyone who thinks they know the difference go take a quick read and verify your understanding. Even though I thought I understood, I found the visual there helpful.

WebKit is awesome, by the way. And as I pointed out in a blog article recently, you can no longer complain about rich text editing on that platform. So please stop.

Comment by Leland Scott — April 19, 2007

Asking users to run on nightly builds is stupid, sorry but thats the truth. In most “open source” projects ppl are “warned” about using nightly builds, developers can check in unfinished code that would cause the software to fail, even if this might not be the case of the Safari Webkit, it is still “wrong” to teach ppl to run nightly builds.

I don’t hate any browser, I just have to deal with their issues all the time, and some things are just stupid. Part of the problem with Webkit seems to be not WHAT they develop and improve, but HOW they do it, as I wrote before, It is like they go from issue to issue fixing things, but never check back if the changes they make breaks something else, its like two steps forward, one step back.

Comment by Mark — April 20, 2007

Leland Scott you’re a well-known Maczealot so obviously you’re absolutely unbiased about any Apple related thing, aren’t ya.

Comment by Sebhelyesfarku — April 20, 2007

Leave a comment

You must be logged in to post a comment.