Friday, February 29th, 2008

Firefox 3 Performance Numbers

Category: Browsers, JavaScript, Performance

<p>

Moving on from the “let me use that API” conversation and only some real stuff, urandom (thanks for the comment) let us know about the Cybernet News article on Firefox 3 performance.

They are reporting that Firefox 3 is now faster than Safari 3, and is close to WebKit nightly in certain benchmarks. I can just picture Steve coming down on people saying “we market this as the fastest browser on the planet!” which is tough, as noone stays the fastest for ever. It is a race, and I am sure that WebKit and Firefox will be switching spots a lot in recently years.

Again, this is great news for developers. I am running WebKit and Firefox 3b3 and I am really happy with both. For some tasks I choose one over another (e.g. Firebug, Greasemonkey vs. lean and mean).

I’m sure what most of you care the most about are the facts, and so I’ve compiled the results of the SunSpider JavaScript Benchmark test for each of the different browsers. All of the tests below were performed on the same Windows machine, and the Firefox 3 nightly builds definitely came out on top. Here are the results sorted from best to worst (each one is hyperlinked to the full stats):

  1. Firefox 3 Nightly (PGO Optimized): 7263.8ms
  2. Firefox 3 Nightly (02/25/2008 build): 8219.4ms
  3. Opera 9.5.9807 Beta: 10824.0ms
  4. Firefox 3 Beta 3: 16080.6ms
  5. Safari 3.0.4 Beta: 18012.6ms
  6. Firefox 2.0.0.12: 29376.4ms
  7. Internet Explorer 7: 72375.0ms

It’s important to know that every time you run the SunSpider Benchmark it conducts each test five times, and the result is the average of the five tests. So it is a rather thorough test, and definitely shows off the speed improvements that Firefox 3 is going to be bringing to the table.

Brendan has said that they are not finished with their performance work for Firefox 3, and I am sure the WebKit team isn’t sitting on their hands…. oh and what about IE 8? It will be fun when that is in the wild to be tested.

Related Content:

22 Comments »

Comments feed TrackBack URI

Impressive for Firefox indeed, though WebKit is definitely not sitting on their hands. The latest nightly from them (r30667 2008-02-29) posts some impressive numbers:

3152.4ms

http://webkit.org/perf/sunspider-0.9/sunspider-results.html?%7B%223d-cube%22:%5B131,125,126,126,124%5D,%223d-morph%22:%5B136,132,129,132,131%5D,%223d-raytrace%22:%5B130,130,130,128,128%5D,%22access-binary-trees%22:%5B66,67,68,67,65%5D,%22access-fannkuch%22:%5B234,232,233,232,231%5D,%22access-nbody%22:%5B135,137,136,138,139%5D,%22access-nsieve%22:%5B57,60,58,57,57%5D,%22bitops-3bit-bits-in-byte%22:%5B68,67,65,64,67%5D,%22bitops-bits-in-byte%22:%5B97,97,94,97,95%5D,%22bitops-bitwise-and%22:%5B167,158,171,159,168%5D,%22bitops-nsieve-bits%22:%5B104,101,104,102,101%5D,%22controlflow-recursive%22:%5B88,86,84,85,89%5D,%22crypto-aes%22:%5B77,81,80,79,79%5D,%22crypto-md5%22:%5B77,78,80,77,76%5D,%22crypto-sha1%22:%5B76,77,77,78,78%5D,%22date-format-tofte%22:%5B126,131,126,128,129%5D,%22date-format-xparb%22:%5B151,152,153,151,151%5D,%22math-cordic%22:%5B167,166,168,166,169%5D,%22math-partial-sums%22:%5B179,186,184,185,185%5D,%22math-spectral-norm%22:%5B79,81,77,80,80%5D,%22regexp-dna%22:%5B189,191,189,190,195%5D,%22string-base64%22:%5B93,92,91,92,90%5D,%22string-fasta%22:%5B173,170,173,176,174%5D,%22string-tagcloud%22:%5B125,124,121,120,120%5D,%22string-unpack-code%22:%5B124,124,125,127,126%5D,%22string-validate-input%22:%5B109,108,107,109,110%5D%7D

Comment by danbedford — February 29, 2008

CONGRATULATION TO THE MOZILLA TEAM, THIS IS A GREAT STEP FOR MANKIND!!!!

Comment by polterguy — February 29, 2008

It’s nice to see that the Mozilla guys are really focusing on what they’ve been cricitised for in the past- performance, memory leaks and so on make up the major complaints against the already excellent browser. Bring it on, WebKit! :)

Comment by JamesHarrison — February 29, 2008

@danbedford:

can I trouble you to run that test on the same machine, but with the firefox nightly (if you can get the pgo optimized build, that’d be even better).

Comment by urandom — February 29, 2008

…isn’t it great to see competition in the browser?!?! Great news!

Comment by Mark Holton — February 29, 2008

You have to admit the only poor results IE7 has is string performance… So IE8 could be sitting in a good position as well. I guess we might find out in a few weeks…

Comment by JustinCarter — February 29, 2008

Someone (me, I guess) should point out that javascript performance – the metric measured in the article – has little or nothing to do with DOM rendering, i.e. how fast a page is rendered.

Most of the “wow!” comments I see related to Safari 3 are from end users impressed with it’s page rendering performance (I think). I’d love to see a comparison of how fast these browsers render different types of web pages. (For example, test how well they render large tables, and CSS-heavy pages.)

As a heavy javascript developer I’m all in favor of increased javascript engine performance, but for the other 99% of the world, the speed with which a browser can render large/complex web pages is probably much more important.

Comment by broofa — March 1, 2008

Safari 3.0.4 on Mac OS X Leopard: 7608.0ms

http://webkit.org/perf/sunspider-0.9/sunspider-results.html?%7B%223d-cube%22:%5B358,358,357,353,373%5D,%223d-morph%22:%5B434,429,441,432,427%5D,%223d-raytrace%22:%5B293,289,298,291,308%5D,%22access-binary-trees%22:%5B125,116,115,119,116%5D,%22access-fannkuch%22:%5B686,688,694,683,683%5D,%22access-nbody%22:%5B339,338,338,335,336%5D,%22access-nsieve%22:%5B429,432,423,440,424%5D,%22bitops-3bit-bits-in-byte%22:%5B308,270,272,272,271%5D,%22bitops-bits-in-byte%22:%5B372,363,365,358,365%5D,%22bitops-bitwise-and%22:%5B359,351,388,363,370%5D,%22bitops-nsieve-bits%22:%5B393,388,391,398,398%5D,%22controlflow-recursive%22:%5B185,211,194,198,178%5D,%22crypto-aes%22:%5B181,181,183,180,181%5D,%22crypto-md5%22:%5B226,226,226,243,231%5D,%22crypto-sha1%22:%5B222,229,227,223,221%5D,%22date-format-tofte%22:%5B204,208,209,206,206%5D,%22date-format-xparb%22:%5B199,212,198,200,201%5D,%22math-cordic%22:%5B520,517,503,514,507%5D,%22math-partial-sums%22:%5B235,235,236,236,236%5D,%22math-spectral-norm%22:%5B209,218,209,216,209%5D,%22regexp-dna%22:%5B283,284,285,285,283%5D,%22string-base64%22:%5B240,241,244,240,239%5D,%22string-fasta%22:%5B280,290,290,283,283%5D,%22string-tagcloud%22:%5B215,191,192,199,188%5D,%22string-unpack-code%22:%5B167,165,166,168,164%5D,%22string-validate-input%22:%5B175,174,175,174,173%5D%7D

Comment by nkm — March 1, 2008

@nkm: Unfortunately a single test of a single browser is not a proof of anything, for better or worse; all the other browsers need to be tested on a same machine and same setup for the numbers to mean anything – you can only compare the numbers to one another if this is true. Otherwise they’re just random numbers (since they depend on the machine / setup the test was run in… for example, if I were to run the current contest winners on a 486 and IE on a brand new machine, I’m pretty certain that on that test IE would seem like a clear winner:)

Comment by DiscoNova — March 1, 2008

@DiscoNova

Of course, I was only pointing out that Safari is not that slow on Leopard… By the way, I’m downloading FF3b4 to test it too :)

Comment by nkm — March 1, 2008

Firefox 3.0b4pre (2008030104) on Mac OS X Leopard: 3609.0ms

http://webkit.org/perf/sunspider-0.9/sunspider-results.html?%7B%223d-cube%22:%5B163,169,166,165,165%5D,%223d-morph%22:%5B118,120,117,119,116%5D,%223d-raytrace%22:%5B140,228,140,135,139%5D,%22access-binary-trees%22:%5B79,72,72,73,72%5D,%22access-fannkuch%22:%5B201,205,202,202,201%5D,%22access-nbody%22:%5B209,169,168,160,174%5D,%22access-nsieve%22:%5B56,57,59,58,57%5D,%22bitops-3bit-bits-in-byte%22:%5B51,51,52,51,52%5D,%22bitops-bits-in-byte%22:%5B74,74,75,74,74%5D,%22bitops-bitwise-and%22:%5B172,163,165,160,168%5D,%22bitops-nsieve-bits%22:%5B112,113,112,112,113%5D,%22controlflow-recursive%22:%5B119,119,120,118,117%5D,%22crypto-aes%22:%5B88,89,93,93,93%5D,%22crypto-md5%22:%5B108,107,107,108,106%5D,%22crypto-sha1%22:%5B110,111,115,112,111%5D,%22date-format-tofte%22:%5B216,216,211,217,217%5D,%22date-format-xparb%22:%5B138,137,140,138,137%5D,%22math-cordic%22:%5B177,175,179,179,174%5D,%22math-partial-sums%22:%5B185,183,193,193,205%5D,%22math-spectral-norm%22:%5B121,122,120,119,119%5D,%22regexp-dna%22:%5B223,227,223,224,224%5D,%22string-base64%22:%5B80,79,78,78,79%5D,%22string-fasta%22:%5B215,222,214,213,214%5D,%22string-tagcloud%22:%5B153,155,155,153,154%5D,%22string-unpack-code%22:%5B201,206,205,202,205%5D,%22string-validate-input%22:%5B110,109,109,109,107%5D%7D

Touche! :P

Comment by nkm — March 1, 2008

If you’re going to compare to other browsers, compare apples to apples… WebKit nightlies have been reported as being even faster.

Comment by tlrobinson — March 1, 2008

Why are so many commenters posting single numbers?
They’re meaningless, folks — you have to test at least TWO browsers for the number to mean anything at all.

Think about it — every result is going to be affected in varying degrees by the memory, processor, OS, disk fragmentation, virus-detection software (and/or actual viruses!), other running software, thousands of configuration options on your OS & computer & the browsers themselves….

If you want to post numbers, do a fresh install of the browsers you’re testing (make sure they aren’t using existing profiles…), reboot your computer, and test them all running one browser at a time.

If you aren’t doing this, at least mention that you aren’t. If you aren’t even posting more than one number, just don’t bother. It’s like saying, “which is better, or an apple?”

Comment by jtheory — March 2, 2008

I’ve tested myself browsers performances for 3DES passwords cracking…
here’s my results:
Javascript password cracking

Comment by binduck — March 3, 2008

http://ardoino.com/32-javascript-online-password-cracking/

Comment by binduck — March 3, 2008

Mhmm. Pretty sweet! Congrats to the Firfox team ^_^

My numbers:

http://webkit.org/perf/sunspider-0.9/sunspider-results.html?%7B%223d-cube%22:%5B731,738,686,701,704%5D,%223d-morph%22:%5B1577,1593,1557,1555,1580%5D,%223d-raytrace%22:%5B478,486,485,487,471%5D,%22access-binary-trees%22:%5B312,221,215,218,258%5D,%22access-fannkuch%22:%5B620,614,621,628,621%5D,%22access-nbody%22:%5B775,753,751,850,727%5D,%22access-nsieve%22:%5B386,334,326,327,329%5D,%22bitops-3bit-bits-in-byte%22:%5B520,469,474,474,470%5D,%22bitops-bits-in-byte%22:%5B428,403,399,409,467%5D,%22bitops-bitwise-and%22:%5B1338,1270,1215,1234,1218%5D,%22bitops-nsieve-bits%22:%5B458,410,410,407,404%5D,%22controlflow-recursive%22:%5B251,202,144,151,153%5D,%22crypto-aes%22:%5B397,361,379,371,359%5D,%22crypto-md5%22:%5B332,215,219,218,219%5D,%22crypto-sha1%22:%5B275,277,260,284,277%5D,%22date-format-tofte%22:%5B605,570,576,630,624%5D,%22date-format-xparb%22:%5B423,367,366,358,366%5D,%22math-cordic%22:%5B838,794,748,746,778%5D,%22math-partial-sums%22:%5B778,752,766,739,769%5D,%22math-spectral-norm%22:%5B455,392,394,372,401%5D,%22regexp-dna%22:%5B507,442,439,435,437%5D,%22string-base64%22:%5B774,748,761,749,739%5D,%22string-fasta%22:%5B732,684,676,694,695%5D,%22string-tagcloud%22:%5B543,474,478,470,470%5D,%22string-unpack-code%22:%5B954,867,927,920,926%5D,%22string-validate-input%22:%5B345,316,312,313,307%5D%7D

and

http://webkit.org/perf/sunspider-0.9/sunspider-results.html?%7B%223d-cube%22:%5B340,385,380,321,329%5D,%223d-morph%22:%5B417,344,417,462,403%5D,%223d-raytrace%22:%5B382,392,342,340,322%5D,%22access-binary-trees%22:%5B168,195,160,212,156%5D,%22access-fannkuch%22:%5B690,630,660,601,684%5D,%22access-nbody%22:%5B354,348,367,387,358%5D,%22access-nsieve%22:%5B168,230,194,192,188%5D,%22bitops-3bit-bits-in-byte%22:%5B197,227,203,220,203%5D,%22bitops-bits-in-byte%22:%5B351,294,318,282,283%5D,%22bitops-bitwise-and%22:%5B325,353,377,345,328%5D,%22bitops-nsieve-bits%22:%5B416,380,363,359,300%5D,%22controlflow-recursive%22:%5B217,204,269,212,200%5D,%22crypto-aes%22:%5B244,214,216,293,212%5D,%22crypto-md5%22:%5B251,198,228,225,247%5D,%22crypto-sha1%22:%5B224,301,248,245,233%5D,%22date-format-tofte%22:%5B373,377,335,342,347%5D,%22date-format-xparb%22:%5B410,402,392,445,492%5D,%22math-cordic%22:%5B541,547,520,495,559%5D,%22math-partial-sums%22:%5B415,427,412,397,472%5D,%22math-spectral-norm%22:%5B291,226,262,272,332%5D,%22regexp-dna%22:%5B491,479,554,515,505%5D,%22string-base64%22:%5B288,344,267,306,271%5D,%22string-fasta%22:%5B455,480,425,461,458%5D,%22string-tagcloud%22:%5B360,330,371,316,294%5D,%22string-unpack-code%22:%5B332,302,350,285,246%5D,%22string-validate-input%22:%5B317,341,289,312,333%5D%7D

Firefox 3 beta 3: 14935.4ms

Webkit Nightly: 8896.6ms

Comment by Carbon43 — March 4, 2008

@Carbon43
Would get better result using Nightly builds!
By another words Firefox got quicker Javascript engine for NOW anyway.

Comment by styledhtml — March 5, 2008

IE8 Beta
Total: 7726.6ms +/- 0.8%

http://webkit.org/perf/sunspider-0.9/sunspider-results.html?%7B%223d-cube%22:%5B278,280,279,278,276%5D,%223d-morph%22:%5B217,219,222,221,221%5D,%223d-raytrace%22:%5B329,323,321,322,320%5D,%22access-binary-trees%22:%5B228,228,227,230,227%5D,%22access-fannkuch%22:%5B560,550,572,547,557%5D,%22access-nbody%22:%5B315,314,311,314,310%5D,%22access-nsieve%22:%5B162,157,157,159,159%5D,%22bitops-3bit-bits-in-byte%22:%5B152,146,152,149,146%5D,%22bitops-bits-in-byte%22:%5B272,194,206,199,199%5D,%22bitops-bitwise-and%22:%5B408,393,390,393,391%5D,%22bitops-nsieve-bits%22:%5B227,228,228,228,227%5D,%22controlflow-recursive%22:%5B211,207,208,207,206%5D,%22crypto-aes%22:%5B244,248,252,247,244%5D,%22crypto-md5%22:%5B154,150,155,151,150%5D,%22crypto-sha1%22:%5B154,153,154,154,153%5D,%22date-format-tofte%22:%5B321,318,322,318,320%5D,%22date-format-xparb%22:%5B287,285,291,288,289%5D,%22math-cordic%22:%5B305,302,302,304,307%5D,%22math-partial-sums%22:%5B216,214,222,212,213%5D,%22math-spectral-norm%22:%5B210,213,212,210,212%5D,%22regexp-dna%22:%5B360,364,361,353,367%5D,%22string-base64%22:%5B1100,1095,1094,1106,1102%5D,%22string-fasta%22:%5B329,325,326,335,334%5D,%22string-tagcloud%22:%5B249,248,246,247,250%5D,%22string-unpack-code%22:%5B264,267,273,268,266%5D,%22string-validate-input%22:%5B253,258,266,255,259%5D%7D

Comment by Zim312 — March 5, 2008

@styledhtml: No I’m sorry, You are incorrect. FF3 nightlies are faster than Safari Betas, but not faster than Webkit nightlies. I’ve been testing.

Comment by Carbon43 — March 6, 2008

Firefox 3 performance review:
on linux: http://www.mininglabs.com/2008/06/16/firefox-3-an-empirical-performance-study/
on windows:
http://blog.pavlov.net/2008/03/11/firefox-3-memory-usage/

Comment by elnarec — June 17, 2008

I wonder how Google’s new chrome browser is doing in comparison with FF 3.0.

Comment by notsotrivial — September 4, 2008

500ish

Comment by Spense — October 9, 2009

Leave a comment

You must be logged in to post a comment.