Tuesday, November 10th, 2009

Sputniktests via Browsers

Category: Browsers, JavaScript, Standards

Sputniktests web runner is a web based porting of the Sputniktests ECMA-262 conformance test suite made by Google.

The aim of this test suite is to understand how compliant is a browser engine and the work @kangax has done in this post is, as usual, complete and loads of analysis about the test suite itself, and provided results.

So how do modern and not so modern browsers stand against standard? Here’s a comparison table (note that less score is better and that score represents total number of errors and failures)


The post goes through different engines behavior over apparently obvious results that could turn into a debug nightmare such:


  1. // what will happen, where, and why?
  2. for (var prop in null) { }
  3. for (var prop in undefined) { }
  5. // is every "casted" space considered zero?
  6. Number("\u00A0") === 0
  8. // is this statement correct?
  9. typeof new RegExp() === 'function'
  11. // what if second argument is undefined?
  12. "test".substring(0, window.whatsup);

I have personally never tried this test suite but accordingly with kangax, it seems to be worth it:

Having extensive compliance test suite can really help modern browsers achieve even better conformance. I hope we’ll see some of the bugs revealed through the Sputniktests fixed in the near future. I hope we’ll also see less regressions, if browser implementors integrate it into existing test suites. I also hope Sputniktests can help people learn and understand ECMAScript better.

Fun enough, Google Chrome does not score better than others but as the post says: it doesn’t always matter!

Last, but not least, Web runner is published on github, so that anyone can contribute easily. Thanks Juriy!

Posted by webreflection at 7:30 am

4.6 rating from 80 votes


Comments feed TrackBack URI

I wonder what happened to Opera 10 to cause a drop from first to almost last place. They’ll probably be all over this. Nice work!

Comment by Jadet — November 11, 2009


David Storey (from Opera) said this yesterday — “It may be that we’ve made some changes to comply more with ECMAScript 5 rather than 3, or real world content, but need to check.” (http://twitter.com/dstorey/status/5595581637)

I’ll try to investigate this myself too if I find time.

Comment by kangax — November 11, 2009

awesome way to push browsers to fix their bugs. IE team on nerves !
By splitting the group in two: the upper 8 are non usable browser !
In this group, except Safari, all are YUI Grade A browsers.
Users love bugs !

Great tools as always, keep’em coming !

Comment by dperini — November 11, 2009

Based on this chart I’m predicting that IE will catch up with FF 3.0 on the standards front after … 167 years :D

Comment by tsonev — November 13, 2009

chrome is #1

Comment by ducminh — November 22, 2009

Leave a comment

You must be logged in to post a comment.