Friday, June 19th, 2009>p>As a follow on from the recent post on Microsoft marketing for the week, a few people have given great feedback on what we actually expect from browser marketing. This is a key point that was missing in the post as it gives those watching something to understand that could make things better. This is not about Microsoft, but about all vendors. If we keep seeing claims like this then no user will be able to take anything said from vendor seriously. If everyone claims to be the “fastest browser in the world” then that claim becomes meaningless for example.
We expect that when claims are made there is a detailed rubric on the methods. You can’t just say “we are faster” for example. Right now the claims made on the IE 8 piece are so subjective that if you were in marketing you would think that you are being sneaky “we aren’t teeechnically lying!” That isn’t good enough though.
With the rubric we (developers) should be able to run the tests ourselves to make comparisons. When future browsers come out we should be able to rerun them.
Benchmarks can be gamed, but it is still hugely different that I can take something like SunSpider and run it on my SAME machine for the various browsers. The numbers won’t be the same as on your computer, but it gives me ball park and more importantly comparisons. We can argue if SunSpider actually means anything in the grand scheme of things…. but that is another story.
As a wise man said: “Tests that aren’t repeatable and claims that aren’t testable (or explained) are just lies, no matter who’s telling them.”
What would you like to see in a Bill of Browser Vendor Ethics? :)
Posted by Dion Almaer at 12:44 pm