Friday, February 29th, 2008

Firefox 3 Mac performance gains due to undocumented APIs

Category: Browsers, Firefox, WebKit

<p>

Vladimir Vukićević has posted on the performance improvements of Firefox 3 on the Mac, and how one hack, albeit “dangerous” has helped ton. Vladamir says:

While figuring all this out, I noticed that Safari/WebKit didn’t seem to be affected by this framerate cap — the fps meter when Safari was running the same benchmark happily went up beyond 60fps.  After I found the plist entry, I checked Safari’s plist and was surprised to discover that they didn’t have this disabling in there.  Doing some more searching, I found this code in WebKit.  Apparently, there is a way to do this programatically, along with some other interesting things like enabling window update display throttling (though it’s unclear what that means!) — but only if you’re Apple.

All these WK* methods are undocumented, and they appear in binary blobs shipped along with the WebKit source (see the WebKitLibraries directory).  There are now over 100 private “OS-secrets-only-WebKit-knows” in the library, many of which are referred to in a mostly comment-free header file.  Reading the WebKit code is pretty interesting; there are all sorts of potentially useful Cocoa internals bits you can pick up, more easily on the Objective C side (e.g. search for “AppKitSecretsIKnow” in the code), but also in other areas as a pile of these WK* methods used in quite a few places.  Would any other apps like to take advantage of some of that functionality?  I’m pretty sure the answer there is yes, but they can’t.  It’s not even clear under what license libWebKitSystemInterface is provided, so that other apps can know if they can link to it.

David Hyatt, the guru lead of Webkit/Safari commented:

The programmatic disabling of coalesced updates should not be public API. It’s actually a very dangerous thing to do. We aren’t really happy with that code in WebKit, but we had to do it to avoid performance regressions in apps that embedded WebKit. Technically it’s wrong though, since we turn off the coalesced updates for any app that uses WebKit! This includes drawing they do that doesn’t even use WebKit.

As for the window display throttling, that was a pref designed for Safari (that we don’t even use any more). It’s not private or magic. It’s nothing more than a pref that we can examine from Safari-land, so linking to that is just silly. ;)

Many of the private methods that WebKit uses are private for a reason. Either they expose internal structures that can’t be depended on, or they are part of something inside a framework that may not be fully formed. WebKit subclasses several private NSView methods for example, and it cost us many many man hours to deal with the regressions caused by the internal changes that were made to NSViews in Leopard.

As you yourself blogged, there was a totally acceptable public way of doing what you needed to do.

For any private methods we use that we think should be public, we (the WebKit team) file bugs on the appropriate system components. Many of these methods have become public over time (CG stuff in Leopard for example). Be careful when you dig into WebKit code, since we may continue to use the WK method even though it’s not public API just because we need to work on Tiger.

Robert O’Callahan is much more aggressive and claims platform tilt:

The source to the WK wrappers is not available; the implementations are in a binary blob library that you download with the Webkit sources. It appears the sole purpose of closing the source to this library is to conceal the signatures of the undocumented framework APIs used by Webkit, presumably so that ISVs like us can’t use them.

A key part of Webkit on Mac is kept deliberately closed source by Apple. That’s unfortunate. Instead of hiding the source, a much more friendly policy for Apple would be to make these APIs public as a matter of course. They may argue that there are unfrozen APIs that they don’t want exposed, but there are ways around that, such as by tying symbol names to specific OS versions (CGContextFooBar_10_4?) and promising they’ll just not be there in future versions.

It’s worth reflecting that if Microsoft was doing this, they’d likely be hauled before a judge, in the EU if not the US. In fact I can’t recall Microsoft ever pulling off an undocumented-API-fest of this magnitude.

WebKit flies on my Mac, and Firefox 3 has almost caught up. The end result is that I am pretty happy with how browsers are improving their performance, and I am sure there is a lot more to see.

Related Content:

Posted by Dion Almaer at 7:36 am
11 Comments

+++--
3.9 rating from 17 votes

11 Comments »

Comments feed TrackBack URI

Its nice to see how frank and upfront the WK devs are when discussing the code.

Good to hear that they are trying to move away from this sort of specialized code. With WK being adopted by so many companies (adobe, et al) it’s important to limit the private APIs as much as possible.

Keep up the great work!

Comment by Carbon43 — February 29, 2008

Here we go again, EEE…
No wonder they can keep the comparisons at apple.com/safari so brilliantly towards their incentives as they do ;)

Comment by polterguy — February 29, 2008

this post should go hand in hand with the recent news about javascript performance boost in ff3b4:
http://cybernetnews.com/2008/02/25/firefox-3-performance-gets-a-boost/

Comment by urandom — February 29, 2008

If IE on Windows used private API’s that they told eveyone else not to use….

Comment by Steve Roussey — February 29, 2008

@polterguy:

Yep. Sometimes unfinished/unstable internal APIs perform better. This is nothing new, not for Apple, not for MS, not even for the Linux community. The difference in the FOSS community is that instead of not documenting unsupported/unfinished/unstable APIs, they simply mark it as unstable and you install it at your own risk with the understanding that it might get broken if one of its dependencies changes.

And as the APIs become stable, they become documented and find their way into other projects. Go figure.

Webkit does, as D Hyatt mentions, a bunch of stuff against undocumented APIs as a necessity, but they try to resolve that as Apple makes the APIs’ functionality available. The same is true at MS, and the same is probably true of many large third party applications that run on those OSes (notice Mozilla Corp had no problem adopting Webkit’s non-standard API calls!).

Comment by Trevor — February 29, 2008

@Trevor
Your logic sounds reasonable, the problem is that this is just a drop in a long chain of EEE from Apple, the last time I read about EEE from Apple it was about some “onscreenflipped” JavaScript event handler they had built for the Safari iPhone version.
Fits right into the “Extend” phase of EEE if you ask me…
I remember when I was first made aware of EEE from Apple, it was just after Steve took over and the iPod was launched chemically cleansed for the opportunity of interacting with songs from other sources than apple in addition to that the “apple songs” could also not be played on other HW. If that ain’t EEE then I don’t know what EEE is…
Safari has turned out to be a *great* browser feature wise and in regards to performance, and I think it’s great to have competition, I just don’t want to have one dictator changed for another one, where the new dictator even have a *WORSE* track record in regards to EEE than the previous one… :(
Apple is *not* better than MSFT, in fact on most parts they’re worse, people just don’t see it that way since we’re used to think of MSFT when someone mentions EEE, but if you study their business you’ll pretty soon have to acknowledge that Apple is by far worse than MSFT. The perfect world would have them “equal out” each other with *nix and FireFox having a third of the market. The problem is that Apple won’t settle for a third and they’re in the position to outperform MSFT in market shares within the next decade…
Apple is one of the “bad guys” and I think it’s time for people to notice that fact, before we get “Idi Amin” exchanged with “Saddam Hussein”…
Go FireFox…!
You have a graph at apple.com/safari to beat!
Go Mozilla…!

Comment by polterguy — February 29, 2008

@Mozilla
…continued…
And for God’s sake; DO NOT LOOSE FIREBUG!
It’s basically the foundation of your existence now…

Comment by polterguy — February 29, 2008

Uhh…Zealot much? ;)

Both FF and WK are A OK in my book. I Use WK at work, and FF at home. I think I’m satisfied by the explanation given for the lack of availability of the APIs.

While it may be true that MS does the same thing with not releasing their proprietary APIs, Apple is not doing it for monopolistic purposes, has openly stated that it prefers to keep the documentation open, and has backed those comments up with actions.

Comment by Carbon43 — February 29, 2008

@polterguy:
I’m really curious how “onscreenflipped” could “extinguish” KHTML. In fact, given Apple’s eventual contributions to KHTML and the fact that the KHTML codebase switched to WebKit, I’m curious how you can make an argument at all that Apple is trying to extinguish it.

As far as the iPod, it can play songs in all kinds of standard formats, including MP3, AIF, WAV. I’ve never bought a song from the iTunes Store, and my iPod works just fine.

I never said Apple was any better than MS. Their differences at the moment can be attributed exactly to marketshare. Other than that, they’re both approaching market dominance in strategically different but morally equal terms. Apple and MS deserve criticism for a lot of things, but that doesn’t mean they deserve poor criticism on a poor logical foundation.

For instance, a better argument than Webkit might be (or might have been a few months back) the use of various BSD frameworks but the closing of i386 Darwin source for, what, 2 years? But then, they opened the source again. LOL. A better argument than the iPod might be simply the iTunes Store, which is definitely about lock-in. Apart from Apple’s free software, what does the iPod lock you into?

Comment by Trevor — February 29, 2008

@Trevor
You are of course right in your saying, and the fine-grained arguments of my comment became far better with these minor adjustments. Sorry for being fussy with the details, though thank you for supporting my main claim which still stands tall as a mountain which is that Apple are doing bad, bad business. I just find it sad that so few are able to see it… :(

Comment by polterguy — March 1, 2008

The problem I have with all of these arguments, of course, is that Apple (and Dell and MS and any other equipment manufacturers, optical media distributors, and other tech industry manufacturers) is doing far worse than EEE, and the arguments that [insert tech corporation here] is evil and like some mass murderer because they are badly supporting people’s programming work—rather than because they’re poisoning thousands of people, and very commonly imprisoning for the purpose of enslaving (or sometimes using imprisoned workers even in the US, but minimally) thousands of people to minimize profits. “Saddam Hussein” might be irresponsibly using the KHTML source, but the banality of evil of maximizing profit by slavery and toxification of untermenschen in backwater countries first worlders don’t care about (or backwater prisons in the first world, which first worlders also don’t care about) is a much bigger deal, and a great deal more frightening.

I have a hard time stomaching arguments that MS or Apple is evil because of their treatment of the GNU crowd, when they’re genuinely destroying lives without any objection being raised. You know, it’s like complaining about the religious views of certain ATF members while they burn children alive in Waco.

[I figure since Godwin's Law was near invocation a few posts back, I might as well keep the analogy around and even it out some.]

Comment by Trevor — March 1, 2008

Leave a comment

You must be logged in to post a comment.