Eyes Above The Waves

Robert O'Callahan. Christian. Repatriate Kiwi. Hacker.

Friday 29 February 2008

Platform Tilt

Vlad blogged about his work on Mac performance and how in the process he (re)discovered Webkit's back door to undocumented Apple framework APIs. This sucks.

Disassembly shows these WK functions are mostly just wrappers around undocumented framework functions. The source to the WK wrappers is not available; the implementations are in a binary blob library that you download with the Webkit sources. It appears the sole purpose of closing the source to this library is to conceal the signatures of the undocumented framework APIs used by Webkit, presumably so that ISVs like us can't use them.

These look like important APIs, especially to browser developers like us. For example, Vlad already blogged about WKDisableCGDeferredUpdates; we can work around the issue for Firefox, but if we had WKDisableCGDeferredUpdates we could avoid having to modify the plist of every application that embeds Gecko. I presume WKDrawFocusRing is used to draw 'outline:auto' in Webkit; we can't implement that feature without that API. WKCGContextIsBitmapContext would clean up some of our cairo code. There are a whole lot of font and text APIs; it's hard to know whether there's a performance benefit to using them or why else they're needed. A lot of the other APIs look like they could be useful but without documentation it's hard to be sure. Presumably in each case, these APIs were added because the Webkit developers found something they couldn't do (fast enough?) with the regular APIs, so of course we're likely to need them too. (With the exception of Apple frameworks we don't use at all, such as the HTTP stack.)

The worry is that the Mac playing field is tilted against us. Linking in the WK library is a losing proposition. We can reverse engineer the hidden APIs but that's painful and fragile. The best we can do is file feature requests with Apple to get hidden APIs exposed, or new functionality that we need exposed. We need to start doing that aggressively. It's hard though, because without knowledge of framework internals, we don't necessarily know when there's functionality we could use if only it was exposed (e.g. a fast path for a particular special case, or a special hidden feature that we can otherwise emulate in a less convenient way). It might not be productive to file requests for every bit of functionality we could possibly use in the hope that some of it's in there. This is where Apple's Webkit team has a permanent leg up on us.

Another implication is that a key part of Webkit on Mac is kept deliberately closed source by Apple. That's unfortunate. Instead of hiding the source, a much more friendly policy for Apple would be to make these APIs public as a matter of course. They may argue that there are unfrozen APIs that they don't want exposed, but there are ways around that, such as by tying symbol names to specific OS versions (CGContextFooBar_10_4?) and promising they'll just not be there in future versions.

It's worth reflecting that if Microsoft was doing this, they'd likely be hauled before a judge, in the EU if not the US. In fact I can't recall Microsoft ever pulling off an undocumented-API-fest of this magnitude.

Of course we will continue to do our best to make Firefox rock on Mac, and version 3 is excellent, even if we have to work harder to make up for lack of access to key APIs. We should be open to reducing our dependence on Apple frameworks where that make sense, although in key cases (graphics, window system) it's not possible. We need to push on Apple to make APIs public instead of concealing them. And we need to pray for open source platforms such as Linux to succeed, so one day we won't have to worry about this anymore.


Brendan Eich
Has Microsoft not done massive secret API fests? How would we know, without open source? Rumors abound going back to the Lotus-123 days.
Don't throw the book at Apple yet. If the WebKit leads mean what they say about being open, they have a chance to prove it here, even if it hurts dealing with the framework teams. It must suck, since AFAICT few ISVs matter to Apple -- how much less might we, who are standing in the way of duopoly junior partner status[*]?
[*] http://farm2.static.flickr.com/1118/549353792_ef59efe867_o.png
Robert O'Callahan
For a long time now, WINE has been able to run Office, IE etc. They're pretty well aware of what secret APIs those apps need to run.
Going back to the DOS days there was less awareness of these issues, so maybe yeah. Although APIs were narrower then and easier to reverse engineer and implement.
Colin Walters
"And we need to pray for open source platforms such as Linux to succeed, so one day we won't have to worry about this anymore."
/me shakes pom-pons
Bill Barry
If you are looking for undocumented APIs in windows, just look at stuff they don't expect people to use.
the shell apis (for custom windows shells like LiteStep or bb4win)
in the .NET framework System.Web.Compilation namespace (though that isn't completely undocumented)
Maciej Stachowiak
Hi Robert, as I mentioned privately, if you file requests for private API to be made public and let me know, I will do my best expedite them. The relevant teams at Apple give higher priority to these requests when they come from third-party developers rather than from the WebKit team.
Brendan, Apple considers Mozilla to be an important ISV. Mozilla Firefox is on a fairly short list of critical third-party apps.
Another difference also is that Microsoft is a lot bigger than Apple on the pc market, and thus is scrutinized more carefully.
But I dont think you should worry too much, if Apple abuses its positions it will not be able to get away easily either. Its not only undocumented APIs, I mean the lock-in to their iTunes could already be the stepping stone for them.
Ian Hickson
The difference between Microsoft doing it and Apple doing it is about 80% of the market. Apple can do it without abusing a monopoly position. Microsoft doing it would mean they were abusing one monopoly (their OS) to gain another (the browser, or whatever).
Not that I'm defending Apple at all; I am a big advocate of free and open source software precisely because it gets rid of this kind of silliness and lets everyone help everyone equally.
Alex Graveley
So when can we expect Mozilla's internal APIs to be exposed with release-versioning? It would have made developing Pyro possible, without petitioning all the way up to Brendan in order to get APIs I needed.
Jesse Ruderman
Firefox uses undocumented OS X APIs, too. See https://bugzilla.mozilla.org/show_bug.cgi?id=366009 for an example.
Sebastian Redl
Microsoft *has* had legal problems due to undisclosed secret APIs. A few years ago, there was a large documentation dump as part of the anti-trust ruling of the EU. It concerned functions that Office used when they were not documented.
So there's definite precedent.
Why not link to WK?
Robert O'Callahan
Alex, since our source is *fully* open, at least you could exactly see what we have and know precisely what to ask for to be exposed.
And if we didn't listen to you, you could make your own Gecko builds exposing whatever you like.
And there's no possibility you'll have to compete with Mozilla PyroFox which has access to internal APIs you don't even know about.
Ian, I think you're wrong about that (and actually that seems pretty obvious to me). Apple can't do this without abusing a monopoly position.
Robert O'Callahan
Jesse: sure do --- carefully and painfully reverse engineered, by others if not by us. I mentioned that as an option, but it's not a good one for us (or for Apple, for that matter). We're certainly not abusing any special privileges by doing so :-).
Jonno: linking in the WK library is not a good idea for a few reasons:
-- it's not clear what the license is, so we don't know if we can redistribute it as part of Gecko
-- the functions are undocumented
-- the functions are specialized for use by Webkit and may not fit our needs. Many of them wrap more general APIs that might be more generally useful.
-- the library is unstable and could change at any time as Webkit evolves
Ian, she: I'm well aware Microsoft is very different from Apple. Apple faces no actual anti-trust issues here, but it's still a sobering thought that they'd be abusing a monopoly if they had one.
Maciej: As I mentioned, it's still tough to know from the outside what it makes sense to ask for. I think it would be sensible to assume that APIs needed by Webkit are likely to be needed by other ISVs. I'm sure we could send someone to explain this to the other Apple teams if that helps :-). Anyway I appreciate your offer and hopefully we'll take advantage of it.
Brendan Eich
I responded to Ian McKellar saying something like what Alex wrote here, over in
Roc made the same point more concisely here, but there's a further element to think about. Whoever has the source, has the power. Yes, we can disassemble, but source code states invariants and intentions (including "this may change, brace yourselves") that aren't in the machine code. With Mozilla, you have all the sources.
We don't expect any project to open up and freeze all internal APIs. Just opening some risks a de-facto freeze. This is all beside the point. Even open source purism is not the point, since we not only use undocumented Mac OS X APIs, we sometimes use unfrozen and under-documented APIs on all platforms.
What's missing is access to critical information, source or doc, however taped over with warning labels, that lets us operate on a level playing field. It's not our "right", but it would be nice. And it would complete the open source intentions of WebKit not to have its Mac build depend on a binary blob.
Maciej Stachowiak
Robert, we're independently trying to get WebKit to use strictly public API (both by dropping unnecessary internal API use, and by getting more things publicized). However, we don't have much more info on these things than what you can tell through decompilation, so for many of these, if you can't tell if you need it from the name and the way it is used in WebKit code, there's not much more we can do to help you make that determination.
Alex Graveley
Roc, the point that open source is important because otherwise I simply can't do what I want is certainly true, from experience.
But if you're saying that internal APIs are fine in Mozilla because I can always fork Mozilla, then that's a little absurdist.
There are overwhelming benefits to being able to work within a piece of software without forking, in ways unimagined by the original authors. As I'm sure you and all the extension developers out there would agree :-)
It's a tough cost/benefit ratio ATM, but I can imagine an automated symbol version scheme for release binaries that could allow all symbols to be public. That would be pretty interesting -- I'm not aware of any platform that's tried it yet.
Robert O'Callahan
I don't think anyone's arguing that it's immoral to have hidden symbols and APIs. It's a practical necessity. I'm concerned about a level playing field where APIs are exposed to everyone or not at all.
Colm Sean Murdoch Ó Cinnéide
i think mozilla should lower the "knowledge bar" the more people that can more easily 'grok' the source code the better.
Colm Sean Murdoch Ó Cinnéide
As Godel said to Bertrand Russell:
"I am going to *rip up* your syntax and turn it into semantics."
It bugs me when I see this kind of blog! Grow up people!! Someone (Apple) is OFFERING you a piece of code (WebKit) at NOT CHARGE!
Use what you can and don't forget to say: "Thank you". For what they don't offer you, guess what, you MIGHT have to do YOUR OWN work!!!
How shocking!
Umm, Microsoft has tons of them (undocumented APIs), mostly in milcore and DirectUI, though one can argue that because the MS apps that use them actually distribute slightly modified versions and then use them from their own directory, it's not an API.
Nope, actually this has nothing to do with Apple, but the fact that the repainting in Fx is broken by design. It's moronic to tell the system "I want to paint more frames than the user ever will see on his display", it's plain waste of CPU resources. Right now Fx works like this:
1) WHILE moreToDo
2) doIt()
3) repaint()
But that is wrong. Repaint could "hang" because it's waiting for a screen refresh. Correct would be
1) WHILE moreToDo
2) WHILE notRepaintNecessary AND moreToDo
3) doIt();
5) repaint();
That way you would only repaint if it makes sense to repaint and the fact that the maximum number of repaints a second is limited does not limit the execution speed of doIt();
Not relevant to this post, but I'd be interested to hear your thoughts on the IE8 announcement, and in particular webslices ;)
Wow Mecki78! You should submit that second loop as a patch or something...
And there I was, thinking the problem was somehow complex.
Bielawski, it's also important to make the distinction between the kind of undocumented APIs. You have the low level ones which, even if you wanted to, you couldn't reproduce unless you used some undocumented methods. There's also the higher level toolkits that are built from these low-level APIs which do provide a competitive advantage on their own but they do not *prevent* any competition.
You can always write those higher level ones yourself (at a cost of course) but there's nothing you can do about the low level ones.
As for the two examples you mention... if everyone is making their own little changes to the APIs before deployment, it might be that they are not quite ready for general consumption yet...
The problem is not complex, the solution neither, the Fx code is the complex thing here.
I'm a Mac programmer for years. I'm writing Mac applications for years. Every App that does some heavy painting is designed to work as shown in my second loop right from scratch. If you have that already in mind when you start planing the app, you will never have any problem with the repaint refresh interval. The system just tries to avoid unecessary repaints, which really makes a lot of sense. Every 3D game works like that.
However Fx is not a native Mac app. It has not been planned that way right from scratch. Though I would argue that the current implemtation is as ironic for any other platform (on Linux and Windows having a higher repaint rate than the screen refresh rate makes so absolutely no sense), these platforms seem to not limit it by default and that's why Fx is running better on them.
Now, having this "beast" of thousands of line of code, fixing mistakes that have probably been done when the first release of Mozilla Browser was planed years ago is definitely complex.
I guess that's the same reason Apple is doing it wrong in Safari, too. Safari could easily do it right, no problem. But Webkit is nothing more than the Linux KHTML engine ported to Mac and this engine is doing it wrong and Apple probably thought it's way too complex to fix this that late in the development process.