Sunday 7 October 2007
Changing Horses
Matt at Allpeers posted about the pros and cons of Mozilla switching from Gecko to Webkit.
This is something that people in Mozilla have discussed, multiple times, because the pros are quite clear. We get feedback from many channels that Webkit is cleaner and a lot easier to hack on, and I believe that; its code footprint is also clearly better. Beyond that, open source Web browsers and the Web itself could make progress much faster if we were pulling together on the same code base instead of doing everything twice. It would also reduce compatibility pain for Web developers.
The cons are also quite clear, however. We really really really don't want to leave XUL developers --- that is, developers of Firefox addons and XULrunner applications, not to mention our own Firefox and Thunderbird developers --- in the lurch. So XUL and attendant technologies (XBL(2), some XPCOM glue) would have to be ported. Generally any Gecko features that Webkit doesn't have would have to be ported. Considerably bug-compatibility would be inevitably lost.
It's not clear how project management would work. Apple probably doesn't want all that stuff in their tree. Anyway, it would be unsafe for the Web to put all our eggs in their basket.
Some people argue that having two open source Web engines is good because it avoids the problems of monoculture. It can also help in standards bodies to have multiple interoperating implementations of a feature. There's some truth to it, although I think I'd rather have one awesome open source implementation than two less awesome ones. (I think in the long run, a single dominant open source system would be a healthier, more monopoly-resistant software infrastructure than several competing implementations unified via fractious standards bodies. I also think that the split between KDE and GNOME is about the worst thing that ever happened to the free software world.)
The biggest problem, however, is getting from here to there. Mozilla can't leave the market for a few years to attempt a risky transition plan. That would be bad for us and bad for the Web. We have to keep improving Gecko to get maximum leverage from our existing market share, mindshare, and developer share. Also, as I suggested in my second-to-last post, Webkit is not the epitome of engines. It would suck to spend all our resources getting there and then discover we should have gone in another direction.
None of this rules out Mozilla --- or a third party --- doing a XUL-on-Webkit project on the side! I actually think it could be terrific territory for a startup ... far more worthwhile than yet another buzzy social networking site. Or, dare I say it, Flock.
I also think we should keep working on converging the XUL universe with the Web. That means projects like standardizing flex-box layout as a CSS3 module, implementing XBL2, implementing WHATWG drag-drop and bringing it to XUL, ditto for WHATWG SQL APIs, getting missing XUL widgets into HTML5, perhaps creating better standardized ways to bind native code to Javascript, WHATWG APIs for Webrunner-style desktop application management, and more. This would be good for the Web and eventually make it easier to port XUL apps to another engine.
Lastly, let me deflect the charge of "Not Invented Here"ism by stating for the record that I have no great love for Gecko. I did not invent it, I think it has major flaws, and I would be most happy if it vanished and was replaced with something superior in every way. For the forseeable future, that's not an option.
Comments
Wouldn't taking gecko now and simplifying it as you did with display lists and dbaron with reflow refactoring (and you seem to still have a lot of ideas to reduce cruft in gecko) while ensuring that there are no regression be better ?
I personally tend to think that rewrites (as a XUL on webkit would be) are often far less useful than dreamt originally; often a lot of the cruft accumulated has a reason to be there, and starting from scratch means that all this knowledge is lost. Of course, some bad decisions (e.g. over XPCOM-ized codebase) may make a rewrite more tempting, but trying to rewrite incrementally to excise bad code is to my mind really worth it.
One problem that I *feel* (this isn't necessarily true) with Gecko development is some sort of conservatism or reluctance to do big changes. Sure, lots have happened during 1.9, but even more could have happened. I should have merged nsEvent and nsDOMEvent, other things should have happened too, but didn't. It is not clear to me what is preventing developers to do more radical changes. Mozilla2 will hopefully help here.
For the future, even near future, we may have to think something quite different, something for which Gecko or Webkit aren't really capable. Working well in multicore environments (current desktop, near future mobile, for example ARM Cortex-A9), supporting animations and 3D, enhancements to user interaction via multimodality and sensors, etc. When keeping this all in mind, I see *no* reasons to switch rendering engine; Safari shouldn't start to use Gecko (although Safari 3 wouldn't perhaps be so damn slow in Windows :p), nor should Firefox start to use Webkit. Hopefully not too far in the future there will be probably something different. Interesting question is, IMO, that who is going to be the first one make the radical change (or did Microsoft already do it, in one way or another? XAML&co?). I hope Mozilla (as a project or Foundation) could be the leader, or among the leaders, in the change, also to ensure that web still stays open under the pressure of corporations. Should the organizations behind the current open source web engines unite and create something new, while maintaining the current engines?
You're right, Gecko can be improved, it is being improved, and we all understand 100% that any switch would be very risky. I mentioned that. That's why we're not doing it.
smaug:
> What I haven't seen is a clear list of features
> which are better (or which people think are
> better) in Webkit than in Gecko and vice versa.
I mentioned two areas: "easier to hack" (which I've heard from multiple independent sources who've worked on both, so I believe) and "significantly smaller code footprint for comparable feature set", which has been directly measured, again by multiple independent sources.
We have done big changes in 1.9. We will do more after 1.9. It's expensive, though, because of all the regression-fixing that we inevitably have to do, and all the work we have to do to make sure that the clean and shiny new parts of Gecko work correctly with the old and dirty bits (which inevitably makes the new parts a bit less clean and shiny than they could be). As you know, it's all about development resources. I do hope Mercurial will speed things up a bit, and Taras' tools will speed things up a bit --- we're definitely investing in the right things there --- but I don't expect magic.
> For the future, even near future, we may have
> to think something quite different, something
> for which Gecko or Webkit aren't really capable.
I agree, this is what I mean when I say Webkit and Gecko are not the epitome of browser engines.
Creating a new engine is certainly tempting. I guess I should blog about that too.
To me, this is mind-boggling. XUL sucks. It's still full of bugs and limitations, especially the instant you try to do something Firefox doesn't already do. Apps such as Joost and Songbird (not that anyone cares about Songbird, to be blunt) could be built with fewer UI bugs, faster, by having Gecko be an easily-embeddable component one can wrap in native UI. Most Firefox extensions could be built as a combination of Greasemonkey and some simple, limited ways to access parts of the browser chrome; but really, in my view, whether the extensions that cared about XUL could be moved over is unimportant, since the vast majority of Firefox users don't even care about extensions, and I view making web rendering better as a higher goal than providing people extensions anyway. (And I think I made it clear on Matt's blog why I think that, over time, a world in which the Mozilla folks are doing something WebKit based is the one that advances web rendering better).
When doing work on the Firefox UI just last year I was struck by how many times XUL was a hindrance, not a help, in getting things done. The Quick Find box looked retarded since it couldn't be made to look like a tiny box, and instead had to be a huge bar. The Safe Browsing folks struggled endlessly trying to get a UI that worked in XUL. Theme bugs in Firefox 2 abounded and still are not all fixed despite heroic engineering efforts -- heck, two people spent a couple weeks of engineering time trying to make the Go button work properly and still couldn't get the UI done in time, so we had to drop detachability from Firefox 2 and put in a crowbar pref to turn it off instead.
I guess I feel like Mozilla cares more about Mozilla-the-platform, which always was a stupid concept and remains so, rather than primarily about making the web better. And they're believing their own PR about XUL.
None of this is intended to say that, today, WebKit is a unilaterally better engine than Gecko, or that I don't agree that taking several years to speculatively switch engines is perhaps questionable, or any of the other _good_ reasons given to oppose such a move. I only feel that having XUL support be a primary reason to oppose such a move is flabbergastingly wrong-headed.
What does this mean?
Cheers,
Jeff
Web developers and users pay attention to Firefox. We can exploit this to make the Web better. For example, the closer we adhere to standards, the more Web developers build standards-compliant sites. As another example, when we ship new standards-based features like SVG, there's more chance Web developers will use them than if, say, only Opera is shipping them.
Peter:
I think it's a mischaracterization to say that the rest is "fairly peripheral", but since you've focused on XUL, so will I, here...
> Apps such as Joost and Songbird (not that
> anyone cares about Songbird, to be blunt) could
> be built with fewer UI bugs, faster, by having
> Gecko be an easily-embeddable component one can
> wrap in native UI.
It already is. Sure, the embedding story could be improved, but many apps do it including serious apps such as Epiphany, Eclipse, and Nokia's browser. And yet people are still choosing to build XUL apps.
> the vast majority of Firefox users don't even
> care about extensions,
But Web developers and power users do care about extensions, and those groups are more important per capita than the average user.
> and I view making web
> rendering better as a higher goal than
> providing people extensions anyway.
So do I, but good easily accessible Web development tools are important for the Web.
> I guess I feel like Mozilla cares more about
> Mozilla-the-platform, which always was a stupid
> concept and remains so, rather than primarily
> about making the web better. And they're
> believing their own PR about XUL.
No. We've been surprised and shocked at how many people are using it despite its deficiencies. Really, no one is or has been pushing XUL or Mozilla-the-platform; reality has outpaced our (non-existent) PR. (I'll note that lately Mozilla has even been offically downplaying support for "Mozilla-the-platform", to howls of protest and the setting up of Mozpad.)
> I only feel that having XUL support be a
> primary reason to oppose such a move is
> flabbergastingly wrong-headed.
Ditching XUL would mean casting a surprisingly large developer community aside (google for "XUL dark matter"), burning up all their goodwill and contributions, and damaging the credibility of open source platforms in general. *Then* Mozilla gets to build three new native Firefox UIs from scratch and reimplement any extensions deemed important. We also lose the benefit of innovations that have been happening around extensions (e.g., Glubble).
A desire to avoid all this is not "flabbergastingly wrong-headed", in my opinion. Especially because we can hope that by migrating pieces of XUL to the Web proper (as I discussed), in the long term XUL need not be an obstacle to transition.
Yes; but, as I said, I think many of those extensions could be done without XUL. And I think power users _and_ normal users alike care about things like rendering speed.
Of course in an ideal world it would be nice to have all the current extensions for Firefox work with some "better" engine (define "better" however you wish). I'm merely saying that when one must make tradeoffs, I'd happily trade off extensions for a better rendering engine, assuming the engine was compelling enough. (Is WebKit compelling enough? That's a much more arguable question, but it's not really what I'm debating.)
I don't view the PR for XUL as "nonexistent" -- it was a pretty widely talked-up point several years ago, which is when some of today's XUL-using projects were conceptualized. But you're right that there are projects starting *now* that are being built on it -- and I question whether the teams in question have done their due diligence.
By way of analogy, I used to work at Green Hills Software, a maker of embedded dev tools. In a world where there were multiple better choices, I saw many people build their embedded apps on embedded Linux, for the mere reason that they'd heard of it. XUL has similar buzz to embedded Linux -- other apps are using it, you should too! -- and, in my mind, a similar level of maybe-ok-but-not-very-good quality compared to other ways to build an app.
You can't stop people from doing whatever they want (and I certainly wouldn't suggest trying to _prevent_ people from using XUL), but it doesn't mean you have to let them control you, especially if their decisions are mistakes (as I think they are). Is Mozilla's mission to serve "a surprisingly large development community", or to serve the average end user of the web? If for the sake of argument we were to agree that that was the tradeoff, I think the latter ought to win.
Also, why would this "damage the credibility of open source platforms in general"? You would neither be somehow removing XUL et. al's source code from the public (not that you could), nor switching to a closed-source solution. I view this as a _strength_ of open source -- Mozilla can make a decision about how it can do the most good for its end users (which ought to be its focus in my view), while still allowing others to use and improve the platform they've selected.
Making it possible to build better UIs on top of web technology (by extending HTML with more XUL-esque capabilities, or whatever) is IMO a good thing, but a separate issue. While as engineers we're always fascinated by platforms, generalized solutions, and the Next Big Thing, our users are stuck with a crappy web _now_. We should fix it. Whatever the fastest path in that direction is, we should take it.
> done their due diligence.
I don't think we have the right to question that.
Given the amount of effort and money that Microsoft, Adobe, Trolltech and others spend pushing their platforms, I don't see people choosing us by default.
By the way, if you were a small outfit without the resources to do a full custom UI for the big three platforms, what would you choose as a cross-platform UI solution?
> Is Mozilla's mission to serve "a surprisingly
> large development community", or to serve the
> average end user of the web? If for the sake of
> argument we were to agree that that was the
> tradeoff, I think the latter ought to win.
I agree. But this is not a binary choice.
> You would neither be somehow removing XUL et.
> al's source code from the public (not that you
> could), nor switching to a closed-source
> solution.
"here's the source, it's the only support you need" may be true and all that was promised, but I still forsee dashed expectations and anger.
> Whatever the fastest path in that direction is,
> we should take it.
Writing three new native Firefox UIs from scratch is not the fastest path, for us.
I think we agree about premises. It's a judgement call in the face of uncertainty, and reasonable people can disagree about what the right call is.
Just would like to point out that "single dominant" == "monopoly"
Mozilla / Firefox already started acting like a monopoly itself, throwing it's weight around politically in the arena. Imagine if it actually were dominant? Mmmm nice mob-rule situation. Not much better than the alternative Mozilla purports to fight. ;)
Personally I would have two options for building GUIs: XUL and wxWidgets. There's also Qt that I have read good reviews about, but the look and feel to me always looks a bit strange. There's also Gtk, but again the cross-platform look and feel isn't so great, and it has been said it comes with a lot of baggage. Are there other contenders for cross-platform desktop UIs? Or do you think the UI should be done using platform native frameworks, in other words building a new UI for each platform?
I used XUL in the very early stages of its life, and didn't really have any problems with it. Things just worked consistently between platforms. Granted, I didn't do anything very complicated or non-standard UI stuff. How is XUL's cross-platform story nowadays, how much platform-specific workarounds do people need to make in their apps? I can see I could be in trouble if I wanted to do something unusual, though.
wxWidgets is good with platform native look and feel. If you have experience with MFC, wxWidgets feels very similar. It also has bindings to Python, my favorite language at the moment. However, just about every time I do something with wxWidgets I find that I need to make little tweaks for each of the platforms I am supporting (like some control is too big by default on one platform, doesn't paint automatically on another platform, ...) It does give you full power to tweak it as much as you like, so you could make a non-rectangulat text entry field, for example, without wxWidgets getting in your way I'd think.
What about HTML? It's by far the most popular language for writing multiplatform GUIs.