More About Amber

I have uploaded a document describing Amber in some detail. It focuses on the back end: the instrumentation, indexing, and compression. I also motivate Amber by discussing the debugging problem and related attempts to solve it. There are a few important ideas here:

• A conceptual framework for classifying debuggers and execution recorders.
  
• Treating instruction execution and memory writes as two instances of a common concept --- "memory effects" --- and providing a single implementation and API for recording, indexing, compressing, storing and querying them.
  
• The "bunching" optimization for memory effects.
  
• The principle that repetitious program behaviour should produce repetitious input to the compressor in order to maximise the benefits of compression, and how to tweak the formatting of trace data to follow that principle.
  

The document also contains screenshots of the prototype XULRunner-based debugger based on Amber, to motivate Amber's design by illustrating what is possible when you go beyond just reverse execution. I assure you that those screenshots are not faked in any way; it really works :-).

A New Approach To Debugging

One of the painful truths about Linux development is that debugging sucks. Specifically, gdb on Linux sucks. Basic functionality simply does not work reliably. The details of the brokenness vary depending on the version, but the general suckiness seems to hold up across versions. Since it fails on the basics, we need not even discuss the lack of modern features or its appalling usability. This is a big problem for Linux because it is driving developers away from the platform.

Here's a deeper and less widely understood truth: all debuggers suck. Debugging is mainly about tracing bug symptoms back to buggy code, i.e. tracing effects to causes. Effects follow their causes in time, so debugging by nature requires backwards traversal of time. But traditional debuggers don't provide this because it's not obvious how to implement it. Instead they let the programmer monitor a program's forward execution, in the hope that the programmer will be able to somehow remember or reconstruct enough history, usually by running the program several times, to figure out for themselves the backwards cause and effect chain. Such tools aren't particularly suited to the debugging task; they're just a compromise forced by either limited imagination or implementation constraints.

Lots of other people have made this observation and have tried to extend traditional debuggers with some support for looking backwards in time. For example:

• SoftICE supports a "back trace buffer" that records a limited number of instruction executions in a certain instruction address range. Only the program counter values are recorded.
  
• Green Hills' "Time Machine" debugger also has a trace buffer, which apparently can include register and memory deltas so that old memory and register values can be restored by "backwards stepping" through the buffer. Unfortunately it seems this buffer is limited to memory, and therefore for a large application only a small history window can be retained. Furthermore Green Hills' implementation requires specialized hardware if you want to trace more than program counter values.
  
• TTVM logs the interactions between a guest virtual machine and its host so that later the exact execution of the guest VM can be replayed. Periodic checkpoints are recorded so that the replay cost required to reach any desired point in time is limited.
  

In Time Machine and TTVM, the UI is a traditional debugger UI extended with "reverse" versions of the standard forward execution commands. The debugger still displays the program state at a certain point in time; there is now some capability to move the current point backwards as well as forwards.

This is good progress --- but those tools still need to overcome implementation limitations to become immediately useful for mainstream desktop application debugging (i.e. useful to me!). More importantly, I want more than just reverse execution. For example, when debugging we frequently want to know exactly when a variable was set to its current (incorrect) value, and see what the program was doing at that time. Reverse-execution debuggers can set a watchpoint on the variable and execute backwards to find that write, but it would be quicker and more powerful if the debugger just knew the history of the variable and could perform an efficient database lookup to find the last time the variable was written before the current time. The debugger should have the entire history of the program available for efficient inspection and querying. With such a debugger, after we've gathered the history we don't need to run the program any more; the user just browses the history in the debugger until the bug is located.

The only other exposition of this vision I know of is Omnisicient Debugging. Bil Lewis has created a Java debugger that takes this approach. Unfortunately it is restricted to Java programs, and apparently rather small and short-running Java programs since it keeps the trace data in memory. So the big question is: can we implement this vision so that it's a practical approach for debugging real applications on commodity hardware?

I have spent a few of the last 18 months investigating this question. I have built a system, which I'm calling Amber, to record the complete execution history of arbitrary Linux processes. The history is recorded using binary instrumentation based on Valgrind. The history is indexed to support efficient queries that debuggers need, and then compressed and written to disk in a format optimized for later query and retrieval. The history supports efficient reconstruction of the contents of any memory location or register at any point in time. It also supports efficient answers to "when was the last write to location X before time T", "when was location P executed between times T1 and T2", and other kinds of queries. I can record the 4.1 billion instructions of a Firefox debug build starting up, displaying a Web page, and exiting; the compressed, indexed trace is about 0.83 bytes per instruction executed. (With some cleverness, we achieve compression ratios of over 20 to 1.) It takes less than half an hour to record on my laptop. 300X slowdown may sound like a lot, but spending computer time to save programmer time is almost always a good idea. Furthermore Amber tracing easily parallelizes across multiple cores so hardware trends are in my favour.

I've built a trace query engine in C and a prototype debugger front end using XULRunner to demonstrate the power of this approach. It works as expected. I've found that basing the debugger entirely on a "complete recording" approach dramatically simplifies some of the difficult parts of debugger implementation. Novell has filed for a patent on my work (and is planning to file another regarding an extension which I haven't described here yet), but I hope to get Novell to release the source code under an open source license so I can keep working on it. Maybe others will be interested in working on it too.

When I get a chance I'll write more about how Amber works, its full set of features, and what needs to be done to get it to prime-time.

Pohutukawa

Someone on IRC asked about the pohutukawas flowering. The pohutukawa is a native tree that produces beautiful red flowers in December --- very apt for Christmas. The Internet has thousands of photos of flowering pohutukawas; here's mine.

Update I forgot to mention --- this photo was taken in Shakespear Regional Park, yesterday (i.e. Saturday). We drove up with some friends, had a picnic, and did the Tiri Tiri Track loop in about two hours. It was magnificent.

Parallelism

Here's an interesting interview with Hennessey and Patterson, the famous hardware architecture researchers. Nothing particularly surprising or new, but they remind me of the importance of the parallelism revolution that is happening right now. Namely, processors aren't getting much faster anymore; instead, we're all getting multicore chips instead. Software that doesn't take advantage of this parallelism will be slower than software that does.

This is a huge huge problem because after fifty years of research we still don't have good ways to write parallel software. Threads and locks are very difficult to use correctly, don't work when you combine software components, and don't even scale well in performance. Language-level transactions are one possible answer but we don't have language and runtime support for them in mainstream systems. (At IBM I worked on the design of X10, a programming language designed for highly concurrent and distributed programming that used transactions as one of its building blocks.)

Even if we get language-level transactions, we still have several more problems to deal with. Reworking large legacy codebases to use them will be tons of work, and often impractical. Then there's the problem that many tasks are intrinsically very difficult or impossible to parallelize. In particular many higher-level programming models (such as oh say Javascript on the Web) assume single-threadedness.

For Mozilla, we need to be thinking about how we can exploit multiple cores in the browser to transparently accelerate existing Web content. We also need to be thinking about how we can add transaction support to Javascript to enable Web developers to exploit parallelism --- they don't ask for this today, but it will take a few years to do it right and by then, they'll probably be begging for it.

For open source in general, we need to be thinking about how the open source stack is going to evolve to incorporate more parallelism. If we can't get useful transaction support into C or C++ then two things will happen: first, most performance-sensitive projects will soon have to move away from them except for projects that can afford to invest in hairy threads-and-locks parallelism. Then, when chips with dozens of cores become popular, even those projects will have to migrate. I'm talking mostly about clients here, not servers; servers are easier to parallelize because one server usually serves many independent clients, so you can parallelize over the clients --- and of course people have been doing that for a long time.

I think there's a huge opportunity here for a client software mass extinction followed by Cambrian explosion. There's a new environmental pressure; projects that aren't agile enough to adapt quickly will be replaced by more agile projects, or by entirely new projects, perhaps ones that rewrite from scratch to take advantage of new languages. Exciting times.

Changes Afoot

I am leaving Novell at the end of the year. Starting from the beginning of January I will be working as a contractor for the Mozilla Corporation.

Novell has been very good to me, and I've very much enjoyed the last two years, but the lure of MoCo has proven too great. In particular Mozilla has offered to support additional developers here in Auckland. When I came back here, I had a vague, long-term goal of helping develop New Zealand's software industry by creating the sort of jobs that would have lured me back. I'm very excited to be realizing this goal much sooner than I expected, and I'm grateful to the corporation for making it possible. I'm also greatly looking forward to working in an office with other real live Gecko developers. I will be able to say more about this soon. Also, staff at Auckland University have shown interest in having students work on Firefox-related projects, and that's something else I'm very excited about. It just so happens I have the ideal qualifications to make that work. Perhaps God does have a plan :-).

A secondary issue is that over the last two years I've also felt I wasn't serving Novell's business needs very well. I spent most of my time working on core Gecko, some time working on Novell-specific needs, and some time working on other things. They never indicated they were dissatisfied with me, but I think they might be better off with someone more focused on Novell-specific issues. As it is, I'll continue to do 80% of what I was doing for them and they won't have to pay for any of it :-).

Just for the record, my move has nothing to do with Novell's deal with Microsoft or any of that stuff. My plans were well underway before I heard about it.

I've been busy in the last few weeks tidying up work at Novell and organizing my new position. Meanwhile it's a lovely time of year to be in Auckland; the pohutukawas are flowering and most people are winding down for the end of the year, assisted by an extraordinary number of Christmas "do's" (i.e., parties). Novell's Auckland staff had a wonderful lunch at Mudbrick bar and restaurant on Waiheke Island. I haven't really been able to wind down myself yet, but now that my Mozilla contract is signed, I can relax a bit ... but only for a couple of weeks. Yet it's good stress; the coming year looks very exciting indeed.

The Ultimate Square Peg

I'm annoyed by stereotypes --- and naturally, particularly so when I'm in the group being stereotyped. So I was pleased to read about a new US reality show "One Punk Under God" about Christian minister Jay Bakker. That's cool.

I also dislike the way that Christians, especially (but not only) in the USA, have become identified with some key causes that all seem to revolve around making other people behave in certain ways. This has unfortunately led to people believing that that's what Christianity is all about --- when actually one of the most important truths is that just doing things is not the way to make things right between us and God.

So, I thought I'd really like this article by Jay Bakker, which leads with:

What the hell happened? Where did we go wrong? How was Christianity co-opted by a political party? Why are Christians supporting laws that force others to live by their standards?

In fact, there's little in there that I would specifically disagree with. But there's a problem with it as a whole: it implies that Jesus was about nothing but unconditional love. That is an error in another direction, and it's also a common one. His parables and lessons do talk a lot about love and forgiveness, but they also talk a lot about hell, judgement, and "I have come not to bring peace, but a sword". To paraphrase a common Christian saying, he does have a message of "come as you are, not as you should be" --- but he doesn't want you to stay that way.

The problem with Jesus is that he doesn't fit into boxes. He doesn't seem to countenance the imposition of a morality state, but he also won't be coerced into being a comfortable anything-goes Saviour. He never conforms to the image we demand of him. That's uncomfortable and disconcerting ... but it's one of the things I love about him.

Patent Sea Change

My brother alerted me to some very interesting news that I haven't seen widely reported yet.

Guess who filed the most US patent applications in November? IBM? Microsoft? Guess again. The answer is Samsung --- and by a mile.

It was IBM 347, Microsoft 360, Samsung 523. And this isn't a one-off spike.

I'm not sure what's happening here. 6K patents per year means tens of millions of dollars in filing fees alone, so it's obviously a huge investment with some kind of strategy behind it. I don't believe Samsung has suddenly built a network of research labs to rival IBM and Microsoft. It seems likely that they've just decided to become very aggressive about patenting absolutely every idea their people can think of. Over the years a lot of people have warned about what could happen if someone tried to exploit the weaknesses of the US patent system to the maximum (and the patent industry has pushed back with "don't worry, that's not happening"). It may well be happening now.

What next? Things could be very interesting. We may see US companies ramping up their patent-farming efforts to match. But I think we will also see good old-fashioned nationalism (perhaps with a racist flavour) coming to bear on the question of patent reform. Will Americans stand for the transfer of public ideas into private monopolies when it's foreigners locking things up? But if the US government starts to backpedal, and possibly even breach its WIPO commitments, then that gives cover to other countries to implement sensibly restrictive patent laws. That would be a good result.

Congratulations David!

... on landing the reflow branch. This is a great day.

Now we finally pass acid2, we've improved performance, and fixed a lot of bugs. Yay!

Also, we can now make major reflow changes again without fear of making David's life miserable with difficult merges to the reflow branch.

Update Just to clarify --- the reflow branch landed after we branched for 1.9 alpha 1 (this was intentional!), so 1.9 alpha 1 does not have the reflow branch and does not pass acid2. You'll have to get a nightly build or wait for 1.9 alpha 2 to come out.

XUL Dark Matter

I had lunch with some of my old friends yesterday, as I often do (one of the great things about being back in Auckland!) and one of them told me about his latest consulting job, involving a project using XUL to build a very large intranet application.

This isn't the first time I've heard about such things, but this is definitely closer to home than before. If a random company in Auckland is working on a big XUL intranet app, then how many other such projects might there be? One might guess hundreds or even thousands. This is rather disconcerting for several reasons. These projects have no visibility in Bugzilla --- why aren't they filing bugs? We don't really know anything about these projects: how many there are, what features they use, how we're hurting or helping them. Gecko development is totally ignorant of their needs. But how can we find out since these intranet apps by design never become visible to the public?

Maybe it's a good thing in some ways to not be giving much attention to these apps. It means we can focus more on Firefox which is more in line with Mozilla's mission of driving the Web forward. And we certainly haven't promised much to XUL app developers. Still, it sucks to cause pain for people, and if there are thousands of XUL apps below the waterline and we could support them better in simple ways, I want to.

For one thing, our Q&A for XUL is basically "do Firefox and Thunderbird still work?" This sounds inadequate now that it seems likely that Firefox and Thunderbird are less than 1% of the XUL apps out there. If we had a lot of unit tests for XUL that would give us some more confidence that we're not breaking things ... or at least that if we are breaking things, we do so deliberately.

The other thing we should do is think about how to harness the power of this community that probably is quite large and influential ... even if we can barely detect it.

Linux

Mike Connor has blogged about the Linux policy discussion we had during the summit and the conclusions we reached. I don't have much to add. Because Mozilla release schedules and distribution release schedules often aren't aligned, distributions are often adding features and non-security fixes to their branch builds. It will be very helpful to be able to share those changes between distributions via Mozilla CVS.

We are going to have to deal with situations where one set of distributions wants only security fixes on a branch and other distributions want to ship features off that branch. I'm not quite sure how that's going to work yet.

NetworkManager, dbus and Offline Status

I've finally checked in support for offline status detection on Linux. We hook into dbus to listen for signals from NetworkManager saying that it has gone offline or gone online. I originally did this work about a year ago but it's taken ... a while ... to get reviewed. After I wrote the infrastructure the Windows support actually got written and landed while I was waiting for review on the Linux bits, so Windows has had this on trunk for a while and it even landed in FF2.

Anyway, now if your network link goes down, FF will automatically switch to offline mode and if the link comes back, FF will switch to online mode. If you manually change the "Work Offline" state then FF will stop switching automatically for the duration of your FF session.

Other Xulrunner apps can take advantage of this too. By default the IOService will watch for network link status changes and automatically toggle its online/offline state. But if an application wants to do things differently, it can tell the IOService to stop the automatic tracking. Then the application can listen for network link status changes via nsIObserver and modify IOService's online/offline status as desired.

We still want Mac support. Also, if you happen to be using a Linux distribution without dbus or without NetworkManager, the build will still work, you just won't get the automation.

Firefox Summit

I feel obligated to say something about last week's Firefox Summit, because it was a big event for our community and for me. For me the most important thing was to meet lots of contributors whom I'd never met face to face before, but whom I've worked with for a long time ... you know who you are. Even the people I had met before, I haven't actually spent much time with, so it was good to meet them again.

One thing I noticed was how huge the community is now. It's something you don't really grasp when you're just working through bugzilla and IRC. A lot of that is due to hiring by the Mozilla Corporation, but we also continue to acquire good old-fashioned volunteer contributors, and also the ecosystem around Firefox and Gecko is growing, supporting other applications and other companies. It's really exciting.

I ran sessions on Text and Typography and Offline Web Applications. The former basically was my summary of the status of the new text code, to bring people up to speed. Some issues were raised that I'll have to look into, like MathML's need to specify glyph indices directly in a string. Simon Montagu asked some difficult questions about ligatures that stack vertically instead of horizontally and how we can support those. We just don't know; someone needs to find an Arabic text editor that supports vertical ligatures and tell us how it works. Another fun problem that was raised: how should selection be rendered when you select part of a string in a font like Zapfino? The "advance width + ascent + descent" rectangle actually doesn't cover much of the characters selected...

The offline applications session had more discussion. I think we've converged on a plan and now we just need someone to implement it! The guts are in the wiki page; we need a way for pages to specify which components should made available offline, and then download and cache those offline pages as the user indicates, probably just by bookmarking the site.

Overall I had a great time. The scavenger hunt was a little boring --- I guess downtown San Jose isn't the best venue. But the other events were fantastic. Hats off to the Mozilla organization for running it and funding it.

Prophecy Fulfilled

Steven Friedlander, head of distribution for Warner Independent Pictures, which had one of the summer's few hits with March of the Penguins, suggested that it was unlikely that Hollywood will learn from the mistakes it made this past season. "In an ideal world, people would say 'OK, we have to think more creatively, we have to think outside the box and come up with new and different things, '" Friedlander told AP. "But I'm afraid what's going to happen is, we're all going to sit in a room and say 'We need more penguin movies.'"

5 September, 2005

The Warner Bros. animated penguin romp "Happy Feet" debuted with $42.3 million, grabbing an edge for the weekend's No. 1 slot.

19 November, 2006

Mail Forwarding

For years I've been using Zoneedit's free service to forward mail for accounts @ocallahan.org to various destinations (currently GMail accounts). Unfortunately Zoneedit's service seems to have deteriorated lately; many mails are delayed by half an hour or more inside Zoneedit's system, and I suspect one or more may have been dropped, although I can't prove that yet. Any readers have tips for a more reliable but free service to achieve the same thing? "Google Apps For Your Domain" is close, but apparently overkill for what I want since it seems to give you fresh GMail mailboxes for the domain email addresses, and I just want forwarding to existing accounts.

The Firefox Summit was fantastic. I met literally dozens of people I've worked with for years but never met face to face, and I had lots of fun. I'll probably write more about it soon.

Update! On the advice of an anonymous commenter, I tried using Google Apps For Your Domain, setting up one "email list" per name@ocallahan.org, each containing one recipient, the address I want to forward to. It works great! Email propagation time is reduced from 10-30+ minutes to about ten seconds, and I only have had to create one extra GMail account (root@ocallahan.org, the administrator). Thanks Anonymous (and Google)!

Making Web Applications First Class Desktop Citizens

I was reading Joshua Schachter's blog and noticed he's commented about something I've been thinking about: we need to make Web apps fit better in the world of desktop applications:

Since web applications aren't first-class desktop citizens they therefore are missing several interfaces that normal applications have: you cannot Alt-Tab to them, you cannot launch them separately, they don't "install" and stick around on your desktop, and notably, they cannot control what happens when you close them.

Not "installing" and "sticking around on your desktop" are actually nice features of Web applications, in most cases, but we could certainly improve the situation here, perhaps by making it easier to create desktop links, and by treating launches from desktop links as instantiating a new application instead of just another browser window or tab.

Another thing that Web apps should be able to do is register to handle certain file types.

Miscellany

Jesse Ruderman pointed to a slideshow by Peter Guttman, longtime security guru at the Auckland University CS department (not far from my office). It's nominally about phishing, but really it's about psychology and user engineering for security. It has the most pointed attack I've ever seen on "security warning" dialog boxes --- the "ARE YOU SURE YOU WANT TO GET YOUR WORK DONE? [OK] [Cancel]" boxes that you, I and everyone else blindly click through every day. It's a must-read for just about everyone who works on software or Web sites. I found it interesting, disturbing, thought-provoking and enraging (the latter especially regarding the near-malicious design of US banking websites).

I am nominally on holiday this week. Actually I'm going to be rather busy catching up on work at home and also a few things at the office. However I do reserve the right to ignore anything unpleasant! The highlight of my week will probably be my talk at the Auckland University Computer Science department on Wednesday at noon. Thanks to the CS department for hosting me.

Partial Ligatures

One of the features of the new text code is the ability to lay out a string of content and then render any substring using the whole-string layout. This is important because, for example, the user should be able to select any substring, and we need to draw that substring using the current selection colors. But it's hard because fancy fonts want to use ligatures, i.e., render certain combinations of adjacent characters using a single specially designed glyph.

The screenshot shows some text in DejaVu Sans using my text code and Pango 1.14. "fl" and "fi" are being rendered as ligatures. I'm selecting inside "fl" to show how that works. Basically it's a hack --- a ligature with N characters is divided into N vertical strips of equal width which I pretend are "pseudocharacters", and then I use clipping to draw the individual pseudocharacters. This seems to work OK.

The screenshot also shows another feature of the new text code: our text layout objects (gfxTextRuns) can span DOM node boundaries. In this example the textrun spans the boundary of an <a> element. This enables us to form ligatures across those boundaries but thanks to partial ligature support, we can still style the link element correctly. Not just ligatures but also kerning and shaping benefit from cross-node text layout.

Note: When I said above that any substring should be selectable, that was a lie. Multiple Unicode characters can combine to form "clusters", for example a character with an accent, and selection should always be restricted to cluster boundaries. More consistent support for honouring cluster boundaries is another feature of the new text code. However, I currently don't support clusters spanning DOM node boundaries, because we really have no way to render those, not even a hack.

By the way, I noticed the hairline crack in the fi ligature. It's probably a rounding error that I need to track down.

Thinking About Performance

I'm amused to read Scoble and others (via Asa) complaining about IE being slower than Firefox on tests they care about. All I ever hear about is testcases where Gecko is slower than other browsers. This is not surprising, when you think about it, but it can be discouraging, so thanks to Scoble and others for reminding me it's not a one-way street.

This also highlights an interesting network effect. When a browser has high market share --- especially among Web developers --- developers will optimize for that browser, working around areas where it's slow and taking advantage of features which are fast. Browsers that are trying to break into the market don't get that benefit so they have to try to be faster in all areas where the dominant browser(s) are fast. This is a considerable burden. Hats off to Opera and Safari for doing such a great job.

Of course, even if developers are working around a performance problem in Firefox, we still need to address the problem ... not just to keep up with the competition, but to make life easier for Web developers. And sometimes our performance problems just stop people from using a feature of the Web platform, and we definitely need to avoid that.

Motutapu Island

On Saturday we went out on my parents' boat for a day trip to Motutapu Island. The boat always brings back many precious memories to me, of spending a decent chunk of each summer holiday on it cruising around the islands of the Hauraki Gulf when I was growing up. The Gulf is one of the best things about Auckland. You can reach many wonderful places by ferry --- Rangitoto, Waiheke, Motuihe, Tiri Tiri Matangi, Kawau, Great Barrier --- but to make the most of it you definitely still need a boat of your own.

Motutapu is adjacent to Rangitoto. The two islands are very different --- Rangitoto a brand-new volcano, barren and wild, Motutapo old sedimentary rock, grassy and operated as a farm, lovely in its own way. The Department of Conservation is gradually restoring Motutapu's forest cover, which is excellent although a very long term project.

Mysterious Cargo

At last, I have obtained concrete evidence of New Zealand's clandestine nuclear missile program!

More Text

Following up on yesterday's issue with line-breaking and kerning... Computing all line break opportunities up front and doing something special when lines could break between non-whitespace characters was going to be too difficult. It's much easier to compute line break opportunities during reflow. Also, I want to have the code structured so that we can have special shaping for glyphs at line ends, and that would apply even when a line breaks at whitespace (or doesn't break at all), which would mean we would want to recompute the text layout at unpredictable times.

So I've opted to provide a hook back to layout from gfxTextRun, allowing it to recover the text used to originally build the textrun (which we can do by recrawling the DOM elements contributing to the textrun). This way textruns can forget the input text and just cache glyphs, and in the uncommon cases where they need the original text again, they can ask for it.

In a few cases (:first-letter and :first-line, I'm looking at you!) it's hard to recompute the text given the information we have stored. For those rare cases I can force the textrun to remember the text at creation time, so I don't have to write complex code to do the reconstruction.

Another round of the delicate dance of tradeoffs completed... for now.

Insufficiently Paranoid

You know those people who see the shadowy hand of Microsoft behind every attack on Linux? The people whose rampant paranoia gives those of us with legitimate concerns a bad name? Well, sometimes they're right.

6. Sometime in 2003, I was approached by Richard Emerson (Microsoft's senior vice president for corporate development and stratedy) about investing in SCO, a company about which I knew little or nothing at the time. Mr. Emerson stated that Microsoft wished to promote SCO and its pending lawsuit about IBM and the Linux operating system. But Microsoft did not want to be seen as attacking IBM or Linux. For that reason, Microsoft wanted to further its interest through independent investors like BayStar.

7. I did some research on SCO, and had conversations with Mr. Emerson about it as well. In the course of my research about SCO, I became concerned that SCO might be merely a litigation company. As a result, Mr. Emerson and I discussed a variety of investment structures wherein Microsoft would "backstop," or guarantee in some way, BayStar's investment. In addition, I had discussions with Kenneth Lustig, Microsoft's managing director of intellectual property and Tivanka Ellawala, from Microsoft's corporate development department regarding the SCO deal. As part of these discussions, Microsoft assured me that it would in some way guarantee BayStar's investment in SCO. However, Microsoft would not agree to put anything in writing on this point.

I'm pretty paranoid myself, so it's disturbing to learn that I'm actually not paranoid enough.

Text Progress

Among other things, I've been working away on the new textframe code. I've (re)written the layout part around the new gfxTextRun abstraction and things fit together pretty much as I expected except for a couple of issues.

One issue is that CSS :first-line and :first-letter styling doesn't map well onto the gfxTextRun concept. gfxTextRun assumes that the glyphs used for a chunk of text are largely independent of the formatting of the text into lines. I worked around this by aggressively recreating textruns when :first-line and :first-letter are in use. Ugly but effective and simple.

Another issue that came up was that we really want textruns to span multiple DOM text nodes, if those nodes are visually consecutive and use the same font. This allows chunks of shaped text to be styled differently ( e.g., as a link) without breaking shaping between chunks. Even for Latin text, it allows kerning and ligatures to work across styling boundaries. It also means that a set of text nodes adjacent in the DOM will render identically to a single text node, which is an important property to have. So I had to rework everything to allow a gfxTextRun to span DOM nodes. This added some complexity but I think it's worth it. It will make textrun setup slower, but it will make other things faster because we'll be able to create fewer and bigger textruns, which means fewer expensive calls down into Pango and other APIs.

The next step is for me to implement gfxPangoTextRun. This is requiring me to delve into Pango and figure out how to use its APIs. A key concern is to minimize the memory usage of live textruns. I think I have a scheme that will use four bytes per character in most common cases. We'll see how it goes. Once nice benefit of textrun objects is that those four bytes directly encode glyph indices and advance widths for the text, and we can also keep cairo_scaled_font_t pointers in the textrun, so painting text becomes very simple: very little more than a single pair cairo_set_scaled_font()/cairo_show_glyphs(). Repeated rendering of the same text should be about as fast as it possibly can be from our end ... any remaining problems can be blamed on cairo :-).

One remaining issue I haven't figured out yet is the problem of line endings changing the shape of characters in the textrun. For example, what happens if a font has a "tt" ligature and we hyphenate "cutting"? Also, since you can't kern across line breaks, the advance width of a character could be different if there's a line break after it. Furthermore, OpenType supports glyphs with different shapes when they occur at the beginnning or end of lines (although I don't think Pango does this yet). Currently gfxTextRuns are nearly immutable, which is very nice; the only visible mutable state they can have is cached spacing data. But I think we'll have to solve the line-ending problem by breaking that immutability and adding line ending state. Implementing this with Pango seems tricky though, for example we'll have to somehow un-ligate ligatures. I really don't want to have to recreate the entire layout, or keep enough data around in case we need to do so. For now I think I will just check at textrun creation time whether there are any break opportunities inside "problematic" text (say between two non-space characters) and only keep around the information to recreate the layout if there is at least one such opportunity.

There is a related issue: how do you draw text where the style changes in the middle of a ligature? E.g. a<a href="google.com">f</a>fluent where the font has a ligature for "ffl". I think we'll be doing what everybody else seems to do, a cheesy hack where you use clipping to cut the ffl glyph into three equal horizontal segments. (But you can't use this approach to avoid the un-ligating I mentioned. For example if you applied it to hyphenate "af-fluent", you're likely to have a pixel or two from the top of the first f appearing on the other side of the hyphen, or something equally disgusting.)

My Father's Days Of Piracy

The New Zealand Herald has story about my father, Denis O'Callahan. It's nearly the fortieth anniversary of the first transmission of Radio Hauraki, New Zealand's first and only pirate radio station, created by my father and a few other like-minded guys in their twenties. At the time the New Zealand Government didn't allow private radio stations, so the idea was to set up a transmitter on a ship in international waters to work around the law. It was a crazy, risky undertaking, full of nautical, engineering, legal and financial challenges, and it succeeded. (After a few years the government caved to popular demand and granted them a land license.) My dad was both the original radio engineer and boat skipper, and of course I'm immensely proud of him.

Here Comes A New Challenger!

A friend pointed me at a very interesting article about the New York Times Reader, which is based on the Windows Presentation Foundation. You should also read The Times' FAQ. It fulfills my expectations that I blogged about before: a classy application that takes eyeballs away from the Web and back to a world fully under Microsoft's control.

Obviously we need to make it possible to do the same things in a Web page. As far as I can tell there are three sets of issues: offline access and layout. The rest of Reader's features could be implemented using existing Web technology, as an AJAX application (although XUL would help).

I believe the offline access issues are actually easier to solve; we're shipping part of a solution in Firefox 2 (DOM Storage). The other part of a solution is the ability to fetch and pin pages in the browser cache so sites can ensure they'll be available offline. I have a proposal for that. It needs some work but doesn't seem to be a huge task to specify and implement.

The other issues are layout and rendering. Clearly there's a big emphasis on columns; fortunately Firefox does columns. Unfortunately our columns don't make it easy to create the sort of page-based UI I see in the screenshots. I think a possible approach would be to use XUL box layout to create the header/body/footer vertical arrangement (via CSS, no XUL elements), and make the body overflow and scroll to the right. Then modify the Gecko event code so that the pageup, pagedown, space, and arrow keys do the right thing (namely, scroll left by a page, scroll right by a page, scroll right by a page, and scroll by columns). This would require just a few code tweaks and no new style properties.

The page navigation bar in the footer could be easily implemented using script, onresize, and onscroll handlers, but I don't know if it's really useful.

The hardest issue that I can see in the Mississippi screenshot, where the image is on the right-hand side of the first page and spans columns. One way to achieve that here would be to allow column content to flow around floats outside the column flow. Then the article title and the image could be placed in left and right floats respectively and the column content flowing around them would produce the right effect. We could also add the ability for content in columns to flow around overflowing content from previous columns, which would let people put column-spanning objects in the column flow.

That still isn't enough to produce some common effects, such as figures at the top or bottom of a page. For those we will need CSS extensions. One possibility is to extend "float" with "column-top", "page-top", "column-bottom", and "page-bottom" values.

Another issue mentioned in the article is fonts and typography. I can't tell much from the screenshots but they don't appear to be using widow/orphan control, so it's probably about font rendering. We do desperately need a solution to font downloading. On the bright side, our current Gecko work will let us improve our story with kerning, ligatures and other font rendering issues.

Even if we can implement all these things, there's still the issue of how we get people to use them in their sites. Microsoft is very good at partnering with major companies to produce showcase applications like this NYT Reader. We need partnerships like that. You would think that certain major Web sites would be interested in pushing back...

Reading between the lines, though, there is a lot of hope for the Web. The application is a 20-minute download for XP users without WPF installed, and it apparently requires a trust decision from the user, so it's not for casual surfers. You will need to leave the Reader application to follow links to other sites or content that hasn't been reformatted for the Reader yet, and links to NYT stories from other sites probably won't open in Reader. The Web's hypherlinked nature give it a lot of gravity and it's hard to see that changing in the medium term. The NYT had to develop their own application with Microsoft's help and has to deal with its content being served out in a different way. If we can produce an experience comparable to Reader in the browser --- and especially if we can make it available as a set of easy, downward-compatible changes to existing sites --- I think we'll have an overall much more compelling user experience.

Status

I've been working my way through a backlog of stuff. Most noticeably, to some anyway, I've caught up on my reviews. I currently know of no outstanding review requests where the ball is still in my court. If I've missed something, let me know.

Lately I've been working on the new textframe code in bug 333659, that reorganizes text layout around the new gfxTextRun abstraction. I've written all the new layout parts, almost completely replacing nsTextTransform.cpp and nsTextFrame.cpp. There's still a lot of work to do to implement gfxTextRun on all platforms. I'm starting first with the Pango-based implementation, but that's stalled while I wait for masayuki's rewrite of our existing Pango code to land. The new code feels nice to me. It handles combining characters and clusters everywhere and seems a lot simpler than the old code. Hopefully it will perform better too, now that we can retain glyph conversion and placement between layout and painting. Only testing can tell.

The dependencies surrounding this code are quite complex at the moment. The new textframe depends on changes to inline layout that I have filed as a separate patch. But the inline layout changes actually break the computation of minimum width for text containing words of more than one element (e.g., <b>H</b>ello). Fixing that really needs the reflow branch. Turns out that the reflow branch is also broken in the same way and fixing it could use these inline layout changes! On top of all that, the new textframe code is really needed to help improve performance with cairo, so the complete cairo switchover needs this code. Sigh...

Timeless prodded me to work on scrolling performance, which has regressed on trunk a bit, especially for certain sites such as Gmail. We found a few issues and I have patches to fix those that I should check in tomorrow. They're all to do with "scroll analysis", how we determine when and where it is safe to scroll a window using "bitblit" (just copying pixels from one place to another on the screen). A full explanation deserves a blog entry of its own.

Next I think I'll shoot down a few random bugs. Maybe fix the file input control so that the textfield is operable again, but typing into it brings up the filechooser dialog. Soon I plan to dive into the ubrowser patches and try to get it running on Linux and ready to land in Mozilla CVS somewhere.

Static Analysis And Scary Headlines

A few days ago Slashdot trumpeted the headline "611 Defects, 71 Vulnerabilities Found In Firefox", based on a post by Adam Harrison who had applied his company's static code analysis tool to the Firefox code. That's not an unfair summary since Harrison's post says "The analysis resulted in 655 defects and 71 potential security vulnerabilities."

The problem is Klocwork, like most other static analysis tools, reports false positives; i.e., it reports problems that are not actually bugs in the code. (More precisely, it may identify error behaviours that actually cannot occur in any run of the program.) That itself is not a problem, but when reporting the results of these tools you must make clear how many error reports the tool produced and how many of those have been verified by humans as corresponding to actual bugs which would affect some run of the program. In this case, it was not clear at all. We're told "655 defects", and then in comments Harrison claims "In this particular analysis we reviewed the entire results to verify the correctness of the defects." But Mozilla developers have been combing through the Klocwork reports and it turns out that most of them are not real bugs.

Here's one example:

902         compNameInFile = (char *) malloc(sizeof(char) * (stbuf.st_size + 1));
...
908         memset(compNameInFile, 0 , (stbuf.st_size + 1));
...
911         rv = fread((void *) compNameInFile, sizeof(char),
912                    stbuf.st_size, dlMarkerFD);
...
923                 if (strcmp(currComp->GetArchive(), compNameInFile) == 0)

The Klocwork analyzer claims that on line 923, there can be a string overflow reading from compNameInFile because it might not be null terminated. But in fact it's clear that there will always be at least one zero in the buffer.

In fact, people have looked at lot of the Klocwork reports and so far 2-3 of them have been judged genuine bugs (be sure to read the comments in those bugs...).

I'm sympathetic to static analysis tools; I did my PhD in the area :-). I really want Klocwork, Coverity, Oink and the rest to be successful, to become a standard part of the software development toolchain. But we've got to be honest and avoid the scare headlines.

---

It seems that even today's best general-purpose static analysis tools have a hard time finding high-value bugs in our code. We're having much more success with testing tools. I hypothesize that our interesting bugs are due to the violation of complex invariants that are not tracked by these general-purpose tools. These are invariants such as "at most one of mFoo and mBar are non-null", or "if frame F has a child C, then C's mParent points to F". Inferring these invariants automatically or proving things about them from the code is very hard, not least because they are often violated temporarily. But tools that don't understand these invariants are likely to always have a high level of false positives in our code.

I think hope lies in a few different directions. Tools like Klocwork can be refined to home in on clear errors and throw away the chaff. There are more model-checking-like tools --- really, tools for doing intelligent, directed, systematic testing, incorporating knowledge of the code --- that can find complex error scenarios and guarantee no false positives. And I would like to see frameworks like Oink mature to make it easy to write custom analyses for specific problems in certain applications, probably aided by annotations. I have several ideas for custom analyses in Mozilla.

Dream Time II

Last night I dreamed about a bug in my new text code ... I'd forgotten to call FlushSpacingCache after adding justification space between reflowing and painting the text.

This morning I remembered, and fixed it.

I don't know if this is a good thing or a bad thing.

Crazy Noodle

There's a lot of good food in Auckland but I find myself repeatedly drawn back to Crazy Noodle in Newmarket. It's relatively cheap, the ambience is pleasant, uncrowded and child-friendly, the food is wonderful, and there's great variety. The Hong Kong-style Western food isn't altogether to my taste, but it's fun to try. Their standard dishes such as beef chow fun and Singapore fried rice noodle are --- wonder of wonders --- not greasy. I believe their "vanilla sago milkshake" is the best milkshake I've ever had; it's difficult to have just one.

The place is also a brisk 30-minute walk from my house, which is nice on a pleasant evening when I have time to spare and no children.

More Status

For various reasons I'm not going to Australia today, probably not at all. I'll be at work most of next week trying to catch up on things.

Out Of Action

I had to go to hospital on Friday to have an abcess drained, and I'll be recovering from the surgery for some days. And next week (26th-September 4) I'll be in Australia for a holiday. So overall I'm going to be hard to reach for a while; my apologies.

Update If you need something from me, say a review, that is critical and needs to be done in the next week or two, email me directly. I don't have the energy to properly triage my bugmail and review queue.

What's Wrong With The SVG Working Group

... is summed up in one email from Chris Lilley, the W3C Graphics Activity lead and Chair of the SVG Working Group.

The current draft of SVG 1.2 Tiny defines <svg:a target="_self"> to work in a way that's incompatible with the de-facto standard implemented by all major browsers for the HTML <a> element. It is, apparently, compatible with the W3C's WebCGM Recommendation. Chris Lilley writes (to Boris Zbarsky, as it happens):

You have not explained why de-facto behaviour of some implementations of
one specification should have primacy over well specified, implemented
behaviour that is already in several Recs for two languages which are
compatible with one another.

Apparently it isn't obvious to Chris Lilley why the behaviour expected by millions of Web authors, Web documents and installed browsers (from a variety of vendors) should have primacy over a behaviour used by approximately no-one but which happens to be enshrined in a W3C specification (for a different format).

Wake up, W3C. We need you and this kind of nonsense isn't doing anyone any good.

Sweet Sorrow

My trip's almost over and I'm sitting in a hot, dingy New York motel room mulling it over.

Overall it's been wonderful. So many old friends and new friends, so much fun, so much talking, so many ideas, so many places, it's thrilling.

But my heart is heavy... These New York days and hot summer nights, cicadas in the trees, the IBM lab, waiting at the familiar North White Plains station, walking again on our old street with our old house, and spending time with dearly loved friends, they bring back such strong memories --- some two years old, some ten. Good memories. I wish we could have stayed here forever with snow and trains and fireflies.

But then there are other friends who I love just as much, in Boston and Pittsburgh and California. I want to be with them too, while we can still pretend we're young. I'm so glad --- and sad --- to have spent a little time with some of them in the last few weeks.

Most of all I want to be back in Auckland with my family and friends, and back at work doing what needs to be done. God willing, in a couple of days I will be.

How maudlin. I need to ride out this nostalgia and thank God for all my friends and every moment I have with them, making the most of the time. Anyway, to all of my friends out there --- thanks, you mean so much to me.

RGBA Windows

Today we had a sort of mini-hackfest at the office. Earlier in the week Larry Ewing had shown how to use Xgl+compiz's support for RGBA windows to implement window translucency in F-Spot in a very nice way, so I thought I'd extend Mozilla's translucent window support to use this mechanism. Up till now, on X we've only used the Shape extension, which only provides 1-bit transparency.

In a cairo build, it is almost no effort to implement. Basically you just need to call gdk_screen_get_rgba_colormap and make that the colormap for the toplevel GTK widget. A general patch for Mozilla will be a little more complicated because we need to set the colormap before "realizing" the widget, at which point we don't actually know whether the window will need translucency or not. I'm not quite sure what to do about this, but it might be OK to just always use RGBA windows all the time when Xgl and compiz are active (which we can detect).

Obligatory eye candy follows. The example is a standalone HTML document with a transparent background, containing HTML form widgets, script, and SVG.

Worlds In Collision

I've had a great week up here in Boston with the rest of my desktop team. It really helps to chat in ways that wouldn't reach the acivation threshold required to send email or ping someone on IRC. Of course we've also spent a lot of time in more structured meetings. The downside is that I haven't been able to do much reviewing or bugfixing this week...

Yesterday I visited MIT. I reflected on the nature of my professional life as I rushed from a talk on the openSuSE build infrastructure to talking with a student about widening operators in abstract interpretation ... prior to giving a talk about my real work on Mozilla and the Web. The talk went well, I think, although there was less audience interaction during the talk than I expected, given that there were W3C staff present --- my talk chides the W3C a little, hopefully in a constructive way. In fact I was pleasantly surprised when Tim Berners-Lee introduced himself after I'd finished, although for better or worse he had missed the chiding part. I did manage to complain to him about the SVG spec a little bit, so I hope something useful got through :-).

On Saturday I'm off to New York, looking forward to seeing friends and giving the talk again at IBM Hawthorne. I just hope my voice is OK, I've been hoarse all week, I think a combination of a virus and too much chit-chat. I'd like to avoid a repeat of my thesis defense when I croaked my way through the talk and had to answer questions in almost a whisper.

The Good, The Bad, And The Three-D

Last Friday I visited Linden Lab, makers of Second Life, to spend several hours talking with Callum, author of UBrowser. UBrowser is an interesting Gecko embedding that needs to direct all rendering to an offscreen area so it can be transformed by OpenGL for display. A surprising number of people have contacted me over time requesting help with this kind of problem, and I'd like to have all these people collaborating on a solution that can be properly supported in the Mozilla tree.

We need to choose between two alternative approaches:

• Extend existing Gecko ports, e.g. the Windows port, to support offscreen embedding
  
• Create a new Gecko port that does not depend on any window system --- the "membuf approach" taken by the first effort in this space, Crystalzilla
  

The decision is not clear-cut. For a long time I favoured the membuf approach because it's somewhat cleaner, portable, and could run on headless servers. But with more thought and after talking things through with Callum, I favour extending existing platform ports, for the following reasons:

• The changes required to Mozilla code turn out to be very minor. The membuf approach requires a significant amount of new code.
  
• A separate membuf port would be poorly maintained compared to the platform code that everyone else uses.
  
• Some important things such as plugins and native themes would be difficult or impossible with membuf.
  
• There are a few things that are harder to do when the native platform gets in the way, but nothing major. For example Mozillla windows must be hidden. Platforms seem to be able to do this OK.
  
• If support in headless servers is required, they'll just have to run Xvfb or Xvnc; not an ideal solution, but probably practical.
  

The plan is to work towards getting the UBrowser code ported to Linux and it and its support patches landed in Mozilla's CVS tree. Then people can use it as a starting point for their own embedding projects and start collaborating on improvements. The code is already xulrunner-based which is great.

Before my visit I had been confused by reports that plugins actually work in UBrowser, since I knew of no way to redirect plugin rendering to an offscreen bitmap. It turns out that the patch detects the offscreen painting case on Windows and asks plugins to print themselves via the NPAPI Print command, and Flash honours this. Other plugins such as Quicktime don't. We can't detect when the Flash plugin wants to repaint its window so UBrowser just updates it at 15Hz. Input events don't work in any plugin although that can probably be faked. On Linux we can probably solve these issues using the X Composite and Damage extensions, but I'm not sure what we can do on Mac or how we can complete the job on Windows. Nevertheless it's a step forward. It means that once we land this patch, the canvas drawWindow API will paint Flash content on Windows, which will be a nice bonus for the tab-preview extensions.

While there I also learned a lot about Second Life. It's a very interesting virtual world focused on the creativity and commerce of its residents. I think there's enormous potential for invention and fun --- and also vice. I certainly hope that people find ways to use it for more good than evil. Either way, I'm sure that Second Life is going to be extraordinarily successful.

Necrolicious

For a while things were going very well. The flight was good and I had a great weekend with friends here in California. Yesterday I went to IBM Almaden and had a lot of fun giving my talk and talking to some of the IBM researchers. There's some really cool Web-related, Firefox-related stuff going on there.

Then tragedy struck. Last night my laptop mysteriously died. It simply fails to boot with a "Fan Error". IBM's service people are shipping me a box with which I can ship it to them to be fixed. I have no idea if they'll actually be able to fix it and get it back to me before I leave California on Sunday. If not, I'm not sure what I'll do. I wonder why I can't take it somewhere to get it fixed quicker. The timing is terrible; it's significantly harder for me to get work done without it. I can however still meet up with people so I'll try to make the best of it.

I did have a good day today at the Mozilla Foundation offices, talking to people about a wide variety of things, and putting some faces to old names. I'm looking forward to more of that tomorrow.

On The Road

It's time for another trip to the USA to visit friends and colleagues. I'm leaving on Saturday night and returning to New Zealand on Thursday August 3. During that time I will be a little slower than normal at responding to email and other communications.

I'll be in California for the first week. I'm giving a talk at IBM Almaden on Monday, mostly about the engineering and strategic challenges we're facing as we try to drive the Web forward and some ways in which people can help. I'll be visiting the Mozilla building and Google Tuesday through Thursday, talking to people about various works in progress. On Friday I'll be up in San Francisco visiting Linden Labs and talking to them about their use of Gecko. I might take some time to rove around the city; I always love to walk across Golden Gate Bridge. On the weekends I'll be catching up with some of my many friends in the Bay Area.

In the last week of July I'll be in Boston visiting my Novell Linux desktop team as we plan work for our next major release. I'm really looking forward to that since I've had very little face time with these people. I will also be visiting MIT on Thursday to give a similar talk and meet some friends there.

The following weekend I'm heading to New York to catch up with friends there, including my buddies at IBM Watson, and I'll be giving the talk again there.

It should be a ton of fun, God willing, as long as tunnel ceilings and tornadoes don't fall on my head.

Eighty Minutes Of Terror

Yesterday I was the featured speaker at the monthly meeting of the Auckland Linux Users Group. I gave a demo of some new features in SLED 10 and Firefox, and talked about Linux and Firefox development. It went well. However, it came very very close to being a complete disaster.

I was working on textframes most of the day. At 5:45pm I got to a good stopping point and turned my attention to preparing for the talk at 7:30pm. (AKLUG meets in our office these days.) I figured that nearly two hours would be enough time to get my laptop set up with our projector, and decide what to show and say, with a dry run.

After futzing around with the projector for a few minutes I decided I should reboot my laptop with the VGA cable plugged in, which sometimes seems to be necessary to get it recognized. Then my troubles really began, because some time ago I had applied software updates --- including a new kernel --- and I hadn't rebooted since. (Part of my problems may have come from the fact that I had a mishmash of packages from various beta versions and updates, not a single clean install.) Linux started up OK but Xgl (a major part of what I wanted to demo, of course) wasn't running, because the installed ATI kernel module didn't match the kernel anymore. So I started fooling around trying to get a new ATI driver installed by various means. During this phase I managed to erase the installed ATI X driver so that I couldn't start any X server at all! At one stage I tried to use yast2 to set things up via Novell's driver update Web site, but for that you have to register on the site, and since Firefox couldn't run it popped me into w3m, a text-mode browser. By this time I was more than a little panicky and I just couldn't get w3m to work.

By 7:20pm people had started arriving, I still had no graphics at all and running screaming into the night was starting to look like my best option. It was painfully clear that it would not reflect well on Novell if I stood up and said "thanks for taking the time to show up, but I COULDN'T GET OUR ENTERPRISE-READY PRODUCT TO WORK. Goodbye!" Yet I had lost hope; getting things working in the last five minutes after you've been working on it for over an hour just doesn't happen in real life. I also knew that even if I could get it working, I had no chance to prepare a talk or figure out what I was going to demo. My prayer was "Lord, help me to get this working, and I'll handle, um no, YOU'll have to handle the talk as well!"

I was more distraught than I have been in years. In some high-pressure situations, like childbirth, you're in the hands of professionals who you simply have to assume will do the best possible job. In other situations, like a car crash, it may not be your fault, or by the time you apprehend the gravity of the situation, there's nothing you can do. Facing the slow approach of a disaster of your own making, which you may or may not be able to mitigate, is an experience I wish to avoid.

Well ... by the grace of God, Hollywood endings do happen. I realized I could speed things up a lot by logging into my laptop from my desktop and running X applications that way. I found instructions for downloading the ATI installer and building a driver package that would work on SLED10, and they worked. I installed the driver, ran SAX, and everything came up correctly, Xgl, compiz and all. Wow. Of course I still had to give the talk and demo absolutely cold --- including demoing features that I hadn't actually ever used before --- but the off-the-cuff presentation seemed to work well. I even tried to explain the basic architecture of Xgl+compiz in a non-technical way --- which I realized immediately was very risky without having worked out my metaphors in advance --- and I think I did a decent job. Phoah!

I felt decidedly odd for the rest of the evening, a cocktail of delayed panic, relief, and enthusiasm. I felt jumpy. I didn't really settle down until I got home and poured out the story to my wife.

CORBA Post Mortem

I don't usually blog links, but this article on why CORBA failed, while not surprising, is a useful and concise reminder of the perils of development driven by a consortium's consensus specifications.

It's a tricky issue. We need multi-vendor specifications to break down the network effects that encourage monopolization in markets of interoperating software. But we have to avoid falling into the traps of design by committee. There is hope; the Internet itself is an example of the process working successfully, although the fundamentals of the Internet are a lot less complex than what we're trying to do on the Web.

The Future Is Now

I've recently done some work to modify Gecko's paint invalidation architecture to work with SVG foreignobject. I've also brought SVG foreignobject up to date with SVG changes and fixed some related bugs. The result is that the foreignobject demo I did at XTech last year is now fully functional on trunk cairo builds! In fact, it's more functional than last year since, thanks to Blake Kaplan's hard work, the caret is now painted correctly in transformed content.

There are still many bugs related to transformed content. Among them:

• Popups (e.g. SELECT comboboxes) --- actually it's not 100% clear how these should work
  
• Some theme problems
  
• Scrolling
  
• Some painting problems on Windows (apparently)
  
• incredibly SLOW performance; some for reasons we understand and can fix, some for reasons that are fundamental (rotating images is slower than just drawing them normally), and probably some for reasons we don't understand yet
  
• Seams between rotated rectangular areas (e.g. visible in the Google logo in the screenshot below). These are fundamental with cairo's coverage-based antialiasing strategy, and we may have to move to FSAA to get rid of them.
  

At least now we are at the point where we can test and fix some of these issues.

Download a trunk nightly (warning: these can be very unstable!) built with cairo (Windows should be), and try this testcase.

Obligatory screenshot:

The Wild West

Today a friend of ours was baptised. Being a tramping, outdoorsy kind of guy he wanted to get baptised in the Piha Stream out on the west coast near Auckland. Great idea, but today turned out to be windy, wet and cold. Thus twenty-odd of us ended up trudging along a muddy, boggy track in the rain for half an hour to get to the ideal spot. Actually, since we were dressed well, it was a lot of fun! I'm glad we had a good excuse to do something crazy like that :-).

On the way there, we stopped at Piha beach. The wind was blowing in hard at high tide and I've never seen it so wild. The surf was roaring, with waves and white foam running hundreds of metres offshore. Waves were hitting the rocks hard, throwing up towering plumes of spray. It was utterly awesome in the best sense of the word.

On the beach there were a lot of very strange blue organisms that I'm pretty sure I've never seen before. They looked a lot like little blue water balloons with a stringy organic appendage. Furthermore they felt just like balloons --- the same feeling when squeezed, the same immediate return to their original shape when released, as if containing water or air under pressure. And when I stepped on one, it popped loudly. I'm not even sure if these were plants, or perhaps some variety of anenome or jellyfish, although part of them looked like it might once have been attached to the sea floor. Perhaps they contained water under pressure because they live fairly deep, and the storm happened to wash them up. Whatever they were, it was quite remarkable.

Duder Regional Park

Today our family went to Duder Regional Park near Auckland. It's a lovely spot, a farm on a peninsula near the islands of the southern Hauraki Gulf. It's winter now, so it was cool (about 10-15C) and a little wet, but still a great walk to the tip of the peninsula ... and we only saw about six other people the four hours we were there. The drive out there is also beautiful, passing through rural areas and right next to the water's edge around Maraetai. It also passes through Auckland's eastern suburbs, which have a high concentration of Asian immigrants and hence there are great places to eat on the way there or back.

The Future Of Western Civilization

I'm been expecting this for a while now: the meeting of MMORPGs and pornography, two of the most fashionable modern addictions rolled into one extraordinary package. I think I know what a lot of multicore CPUs and GPUs will be simulating. I predict the first group to get the total formula right will reap billions of dollars, and tarnish millions of lives in the process. In fact, if I was an evil mastermind (something I think I'd do rather well!) I'd be doing it myself. Actually, I'd also pull in another great addiction, gambling. People will figure that out soon enough.

If this takes off the way I think it might, the impact on the world at large could be very significant. We're doing a grand experiment on ourselves. More than ever, us Christians will have to look for God's help to "be in the world but not of it" ... raising kids is going to be an adventure too. Fun times.

Bad Choices

I was reading an interesting critique of the KDE Phonon framework and its comments. Without taking sides in that particular debate (although adding a layer of abstraction over complex frameworks that themselves are basically the same sort of abstraction over other components gives me a very queasy feeling), I do feel like ranting about comments like this:

It also lets the user choose whatever he or she wants. If a user likes Xine more than GStreamer, he or she can choose to use it.

ALERT! Most "users" have no idea what a multimedia framework IS. They don't know which one they're using, or which one they should be using. Pretty much everyone just wants something that works out of the box. If the user is ever forced to understand multimedia frameworks and choose one, just to get things to work, then something has gone catastrophically wrong.

I sometimes hear the slogan "Free software" (or "Linux") "is all about choice" used to defend this kind of diversity. That is a very misleading statement. Free software gives you all kinds of choices about what you can do with code, but a lot of those choices are bad ones. Sharing code has many advantages over forking or duplicating --- mainly avoiding wastage of development resources, but also simplifying deployment and support, and avoiding the need to pile on abstraction layers like Phonon which have their own costs. You shouldn't fork or duplicate unless you have a very good reason. Interestingly, we see very few forking decisions, and those we see seem to have good reasons behind them. Sadly, we see a lot of duplication decisions and mostly their justifications don't convince me.

[It could be that there are actually a lot of forking decisions but forks get resolved quickly (by all but one of the forks dying or merging back), while duplications don't. I wonder...]

This is particularly important when dealing with libraries and frameworks which other components depend on. If we have five MP3 players, that's not a terribly big deal. But if we have five GUI toolkits, then every GUI app has to choose one of those toolkits which encourages the creation of four clones of that app, i.e. more fragmentation.

Anyway, offering users a choice of which largely-equivalent desktop to use, or which multimedia framework to use, or which of five dozen crappy text editors to use*, isn't doing anyone any good compared to what could have been if those developers had pooled their efforts. It's far too late for the KDE/GNOME breach to ever be healed, but I still wish for consolidation in other areas of effort.

[* I have yet to find one Linux text editor with all the features I want. If the authors of some of the dozens of text editors out there had instead contributed to an existing editor, I think I'd have a better chance.]

DVD Madness

Apple and the "entertainment industry", you really clock my goat.

In the USA we bought a number of DVDs. We moved back to New Zealand and would like to buy some more DVDs, and also borrow some from friends. The region code control on our iMac's DVD drive prevents this; we have to choose to activate it for region 1 (USA and Canada) or region 4 (Australia, New Zealand, Mexico, the Caribbean, and South America) and you are only allowed to switch a few times.

What's really annoying is that these artificial constraints on what customers can do with their property don't actually achieve the goal for which they were designed --- to allow DVDs to be released to different markets at different times. In New Zealand (and as far as I know, every other country outside North America) all DVD players you buy in shops have been modified to ignore the region system, and are openly advertised as "multizone". (We don't have a DMCA here so I believe that such modification, done by non-contractually-bound third parties, is legal). If the purpose of the region system was to prevent DVDs released first in the USA making their way to other markets "early", that's evaporated.

Well, there is one class of drives that's not pre-modded: DVD drives in computers. But it's easy to find and download firmware and/or software hacks to get around those. In particular, DVD-playing software such as VLC and the Xine/mplayer stack can actually just crack the region scheme cryptography in real time. That's how I played DVDs in my Linux PC back in New York.

But it turns out that one group has been singled out by Apple and the DVD industry for punishment: owners of late-model Mac hardware. Newer Macs include Matshita drives that are apparently exceptionally difficult to mod, and for which no modded firmware is available. Furthermore these drives forbid access to the raw DVD data if there is a region mismatch, so the VLC and Xine/mplayer approach doesn't work. Foiled!

I could go out and buy a multizone do-everything DVD player from a local mall for 80 NZD (about 50 USD). But then I'd have to buy a TV to plug it into. And then I'd have to buy some furniture to put the TV on. And I'd have to start making and enforcing rules about our children's access to it. That's not appealing, although we may end up going that way.

I could buy an external DVD drive for my Mac and mod its firmware, but I can't find one under 300 NZD.

I could borrow a PC and rip all my DVDs onto blank disks or a portable hard drive, but that's a pain and would potentially lead to copyright violation, something I really want to avoid.

So, Apple and the entertainment industry, consider me one extremely dissatisfied customer. The Mac's been fun but I don't think I'll buy one again.

Assorted Thoughts

As a vehement opponent of monopolies, I am glad to see that the New Zealand government is taking a hard line against our local telecommunications monopoly. Yay!

The news from Iran gets increasingly more depressing. I'm amazed nuclear weapon technology has been contained as well as it has for the last sixty years ... just think, nuclear weapons are nearly as old as television and older than general-purpose computers. But the nuclearization of India, Pakistan, North Korean, and now --- seemingly inevitably --- Iran may indicate that we're reaching a tipping point. I just can't see a good future for a world of loose nukes. New Zealand may be one of the best places to be, but that's not saying much. Christians, start your prayers.

I've been fixing random Mozilla bugs lately, mostly regressions plus a few security/stability issues. I've also been spending a lot of time reviewing Darin's event/thread rewrite. It's great stuff which will simplify our code, fix a lot of bugs, and seems to improve performance.

Over the last few months I've also reviewed a lot of SVG fixes --- the SVG issues that I highlighted in February are largely fixed! A lot of the remaining performance issues are lower-level problems with cairo and platform-specific rendering code.

Now that SVG is getting cleaned up, I'm hopeful that Brian Birtles and the rest of the SVG crew will be able to look at updating and landing Brian's SMIL animation patch without much more delay.

I'm trying to finally land a patch which will unify all documents into a single view manager hierarchy. This will let you do things like have the browser stack translucent XUL elements over content windows. Every time I try to land it, we encounter performance problems, and my attempt tonight did too, so I backed it out. I did gather some data so hopefully I'll be able to figure out what the problem is and get it right.

I've started working on pulling plugin windows up to the top-level window. We should be able to do it in a way that also lets us compute visibility information for plugin windows, so you can have a DIV with a solid background actually render on top of the plugin window --- by clipping a hole in the window where the DIV needds to appear. It's not hard in theory but as usual the devil is in the details, getting reasonable performance and dealing with fallout from touching the fragile, platform-specific plugin handling code.

I've installed SuSE Linux Enterprise Desktop 10 (hope I got that right!) on my desktop and laptop. It's very slick. Xgl and compiz look fantastic on my laptop, and work flawlessly with my ATI graphics hardware, but I've had some problems with my desktop's NVidia setup ... this is ironic because NVidia has usually been considered reliable and ATI the problem child of Linux graphics drivers. I have to say that due to some combination of Xgl, compiz, the fonts, the default theme, and my laptop's high-density screen, my laptop now has the sexiest-looking graphical environment that I've ever used (and I'm writing this on an iMac at home). I wonder what more Windows Vista will offer that required Microsoft to dictate massive changes to their graphics stack and onerous hardware requirements, not to mention spend so much time developing the thing.

One thing that I really like about compiz that you don't see in the demos is that it grays out windows that aren't responding to events (a lovely effect, it makes the window a little darker and turns it black-and-white). This is a real usability win; you know that the application is not going to be responsive, so you don't keep clicking on it in hope, and there's no uncertainty about "maybe it didn't work because I clicked in the wrong place". Even better, you can see when it becomes usable again without having to periodically click on it to check.

One effect is still missing ... applications that crash just disappear abruptly. Abrupt shutdown should make the application's window explode in a realistic particle simulation.

Another Mozilla thing I'm planning to work on soon is getting the new textframe code into the tree so we can get people working on improving it in multiple directions. Before I do that I want to simplify the way textframes and inline reflow handle some corner cases, which I'll talk about in another blog post.

Update: By popular demand, I've made a screenshot. Because a great deal of the sexiness is the animation, you really need a video, something I can't be bothered doing (and which would probably get me booted off weblogs.mozillazine.org). All you can see here is the obligatory cube in action, plus window drop shadows and translucent window borders. Because the cube sides are lit, the actual desktop is considerably brighter than you see here.

Here's another showing the desktop in normal mode. Here I've stopped the OpenOffice process to demonstrate the grayed-out effect.

Dream Time

I have interesting dreams. Many involve the end of the world in various kinds of techno-apocalypse. Last night's was different.

Last night I visited a digital commune in Silicon Valley. Many people lived in a huge and lovely building, always online and connected to an all-encompassing wiki/blogs/social networking system. Cameras and sensors fed data upstream, and in turn the system regulated downstream behaviour, every action and interaction a shadow of an online transaction. I wandered around, disconnected, irrelevant to its participants.

Sometimes the feed from one camera seemed to catch the attention of the entire community, the people nearby would point to it, and there'd be a rush to see who could touch it first, no matter how inaccessible. People spontaneously formed structures, piling up to lift someone's finger to a great height. Now it reminds me of the Sistine Chapel.

The system broke down. On screens that tracked the status of community members, I could see people disconnecting, losing karma. A confused crowd gathered in a large hall and was addressed by a group of leaders. I looked on from a distance, discussing with another observer, inexplicably Andrew Appel. There was an air of finality, of an experiment that failed.

I'm not sure what it all means. It smelled like Google, Orkut and 1984. It did, however, make a nice change from being hunted by robots.

GNOME Themes In HTML

A while ago we turned off GTK themes in HTML content in Mozilla cairo builds because of some bugs and performance issues. I wanted to get them turned on again and fixed some of the bugs, and then ran into a fundamental problem in the Clearlooks engine (which Novell's new theme is based on), and which probably applies to other themes.

The general problem is that the Clearlooks design simply isn't suitable for use with HTML. The immediate problem is that it wants to render the borders of buttons, textboxes and other widgets with a gradient based on the colors of their containing window (from the window's inset_dark to inset_light). This is no good for HTML because in HTML the widget could be (and often is) drawn over any color background, or an image, or any combination of anything, so one can't just "set the right colors" and be sure to get good results. This is not a technical problem, it is a problem with the design of Clearlooks ... it wouldn't look good even if you had a pure GTK+ application using Clearlooks widgets over complex backgrounds.

I'm not sure where to go from here. I think we need theme designers to think about what they want for themes in HTML content, and then we can figure out the best way to implement it. Certainly, Web uses will have to be taken into account during theme design. I plan to talk about this with Novell's theme artists when I get a chance.

Asiana IV

One of the things that struck me when I first got back to Auckland last year was how great Auckland's Asian food is. Over the last year it has continued to impress.

Sunshine remains the best yum cha in town in my opinion, and would be ranked among the best in North America. But there are many other quality places in Auckland. Just a short walk from our house is Hee's Garden on Mount Eden Road, which despite being a lone Chinese restaurant in what's practically the suburbs, and despite being of modest size, serves quality yum cha seven days a week. The selection isn't huge but what they do, they do well. Dragon on Eliot Street in town is also good.

Recently some friends introduced us to Hansan in Mount Wellington. Among car yards and strip malls, this place offers a big menu of cheap and really excellent Vietnamese food. They also have great milkshakes. Highly recommended.

Our family has been working our way up and down the dozens of restaurants along Dominion Road. There are a lot of cheap grease pits and MSG dispensers, but there are also places where you can have a good meal at a fair price and get free soup and dessert if you're lucky. We like Love A Duck and Sum's Kitchen.

In Newmarket we really like Crazy Noodle, which along with Chinese food offers Hong Kong style Western food. Toasted sandwhiches with condensed milk, baked fish on rice in curry sauce, that sort of thing. Sunnytown and Sun World are acceptable places for yum cha or dinner. Gengy's is a fun Mongolian BBQ place but either it's going downhill or we're tiring of it.

I still haven't made it to Empress Garden, which used to be a regular stop in my university days (12 years ago ... erp). I've heard good things about Yum Cha on Quay St. And ... something ... Pearl in Newmarket, which we didn't like, has reopened as a steamboat restaurant so we'll have to check it out again. The one thing we haven't found yet is shabu shabu as good as Minnie's in Flushing, New York.

Read My Lips

Over the weekend I completed my last set of US tax returns (God willing!). This is an occasion for great rejoicing.

I've always found the US tax return system very poorly designed. There are so many rules, categories and exemptions --- presumably due to the sheer breadth of citizens, territories and activities the system covers, compounded by endless Congressional modifications driven by special interest lobbyists. The need for file one or more state returns along with the federal return, each with slightly different rules, exacerbates the pain. I'm never really sure that I've gotten my returns right, especially whether I've claimed everything I could have.

This situation is actually really bad. The compliance burden on the economy must be significant. But worse, it favours those who can amortize the cost of a good accountant over a large income and obtain every tax benefit available. Such a complex system must also contain anomalies --- bugs ---- at any given time that can be further exploited by those accountants.

As far as I can tell, this situation is not really fixable. The US political system is too broken for for anyone to pursue successful comprehensive reform, and in any case the system is so large and has so many requirements that it resembles one of those software systems that can never be successfully rewritten. And of course some of the big problems, such as the existence of state taxation, are constitutionally fundamental.

In smaller countries like New Zealand this is not true, and indeed, over the years there have been successful efforts to simplify our tax system. What we have right now could be improved, but it's far simpler for most people than the US system. This ability to reform and simplify is simply an advantage that I believe comes easier to small countries then large ones. Of course it's merely one part of the size tradeoff, and large countries have many advantages over small ones ... probably much greater advantages, all told.

Having A Miserable April Fool's Day

Each year I find the first of April to be really unpleasant. For an information junkie like me, it's the one day a year when everyone conspires to pee in the water supply.

And So It Begins ...

... the great battle of our time.

"Certainly one thing is clear�even with Atlas, the browser's capabilities simply don't match those of Windows itself," Lhotka said. "The more you want your Web pages to act like Windows, the more expensive it becomes. Atlas helps ease some of that cost and pain, but my feeling is that ultimately Atlas is a bridge between simple HTML and WPF [Windows Presentation Foundation], filling an important niche."

There's a high chance that that quote was fed directly to Lhotka by Microsoft's PR agency. When I was awarded a Microsoft student fellowship, the PR agency sent me a press release complete with a quote from me that they'd made up, asking me to just give the OK. I suppose that all PR agencies work the same way.

Microsoft's agenda is unfolding just as I thought it would. But Atlas and Expression are an interesting twist, because at first glance making Web development easier might actually discourage WPF adoption. I presume that Microsoft is afraid of losing developers to other tools, such as IBM's Ajax Tools Framework; by keeping them on Microsoft tools, there is a reduced chance they will use non-IE features (SVG, canvas, columns and so on), Microsoft gets more leverage on servers, and the developers can be more easily transitioned to WPF. And as Doron noted, they can structure their framework to maximise IE performance and compatibility compared to other browsers.

The good news here is that Microsoft is having to play our game for now. Producing good tools for the Web is good for the Web no matter what the motivation. It's up to Firefox and Opera and Webkit and WHATWG and the ATF people and all our other allies to produce great tools and browsers to avoid the "extinguish" phase of this round of triple-E.