Wednesday, 28 December 2011

A Case For Non-Fatal Assertions

Jeff Walden is doing awesome work on the foundations of Gecko to unify the lowest-level infrastructure used by JS and the rest of Gecko and to rebase it on modern C/C++ standards. But there is a controversy about whether non-fatal assertions should be part of that infrastructure. This issue came up before, not long ago.

I strongly believe that non-fatal assertions are valuable when used to report the presence of a bug that is not as severe as a browser crash. An example I just pulled out of nsBlockFrame::Reflow:

    ReflowBullet(state, metrics, lineTop);
    NS_ASSERTION(!BulletIsEmpty() || metrics.height == 0,
                 "empty bullet took up space");

If this assertion fails, then we have detected a Gecko bug which should be reported and eventually fixed. If the assertion failure is a regression, and we detect it in time, we will try very hard to fix it before the patch ships in a browser release. (If the regression is triggered by our layout reftest test suite, we will almost certainly detect it on checkin since reftests go orange on new non-fatal assertion failures.) However, if this assertion failure is the only thing that goes wrong, there will almost certainly be no ill-effects beyond the page having a slightly incorrect layout --- maybe. Some pages might trigger the assertion but appear to render correctly. Even if the bug causes a detectable test failure, the assertion helps to narrow down the cause and understand the code.

Using a fatal assertion here would have the same benefits but additional costs. A test run hitting the assertion would abort the suite, meaning we lose the results for the rest of the tests in the suite. This makes fixing test failures slower and more painful than necessary since more runs of the suite will often be required. (It's similar to a compiler always aborting after the first syntax error in a compilation unit.) Another cost is that if you are using a debug build for some reason and you hit this bug while trying to work on something else, your work will be unnecessarily blocked.

At this point some will say "Ah! But assertion failures should always just be fixed. Since non-fatal assertions are more ignorable, they encourage you to leave bugs unfixed."

That statement ignores the reality of bug priorities. An assertion failure is just a bug and needs to be prioritized along with other bugs. If our assertion infrastructure and associated project rules forced us to always prioritize some rare list bullet spacing bug above all bugs that don't trigger assertions, then that infrastructure would be actively damaging our project. We would have to respond by simply removing a lot of our assertions and losing their benefits. It is crucial to be able to ignore unimportant bugs.

Having said all that, fatal assertions certainly have their place. I imagine that in the JS engine almost any bug will lead to a crash sooner or later, so it makes sense for fatal assertions to prevail there because the downside is minimal; you were going to crash anyway, and crashing later might just be confusing.

Addendum: an objection to non-fatal assertions is that libc "assert" and some other assertion mechanisms are fatal, so to call something non-fatal an "assertion" is confusing. That may be true for some people, but it's cultural. The culture in Gecko is that NS_ASSERTION is non-fatal. Maybe renaming our non-fatal assertion mechanism would end up being a net win by some point in the future, but I'm dubious. Although I wouldn't actually mind too much; I just want non-fatal assertions, whatever we call them.

Tuesday, 27 December 2011

Revelation

Saw this Penn Jillette quote:

There is no god and that’s the simple truth. If every trace of any single religion died out and nothing were passed on, it would never be created exactly that way again. There might be some other nonsense in its place, but not that exact nonsense. If all of science were wiped out, it would still be true and someone would find a way to figure it all out again.

Well, yeah. Christianity (and some other major religions) unabashedly depend on revelation, the idea that God is so transcendent that humans can't figure out much about him on their own, so he has to tell us. And most of that telling doesn't come to us individually, it has to be passed on to us. Annihilate the revelation and we're in the dark again. So ignoring the pejorative language, and apart from the first sentence, there isn't much for a Christian to disagree with in that quote.

Sunday, 11 December 2011

Television

I haven't watched much broadcast television since I started graduate school. In the first few years in Pittsburgh we didn't have a TV in the house and I fell out of the habit. About ten years ago my wife and I started buying or borrowing DVDs and working through particular shows that I'd heard were good. The first was Mad About You, followed by Buffy The Vampire Slayer, 24 (season 1 only; my wife found the cliffhanger endings intolerable), Fawlty Towers, Band Of Brothers, Angel, new Doctor Who, new Battlestar Galactica, The Wire, and a couple of Hong Kong kung-fu shows that I can't explain. We started a few other shows and then abandoned them after discovering that my wife or I didn't like them.

Most of those shows are very good. The Wire is exceptional. Lots of other people have written about it more eloquently than I can. I can understand why the mayor of Reykjavik semi-seriously demanded his allies watch all five seasons. Anyone interested in politics or social institutions should watch it.

However, the best of the lot is Buffy The Vampire Slayer. It starts as silly fun --- and remains so --- but as it progresses, it constantly reveals new levels of creative genius. The imagination, writing, execution are staggering. It's not flawless, but it's the only show I seriously consider watching again. It's not for everyone, but it is undoubtedly great art.

Love

Our family went to "Christmas In The Park" tonight at the Auckland Domain. Overall it was pretty fun but the choices of songs to cover were not entirely to my taste. There were too many insipid pop songs about "love", a genre I detest. Who writes this stuff? I appreciate the need to sell music to teenagers, who may think "love" is a fleeting obsession with another person, but celebrating that is just dumb. What is that compared to real, long-lasting love, that is based on self-sacrifice, not just in moments but over the long haul? Down with angsty superficial "love" songs!

Wednesday, 7 December 2011

Developernomics

Forbes has an interesting article:

The one absolutely solid place to store your capital today — if you know how to do it – is in software developers’ wallets. If the world survives looming financial apocalypse dangers at all, this is the one investment that will weather the storms. It doesn’t matter whether you are an individual or a corporation, or what corner of the world you inhabit. You need to find a way to invest in software developers.

The whole article is rather over-the-top, in my opinion, but its points are mostly good and it's thought-provoking. It's also probably useful to point people at when they're considering career choices ... such as the parent I met who was worried that if her Linux-loving daughter became a developer, all those jobs might disappear in the recession. (Her daughter ended up not going into computer science; university counsellors steered her away from it, apparently :-(.)

Tuesday, 29 November 2011

Moves In Computer Science Education

Apparently there is a push in the UK to replace teaching of business software in schools (what one might refer to as "IT education") with teaching of programming (which I might refer to as "CS education"). That would be a very good move.

No doubt I'll be preaching to the choir on this blog, but "IT education" teaches kids boring, low-grade skills and prepares them to work as office drones. It does nothing to prepare them to work on interesting high-value high-tech projects. In fact it very likely damages the high-tech economy, by teaching bright kids to avoid computer-oriented skills and jobs at all costs. It would be much better for schools to avoid computers altogether than "teach" Word, Excel and Powerpoint.

For what it's worth, in my house our kids spend almost no time on the computer(s). I am very happy about that. I assume that they'll dive into it as teenagers, and I hope being cognitively much better equipped they won't develop a habit of just consuming content, but instead will be more likely to learn to master the theory and practice of the machine.

Wednesday, 23 November 2011

Politics

A New Zealand election is coming up and I'm struggling to figure out who to vote for.

I live in the Epsom electorate, where National is trying to leverage our MMP system by encouraging people to vote for the righter-wing ACT candidate instead of their own. (Someone put up a billboard for the National candidate and his own team took it down!) If the ACT candidate is elected, ACT will get Parliamentary representation proportional to their party vote even if their party vote is under the normal 5% threshold. Those ACT MPs will have to ally with National, boosting National's chances of forming a government. Although I like the MMP system in general, I dislike this kind of gamesmanship so I'll try to make it fail by tactically voting for the National candidate. Current polls suggest this is going to work.

Having become politically aware during the Muldoon years under the first-past-the-post system, in elections where National got the majority of seats with a smaller popular vote than Labour, I have no desire to return to that system. I also think that the influence of third parties has mostly been beneficial under MMP. The alternative proportional systems on offer are not clearly better than MMP IMHO. So I'll vote to keep MMP.

My difficulty is in assigning my party vote, which is what really matters. Generally I think private enterprise works better than central planning, incentives work better than handouts, and it's better for people to take responsibility for themselves than look to governments to solve their problems. So I'd veer right, but right-wing politics tends to attract people who for selfish reasons promote --- or at least do not resist --- social injustice and unbridled corporate power. So ... hmm.

I tend to think that when policy differences are relatively small, as they tend to be in New Zealand, efficiency and sensible decision-making trump policy details. So give me capable, results-oriented government that's willing to choose the approach that works best regardless of whether it fits some preconceived ideology.

In this particular election I like some policies from both major parties, and dislike others. I particularly dislike Labour's promise to repeal the 90-day trial period for new employees --- that helps Mozilla to hire people we otherwise couldn't. On the other hand, I like Labour's plans to raise the retirement age and introduce a capital gains tax. What do do...

Unfortunately this post hasn't helped me make up my mind. It was worth a try!

Tuesday, 22 November 2011

ITEX And TVNZ

Tomorrow I'll be speaking at ITEX, giving a similar talk to my one at Web Directions South about the current challenges to the open Web. I'll need to update it a bit to cover the recent Flash announcement, but the message is unchanged: the world of open Web standards has overcome some challenges (the threat of plugins, browser stagnation) and is evolving rapidly in terms of functionality and reach, but faces even greater challenges: proprietary mobile platforms, and platforms that are based on Web technology but find other ways to lock users in (e.g. browser-specific apps and app stores as found in Chrome and Windows 8).

Also, I'm going to TVNZ early tomorrow morning for a short interview on their Business Breakfast show, talking about some of that. It should screen around 6:05am so even if I have a Rick Perry moment I expect only a dozen people or so will be watching.

Wednesday, 16 November 2011

Latency Of HTML5 <audio> Sounds

One of the common complaints about APIs for Web gaming is that there's no standard API to just play sounds in response to game events with low latency.

However, the <audio> element API is perfectly adequate for this. Use <audio src="shot.wav" id="shot" preload> and every time you want to play the sound, use document.getElementById("shot").cloneNode(true).play(). The HTML5 spec says that the browser can use the same resource data for the clone as for the original element (no HTML request should ever be needed). The cloned node is not in the document and has no event handlers so most of the overhead of media elements can be optimized away. If there are latency issues, then they are simply browser implementation issues that we should fix. New API is not needed here, and it hardly ever makes sense to respond to browsers poorly implementing one API by asking them to implement another one.

I believe people who say these <audio> implementation issues exist. Unfortunately, in our limited testing the latency of cloning and playing an <audio> element is pretty good in Firefox and Chrome on Window. So from people encountering problems in this area, we really need actionable bugs with testcases to be filed against the offending browsers. Please!!!

Tuesday, 15 November 2011

Freedom Never Gets Old

A venturebeat.com article about mobile and Web apps says

But he [Jay Sullivan] doesn’t resort to the familiar (and tired) ideologies about freedom from corporate technological tyranny that figure large in Mozilla’s current ad campaign. Rather, he gets downright practical.

I assume "tired" is the reporter's spin, not Jay's. The pejorative use of "ideologies" is gratuitous; a better word is "principles".

The struggle to keep the Web open has been going on a while now and isn't going to end. I'm not surprised if people are getting bored by it. But we can't afford to weary of it. Vendors never get tired of binding developers and customers into their proprietary stacks.

Computerworld Interview

A couple of weeks ago I spoke to a Computerworld journalist on the phone. The resulting articles have now been published.

These articles are pretty faithful to what I said. Great!

Friday, 11 November 2011

HTML Video DRM

Based on the recent Adobe and Microsoft announcements, I expect most Flash and Silverlight Web developers are now thinking about how they'll move to Web standards. This is a good thing, but there are still features in those plugins that are missing from Web standards. One big feature is DRM for video. The problem is that some big content providers insist on onerous DRM that necessarily violates some of our open Web principles (such as Web content being equally usable on any platform, based on royalty-free standards, and those standards being implementable in free software).

We will probably get into a situation where Web video distributors will be desperate for an in-browser strong DRM solution ASAP, and most browser vendors (who don't care all that much about those principles) will step up to give them whatever they want, leaving Mozilla in another difficult position. I wish I could see a reasonable solution, but right now I can't. It seems even harder than the codec problem.

I am curious about what IE10 will do in this area. If Windows 8 Metro IE10 doesn't support Flash or Silverlight (as Microsoft says), and they don't support a strong DRM solution (and I haven't heard they will, yet), how will distributors of expensive HD content satisfy their DRM requirements and play back in Windows 8 Metro IE10?

Thursday, 10 November 2011

The End Of Plugins

If you haven't already heard, Adobe is going to stop Flash development for mobile browsers, which given the rising importance of mobile means they're effectively giving up on Flash on the Web altogether. Now there are strong noises that Microsoft is going to stop Silverlight development. This comes on top of Windows 8 Metro mode not supporting plugins (which of course follows Apple's early lead in iOS). This is huge! It means that browser plugins are going to go away. Oracle hasn't said anything yet, but there's no way browser vendors are going to carry NPAPI support forever, or add any new NPAPI features now, just to run Java and some other relatively little-used plugins. (This is not official Mozilla policy, mind you, just my prediction.)

What does this mean to Mozilla? It means we need to stop investing in new features for better integration with NPAPI plugins. Our investment in NPAPI support should focus solely on improving the user experience for consumers of existing plugin content, but limited by the knowledge that the consumption of such content --- and production of new content --- will decline dramatically over time. There's still probably quite a few things we should do there. Eventually we'll be able to disable plugins and rip out a lot of code.

What does this mean for the Web? This is a big victory for open Web standards. Two powerful proprietary competitors have been defeated. Sooner or later we'll be able to stop taking plugins into account when we design new features (e.g. our full-screen support has to consider how plugins work in a full-screen document).

Hooray!

Monday, 7 November 2011

Drawing DOM Content To Canvas

I see that one feature Web developers are asking for is the ability to draw DOM objects to an HTML canvas. I've got good news and bad news about that. The bad news is that such a feature has to be designed very carefully or it's a security risk. The good news is that this feature already exists in Web standards, and it works in Firefox and Chrome (at least)!

Here's a demo of rendering HTML elements into a canvas. The trick is to create an SVG image containing the content you want to render, in this case a <foreignObject> containing HTML, and draw that image to the canvas. Constructing the SVG image as a data: URI lets you compose the content dynamically without requiring an external resource load. There is nothing tricky going on here; this feature works exactly as it should according to the specs.

The resulting canvas should even be "origin-clean" so the toDataURL() method works on it. Currently that isn't true in Firefox or Chrome. In Firefox, that's because of bug 672013; we clear the origin-clean flag whenever we draw an SVG image to a canvas. We'll fix that shortly. The problem with Chrome is a general Webkit bug that documents loaded from data: URIs are considered to have a different origin from the containing document, contrary to the spec.

One obvious question is how this feature can be secure, in light of the concerns I raised above. We're relying on the fact that our implementation of SVG images is very restrictive. For example, we do not allow an SVG image to load any external resource, even one that appears to be from the same domain. Resources such as JPEGs or IFRAMEs in an SVG image have to be included as data: URIs so they're inline in the image. Scripting in an SVG image is not allowed, there is no way to access the DOM of an SVG image from other scripts, and DOM elements in SVG images cannot receive input events. Thus there is no way to load privileged information into a form control (such as a full path in a <input type="file">) and render it. We don't apply visited-link styles in SVG images, and we don't render native themes in SVG images. Some of those limitations probably make this technique unusable for some use-cases, especially the restriction that script can't directly touch DOM nodes that get rendered to the canvas --- but that restriction is important for security.

Another limitation that's a bit annoying is that you have to use XML/XHTML syntax for the SVG image. It would be nice if you could use HTML syntax for the HTML content at least. Maybe it would be worth defining an "HTML image format" (image/html?) that uses the HTML5 parser to parse the image contents. Another issue is having to wait for the onload event to fire, drawing asynchronously. It might make sense to add a convenience API like "drawHTML(string, x, y, w, h)" which behaves like drawing an SVG image containing <foreignObject> (so doesn't introduce any new security concerns) but uses the HTML5 parser and is much more convenient. On the other hand, it might be better for developers to experiment with the existing solution and use that experience when designing a better API.

Saturday, 5 November 2011

Fun Things To Do In Auckland

Last weekend we went to Tree Adventures out in Woodhill Forest and had a great time. I have a considerable fear of heights and still had a great time. This would be a great outdoor activity for anyone over the age of six or seven. I'm already looking forward to going back.

Tonight is Guy Fawke's Day. It's the first fine November 5 for a few years so we did what I've always wanted to do: walk up Mt Eden and watch fireworks from there. I was a bit worried there might be crazy drunk people letting off fireworks, but that wasn't a problem. The only problem was that it was a bit cold (just dress warmly). Other than that it was fantastic. You can see people's fireworks in their back yards all over Auckland, it's quite magical. The nearest major fireworks show tonight was in Waitakere, and although distant it was very clearly visible from Mt Eden and lovely to watch. There were probably two or three hundred people up there --- not too crowded, and almost all very well behaved. There were a few people letting off their own fireworks, and one guy lost control of one in the carpark, but it wasn't a big deal. Probably any of Auckland's volcanic cones would give a good experience --- highly recommended.

Tuesday, 1 November 2011

Public Service Reminder For GMail Users

I was just reading the account of yet another victim of identity theft, whose GMail account was broken into. It's tragic, and preventable. If you have a smartphone, you really ought to set up GMail's two-factor authentication right now. It works very well for me.

Update And encourage your GMail-using friends to do the same!

Monday, 24 October 2011

A Sermon For The Rugby World Cup Final

Yesterday I had the opportunity to speak to our church congregation on the subject of "being content in all circumstances" ... a timely subject, I thought, given the impending Rugby World Cup final.

I started with 1 Thessalonians 5:16-18, Romans 8:35-39, and Phillipians 4:6-7 and 11-13. A simple summary is that people who trust God and receive forgiveness through Jesus should not be perturbed by changing circumstances. It's sometimes appropriate to feel sad and grieve --- Jesus did --- but our internal peace and security should not be affected.

These are relatively well-known passages but it's easy to see that this is not true in the lives of most Christians ... certainly not for those NZers I know who watch rugby games from behind the couch! So the focus of my message was getting from knowing how we should feel to actually feeling it.

Some people believe that our feelings are mostly immutable, beyond our conscious control. I don't agree. God doesn't ask us to do the impossible. Furthermore I have some experience at training my feelings in certain directions. A small example is how I've vowed, even on this blog, to get less emotionally involved in rugby games. Thinking and talking about that helps make the feelings follow.

Another example is love and marriage. I think it's very important to act lovingly towards your spouse whenever possible, even if you don't feel like it at the time, or you think it doesn't matter. In our culture people habitually joke about having a poor relationship with their spouse --- jokes about nagging wives, selfish husbands, relationships with in-laws, and all that. I think these traditions are destructive. Even small intimacies like holding hands are good habits and promote positive feelings. They're worth consciously maintaining.

The idea of acting in a certain way to encourage feelings to follow raises the possibility of that one becomes a hypocrite. I think it depends whether your goal is to promote genuine feelings in yourself, or to convince the world you're something you're not. In C.S. Lewis' "Surprised By Joy", he wrote:

The distinction between pretending to be better than you are and beginning to be better in reality is finer than moral sleuthhounds conceive…. When a boor first enters the society of courteous people what can he do, for a while, except imitate the motions? How can he learn except by imitation?

Christians must remember that change is not something that comes about just through our own efforts. Paul says he can do all this "through he [Jesus] who gives me strength". We believe in self-improvement, but "of self", not "by self".

Christian equanimity isn't about detachment. None of the passages quoted tell us to care less about other people or what's going on in the world. We don't need to walk along a razor's edge between caring too much and caring too little. Instead, we can continue to care about all sorts of things, but put them in the right perspective: dwarfed by God.

As it happens, the All Blacks managed to win last night, but it was a tense game and I hope some of the people who heard this message put it into practice. :-) There will be lots of more important opportunities to do so.

A Few Words On The Rugby World Cup

Now that we've won the RWC, I can make a few comments without sounding like a sore loser.

The RWC knock-out format means that the result is largely determined by luck --- more precisely, factors beyond the control of the winning team. The All Blacks were lucky to win last night. France were unlucky to lose, but were lucky to make the final at all.

It makes no sense for commenters to ascribe victory to "destiny", "will to win" etc when France could have won with a penalty kick that they simply missed.

I like watching rugby, but the subjectiveness of referring is a great flaw in the game. Ideally a game should not only be fair, but should be clearly seen to be fair. Rugby is seldom the latter.

I think all NZ supporters should sympathise with French supporters who feel they were cheated by the referee. Most of us know how they feel, from 2007.

The All Blacks choked. They still managed to win the game (luckily, see above), but it was clear they crumbled under the pressure, at least some of them did. It was surprising since it seemed that last week's game against Australia was higher-pressure.

I hope success at this World Cup doesn't obscure the lessons that should be learned for the next one from this near-disastrous performance.

I really enjoyed the game and I'm glad we won, but I think I could even have enjoyed it if the French had won, as I earlier pledged.

Thursday, 13 October 2011

Web Directions South

Today and tomorrow I'm going to be at Web Directions South in Sydney. Today I'm giving a talk about the importance of Web standards in light of the challenges to the Web from mobile application platforms, and also platforms that are based on Web technology but "privatized" by various means (such as Windows 8, the Chrome app store, and Webkit-only mobile Web sites). It should be fun!

Thursday, 6 October 2011

Jobs

I've never really liked Steve Jobs and I see no reason to start now.

However, I think the story of the rise and fall and rising again of Steve Jobs and Apple is amazing and deserves much telling. It's far more interesting than Facebook, so far.

I remember the dark days of Apple pretty well, when Guy Kawasaki leading a small band of crazy fans seemed to be all that was keeping Apple going. Those were the days of Copland, Cyberdog, OpenDoc, Dylan and Newton. I remember "The Apple engineer <unknown> has unexpectedly quit" T-shirts. I remember when Jobs returned and started shaking up Apple, scathing comments from ex-Apple employees at CMU describing him as "mentally ill". (Jim, I'm thinking of you :-).) I remember when the iPod was launched in 2001 and friends said "that'll never sell!"

I also remember when it was unthinkable that a platform vendor could dictate by fiat what applications they would and would not allow to run on their platform. I remember when the idea that a major company would wield bogus software patents to eliminate competitors was anathema.

How far we've come.

Friday, 30 September 2011

Shifts In Promoting The Open Web

Historically Mozilla spent quite a bit of energy promoting use of the "open Web" over proprietary platforms and non-standard browser extensions (IE6). This is still needed, but the landscape has shifted and I think our emphasis needs to change with it.

The platforms we need to worry about have changed a lot. Instead of WPF, Silverlight and Flash, the proprietary stacks competing for developers are now iOS and Android. Accordingly, the features the Web needs to catch up on are mobile focused. We need to be knocking down the barriers that make mobile app developers write native apps instead of Web apps (and we are!) and we need to be promoting the development and use of Web apps instead of native mobile apps. Demos that only work on desktop browsers are less important.

The open Web also has some interesting new platform competitors: platforms that build on (embrace and extend?) Web standards but take control of application delivery to create a vendor-controlled platform. The Chrome app store and the upcoming Windows 8 Metro app store are examples of this. I was very disappointed to see offline GMail and Google Calendar restricted to being Chrome apps. Even though Angry Birds works great in Firefox, its Chrome branding alone probably makes people think it only works in Chrome. To counter this we have to make sure that browser competition stays strong and offer developers browser-neutral Web app stores. Mozilla is working on these, of course :-). We also need to send a clear message that browser-specific app stores run counter to the open Web.

A less obvious form of platform competition is application developers targeting a single browser or browser engine. Google is explicitly telling its developers to target Chrome-only at first and support other browsers as an afterthought. That's understandable, but still disturbing. Also disturbing is that many mobile sites only target Webkit (sometimes implicitly by relying on Webkit bugs, more often explicitly by relying on -webkit-prefixed features). Many mobile developers, even developers at good places like Google, are reluctant to change this behavior. This is a huge problem for the open Web. We need an open-Web-standards campaign targeted at mobile Web developers. We need to be clear that apps working only a single browser engine, whichever engine that is, run counter to the open Web.

It's unfortunate that only Mozilla (and maybe Opera) of the major browser vendors has no vested interest in the success of a non-open-Web platform. I'm glad I work here.

One of the great things about the Web right now is the explosion of new features and standards for Web developers. However, we need to carefully distinguish good open standards from "open-washed" single-vendor initiatives. Not every proposed standard is good for the Web, even if it comes with an open-source implementation. Maciej Stachowiak points out a few Google projects --- VP8, SPDY, Pepper, and Native Client --- that, while they may be good ideas, fall short of true open standards to varying degrees. (The lack of a good spec for VP8 is an issue that we at Mozilla can and probably should address ourselves.) There are also cases where even though a good multi-vendor spec exists and some Web developers want it, the feature is not good for the Web and should be resisted. So I think when we promote the open Web we need to be very discerning about which specs we promote. Just because someone pushed out a draft spec with "CSS" (or "HTML" or "Web") in the name, and shipped a prefixed implementation, doesn't mean that that spec is, or should be, part of the open Web. People need to ask: is this feature good for the Web? is there a thorough draft spec that doesn't require reverse-engineering of an existing implementation? are there multiple implementations? is the spec actively edited with feedback from multiple vendors, and Web authors, being taken into account?

It's a challenging time. It's an exciting time. Despite the threats I mentioned, it's great to see massive investment in improvements in open Web technology. It's great to see Microsoft moving away from Silverlight towards a standards-based platform. We've won some battles, but the war for open Web standards is not over and we need to keep fighting it, on the right fronts.

Wednesday, 28 September 2011

Graphics API Design

For several years Gecko used a C++ wrapper around cairo as its cross-platform rendering API. We eventually found that the cairo API led to some inevitable performance problems and designed a new API, Azure, to address these problems. Joe Drew and Bas Schouten have already discussed Azure a bit. I want to mention some of the specific lessons we learned about graphics API design.

Stateful Contexts

Cairo uses a "stateful context" model much like Postscript or Quartz 2D. To draw content, you make individual API calls to set up various bits of state in a context object, followed by another API call to actually perform drawing. For example, to stroke a shape with a dashed line, you would typically set the color, set the line width, set the dashing style, start a new path, emit path segments, and finally draw the stroke --- all as separate API calls. In cairo, even drawing an image requires the caller to set the source surface, emit a rectangle path, and fill --- at least 6 API calls.

This design has some advantages. In typical applications you can set up some state (current color etc) and reuse it across many drawing operations. The API can consist of many logically independent operations with small numbers of parameters, instead of a few giant operations taking dozens of parameters. If your application never needs to use non-default values of certain drawing parameters, you can completely ignore them.

Downsides

Unfortunately we found that with GPU-accelerated rendering on intensive HTML <canvas> benchmarks, we were doing so many drawing operations per second that the overhead of making many cairo API calls was becoming a significant drag on performance. (In some configurations we can do over 100,000 image draws per second; a single malloc per draw call becomes significant in the profile.) We could have improved that situation by adding new cairo APIs matching the <canvas> 2D APIs, e.g. a cairo_draw_image API. However, other problems with stateful contexts and cairo's implementation led us down a different path.

Cairo collects state in its cairo_t context objects --- in its "gstate" layer --- and each drawing call passes all relevant state down to a "surface backend" object to do the drawing. Essentially it maps its "stateful context" API to a "stateless" backend API. This mapping adds intrinsic overhead --- floating point to fixed-point coordinate conversion, sometimes memory allocation to store path data, and generally the overhead of storing and retrieving state. This is an especially big deal when the underlying platform API we're wrapping with cairo is itself a stateful API, such as Quartz 2D; in that case each stateless backend drawing call performs several platform API calls to reset all the state every time we draw. Cairo forces us to go from stateful to stateless and back to stateful as we move down the rendering stack.

HTML 2D <canvas> is a "stateful context" API much like cairo's, so cairo was actually a pretty good fit for <canvas>. In the rest of the browser we use graphics APIs somewhat differently. When rendering CSS, we typically have to reset all the context state every time we draw. For example, every time we draw a border we have to set the current color to the CSS border color, set the current line width to the CSS border width, set the line dashing style to the CSS border style, etc. Effectively we treat our graphics API as stateless. Cairo's accumulation of state in its context before calling into the stateless backend to draw is just unnecessary overhead.

Another consideration is that the state of a 2D <canvas> can't be represented directly in cairo; for example, cairo has no concept of global alpha or shadows. Therefore our <canvas> implementation has to maintain its own state tracking in parallel with cairo's internal state tracking. Given that, tracking all state in our <canvas> code (above the graphics API) is not much extra work.

Given all this, it made sense to make our cross-platform drawing API stateless, so we created Azure.

Azure

Almost all the operations on an Azure DrawTarget (the nearest equivalent to a drawing context) do actual drawing and take most relevant state as parameters. The only state carried by a DrawTarget is the destination surface itself (of course) plus a current transform and a current clip stack. We let the transform and clip state remain in the DrawTarget because those are the only pieces of state not constantly reset by CSS rendering. Our CSS rendering needs to always render under some given transform and clip, and we don't want all our rendering code to have to pass those around everywhere.

So far we're only using Azure to for 2D <canvas> drawing. For optimal canvas performance we ensure that every canvas operation, including global alpha and shadows, is directly supported in Azure. The only Azure backend shipping right now is the Direct2D backend. Direct2D is mostly stateless so it's a good fit for Azure, although we've designed Azure carefully so that stateful backends like cairo or Quartz 2D will work pretty well too. (Mapping from stateless down to stateful is pretty easy and can be optimized to avoid resetting state that's constant from draw to draw.) <canvas> performance with Azure/Direct2D is much improved.

We have given up some of the advantages of stateful context APIs mentioned above --- Azure's probably a little less convenient to use than cairo. To some extent that just doesn't matter; writing more verbose API calls to get a significant performance improvement is completely worthwhile for us. But we can mitigate the worst aspects of a parameter-heavy API anyway. We group parameters together into structs like "StrokeOptions" and "FillOptions", and use C++ to assign sensible default values to the fields; this means that callers can continue to ignore features where they only want the defaults, and callers can reuse state values across drawing calls.

Insights

One of the key insights here is that Gecko is a framework, not an application, and APIs suitable for applications to use directly may be less suitable for frameworks. When you're writing drawing code for an application you know the content you're going to draw and you can write code that takes advantage of "state locality" --- e.g. setting the current color and then drawing several shapes with that color. For most of what Gecko draws, we can't do that because we don't statically know what a Web page will draw. Also, because we're a framework, we have fewer calls to graphics APIs in our code than an application might. For example, we have only one piece of code that draws all CSS gradients; we might have less than half a dozen places in our entire codebase that actually create gradient patterns. Therefore making the API calls shorter or more convenient offers us very little value.

Aside People rightly ask why we didn't just modify cairo to fit our needs better. Of course we considered it, since we've done a lot of work to improve cairo for our needs over the years. We'd need to make massive architectural and API changes to cairo, which would probably be at least as much work as writing Azure from scratch --- quite likely more work given that cairo promises a stable API, so we'd have to maintain our stateless API alongside cairo's stateful one. Cairo's API stability guarantee often gets in the way of us making improvements, and it has little use to us since we ship our own copy of cairo on most platforms; getting away from the cairo API will be helpful in that respect.

I don't think this should be considered a failure of cairo's design. Cairo was originally designed to be used directly by applications, over a very low-level backend (XRender), with much lower performance targets on very different workloads. Using it in a framework, wrapping very capable and high-level platform APIs, and expecting 100K image draws per second is far beyond those design parameters.

Thursday, 22 September 2011

Risks Of Exposing Web Page Pixel Data To Web Applications

Some Web applications require the pixel data of Web pages to be exposed to Web applications, e.g.

  • A 3D bookreader application that draws arbitrary Web pages into WebGL textures (from there, the pixel data of the pages can be extracted directly or using timing attacks)
  • An interactive virtual environment that wants to render Web content onto 2D surfaces in the environment via WebGL
  • A visual effect using 2D canvas that wants to draw a Web page into the canvas and cut it up into shards that move around under animation
  • A screensharing application that sends the contents of Web pages over a video stream to help with support issues
  • A bug-reporting tool that wants to grab the rendering of a Web page to capture in a bug report

There are some pretty big security implications here. The biggest problem is cross-origin information leakage. For example, an attack page could load a page from another origin in an IFRAME; rendering the attack page content will then capture the other origin's content and allow it to be returned to the attack server. The same goes for cross-origin images and other resources. To close this hole, we'd need to track the origins of data during painting and detect and/or block the painting of cross-origin data, which would add considerable complexity to the paint path and probably be error-prone.

Another problem is <input type="file">. In many implementations the file input control renders the complete path of the file, or at least more than just the file name; capturing the pixel data of the page would leak that information which we intentionally conceal from Web pages.

Theme drawing is another problem. By capturing the rendering of themed form controls, a page could determine what system theme the user is using. This isn't a big problem by itself but it would contribute to fingerprinting.

Update Commenters point out another problem I forgot to mention: CSS history sniffing. Access to rendered pixel data makes it easy to determine the visitedness of a link.

Any solution to the use-cases listed above needs to prevent the above problems. In Gecko we have the drawWindow API which lets you render the contents of arbitrary windows into a canvas, addressing all of the above use-cases, but it's only available to privileged content such as Firefox extensions. We've considered making it available to untrusted apps in some form, but the above issues have prevented that.

However, a little-known fact is that in Gecko we do have a limited way to render HTML content to a 2D or 3D canvas with access to the pixel data of the results! You can construct an SVG image containing a <foreignobject> containing arbitrary HTML, draw it to the canvas, and (if the image is same-origin with the page) call getImageData on the results. This approach avoids the problems above because the content of SVG images is extremely restricted in Gecko. The biggest restriction is that in Gecko, SVG images can only reference resources in the same document or loaded from data: URIs! Basically an SVG image has to be stand-alone. This prevents any kind of cross-origin attack. Issues with file controls or other interactive features are prevented because it's impossible for users to direct events to or otherwise interact with the contents of SVG images. Script can't run in SVG images, nor can script access the DOM of SVG images. Theme drawing will be disabled in SVG images.

Unfortunately this limited solution doesn't address most of the use-cases above. I don't have any good answers for those; this is a really hard problem.

Monday, 19 September 2011

Doing The Same Thing Over And Over Again And Expecting Different Results

The quote "The definition of insanity is doing the same thing over and over again and expecting different results" is popular, being widely attributed to Einstein (probably incorrectly). Unfortunately it's often used to support a common and dangerous fallacy: that if you're trying something and it's not working, you must not be using the best strategy and you must try something else.

It's easy to see the fallacy using a thought experiment. Imagine you must regularly play a lottery which offers red tickets and black tickets, and red tickets have twice the probability of winning compared to black tickets. So you choose red tickets, but you don't win. Would Einstein tell you to start choosing black tickets instead? Hopefully not!

When the best available strategy has a low probability of success, sticking to that strategy requires discipline, wisdom and courage. This is one of the themes of The Lord Of The Rings :-). You see people fail by panicking and flailing, abandoning their best hope to try something, anything, and essentially self-destructing.

Of course, one must also avoid stubbornly sticking to a strategy which is not the best available.

Overall it's probably best not to use aphorisms in serious discussions :-).

Saturday, 10 September 2011

France Vs Japan

Our family went to see the France vs Japan Rugby World Cup game at North Harbour Stadium tonight. I knew it would be fun but I didn't expect the game to be so exciting! France is ranked far above Japan (Japan have never come close to beating France, or any other team in their league) but with 20 minutes to go France was only leading 25-21 and the Japanese were attacking with gusto with the crowd roaring them on. We really believed they could win --- it was fantastic.

In the end, France ran away with it by scoring a few easy tries in the last ten minutes. It was, nevertheless, a wonderful game. It was great to see a lot of Japanese and French supporters, as well as a lot of locals supporting one or another of the teams. I got the feeling there were more supporters of Japan, but the French supporters were more vocal. Having La Marseillaise as the anthem certainly helps!

The atmosphere at the game, back in town on the way home, and also at Mt Eden last night where we watched the opening night fireworks, has been excellent. There's a strong festive mood in the city. Even on our bus home, there were random strangers wrapped in French and Japanese regalia shaking hands with each other. It's all a bit surreal and wonderful.

Friday, 9 September 2011

So It Begins

The Rugby World Cup starts today. The weather's great and it should be a wonderful six weeks. As previously noted, I have pledged to enjoy the entire event even if the All Blacks don't win --- and I don't think they will. The knockout format means that even if they would beat any given opponent 75% of the time, they have only a 42% chance of winning all three knockout rounds. People seem to have a hard time distinguishing "is the most likely team to win" vs "is probably going to win".

I think New Zealanders are coming to terms with it though. In 2007 the media was far more full of outrage and "doom and gloom" than the actual people I know --- heck, there was strong public support to retain the coaching staff, which the media found unfathomable. "Rugby-mad Kiwis despair" is too good a story, especially for overseas audiences I guess.

Still, I hope that sermons on "being content in all circumstances" are forthcoming from pulpits across the country over the next month :-).

I'll be at the stadium watching France vs Japan tomorrow, and then I'm off to California for a Mozilla meeting. Fun times.

Monday, 29 August 2011

Cost Of Living

Does it make sense to factor in the cost of living when determining the salary of remote employees?

Companies often do this. I don't know exactly why --- perhaps they don't either --- but I can think of a couple of plausible reasons. It seems fair to try to normalize the "real compensation" across employees. It seems likely that salaries are already generally higher where the cost of living is higher, so new offers need to be higher to be competitive.

On the other hand, supposing you have a roving developer who doesn't work in an office, it doesn't make sense to me to pay them five times as much because they choose to live in London rather than Bhopal. If they're going to be equally effective in each place, their location is their business and if they want to live somewhere cheap and live like a king, why should they be penalized for that?

There may be an interesting economic effect where companies that allow employees to be mobile and adjust pay for cost-of-living are effectively subsidizing the economies of expensive locations. I wonder if anyone's looked into that.

I think the answer to my initial question must be complicated. It should depend on how the location matters to the employer. If location doesn't matter at all to the employer, it seems to me cost-of-living should not be taken into account directly. It may have an indirect effect by raising the level an offer needs to be competitive.

Disclaimer --- this post is my opinion and is unrelated to Mozilla's policies.

Thoughts On Management

I've been managing people for a while now. It's not my favourite job --- technology is my passion --- but it has to be done, and it has to be done well, so I'm happy to do it while Mozilla needs me to. We recently did a round of performance reviews and I've been reflecting on what I've learned.

I'm a slack manager. Most people on my team probably don't get more than fifteen minutes a week interacting with me "as a manager". I mostly get away with it for two reasons. Most people on my team are awesome and need very little supervision; they're self-motivated, capable, and function well when left alone to "get on with it". Also, I interact with them a lot in non-manager ways: code reviews, Bugzilla traffic, IRC, and other project-level communication that I'd have even if I wasn't a manager. I admit that the real reason I'm a slack manager is to give me time to work on code.

Another strange thing about this team is that we don't have regular group meetings. In my view, meetings are a tax on productivity and should only be held when necessary, and I think so far the projects executed by my team have not required whole-group coordination. There's some value in having people know what other people are doing, but I'm not convinced it justifies the cost yet.

I haven't been having one-on-one meetings with the people in the Auckland office because we were able to talk about what everyone's working on over lunch every day. However it became clear that actual one-on-one meetings would help bring out issues, so we've started doing that.

One thing I've mostly ignored is "career development". Two options that seem to be common outside Mozilla are training and promoting people into new roles. For most platform developers, though, I don't know what training would be effective. There isn't a manual for browser development, and if there was a lot of it would be written by Mozilla people; we pass that knowledge on through mentoring, code reviews, and shared experience. Once they've mastered basics like knowledge of algorithms, logic and thinking in terms of invariants (which most people have learned before they get to Mozilla), the only ways I know to improve their general development skills are experience and imposing good tools or processes (e.g. testing).

During the last round of performance reviews I tackled the career question by asking people whether there was anything they would like to do at Mozilla that they weren't currently doing. Almost without exception they answered that they love the technical work they do and would be happy to continue doing it indefinitely. I think this would seem strange to a lot of people (I think there's a widely held view that if you're in the same role for too long, there must be stagnation) but I understand it very well, because I feel the same way myself! I enjoy the fresh technical challenges that keep coming along --- new bugs, new features, often whole new technical areas to explore (typography, graphics, video, RTC...). People who keep working on Gecko, who get better and better at it with deeper and deeper knowledge of the Web and our code, keep increasing their value to Mozilla. So I definitely won't ever push people out of their current role for the sake of change.

However, we do need to make sure that we don't miss any opportunities to find roles that might make people even happier or more productive. Just asking them isn't necessarily effective since there may be opportunities they're not aware of or incorrectly assume they won't enjoy. I need to be a bit more proactive in this area.

Friday, 19 August 2011

New Mozilla Auckland Office Is Live!

On Wednesday we moved into the new Mozilla Auckland office! For about a week the only thing holding us back was getting Internet access hooked up. Once that happened around 4pm Wednesday, we lost no time in shifting over our most important gear (laptops and monitors of course). Carrying all that gear through the streets of Newmarket, I'm glad we didn't start a riot. We capped off an exciting day with a "pizza and Settlers" evening in the office.

The new office is very close to the old office. It's on the 7th floor of 5 Short St, Newmarket. This is the top of one of the tallest buildings in Newmarket so we have great views! We can see the Hauraki Gulf, Rangitoto Island, Motutapu Island, Motuihe Island, Waiheke Island and Browns Island, and Coromandel (including Moehau Range and Castle Rock); Bastion Point, Tamaki Drive, Hobson Bay, Kepa Bush, Thomas Bloodworth Park and Outhwaite Park; the Hunua Ranges; Mt Wellington, Mt Hobson, Mt Eden, One Tree Hill and trees of the Domain; Lumsden Green, Broadway (all the way to Parnell), Khyber Pass, Short St, Kingdon St, Shore Rd and the railway; Auckland Hospital, the Sky Tower, Auckland Grammar School, and the old Dominion Brewery; and of course large tracts of Remuera, Mt Eden and Newmarket. OK, you have to squint to see some of those :-). You can see six volcanic cones in all (not counting the Moehaus which are a much older volcanic range). The downside is that now I have to resist staring out the window wishing I was outdoors :-).

We have great new facilities: more desks, work tables, an eating area, a much better kitchenette, a decent-size meeting room and two smaller meeting rooms. The network is already better than the old office's and will get better still when it's upgraded next month. With these facilities we will be able to do more --- hire more people, host more visitors, host meetings of 30 people or more, and hopefully get more work done!

To all friends of Mozilla in Auckland or passing through: please stop by and check it out! At some point --- hopefully soon --- I want to organize an official "open day" event for the public where we can talk about our work and Mozilla.

Enormous thanks to the Mozilla team that put all this together, especially Karl Tomlinson who has been very busy handling the local arrangements! The Stack design team has also done a great job.






Tuesday, 16 August 2011

Securing Full-Screen

Some Web apps would benefit from displaying full-screen, without any non-app content visible. Obvious examples are watching video and playing immersive games. A while ago I proposed a Web API to enable this, and Chris Pearce is making good progress implementing a version of it in Firefox. A slightly different version of it is in Safari now too.

Our biggest issue right now is how to make it secure. The perceived threat is a malicious page going full-screen and then displaying something that looks like the content of another site, say the user's bank, complete with false URL bar --- or the content of a native application or the operating system, for that matter.

There are a few things we can do to make it harder for a malicious application to go full-screen. We can ensure that full-screen requests are only honoured from scripts triggered by user input (mouse clicks and keystrokes), much like requests to open popup windows. We can make sure that when going full-screen, we display a clear message describing how to leave full-screen --- like Flash does, but hopefully better. Then if a malicious page goes full-screen when the user didn't want to, the user will probably exit full-screen immediately.

A harder case is when the user intentionally goes full-screen to watch a video or play a game, but the application later tries to abuse full-screen status by spoofing another page or application. Most spoofing attacks require user input that the browser can detect, so for full-screen video and other applications that don't require much input, we could show a real URL bar while the user is typing input, so the user knows the true domain. However, many applications (games) want to be full-screen while receiving the full range of user input. It's unclear how we could distinguish such applications from a spoofing attack. The risks here already exist to some extent, if an application can persuade users to manually go full-screen, for example by pressing F11 in Firefox.

This feature seems particularly challenging to design because browser security often depends on the assumption that the user can visually identify the domain of the current page at all times, and full-screen violates that by definition. (However, mobile browsers seem to violate that assumption by hiding browser UI most of the time; I wonder how browser security people feel about that.)

This problem would be a good one for security researchers to study and solve! I think the world would be a better place if security research focused more on constructive solutions to problems like this one :-). Anyway, feedback and ideas welcome from anyone, security researchers or not.

Tuesday, 9 August 2011

Eden Park

On Saturday night our family went to Eden Park to watch the All Blacks play Australia. We are fortunate to live within walking distance of the stadium (OK, it's a longish walk --- half an hour, for me). We won't be watching any World Cup games at Eden Park so this was our only chance to see it at its full size with the temporary stands. In fact, we had seats at the top of the temporary West Stand. The seating and the view was really excellent. I had worried that it would be cold since wind and rain were predicted, but the weather turned out fine and we had a wonderful time. From that height you're far away from the action but you have a really good angle to watch the entire field. You can watch how the teams position themselves and organize their defence across the entire field, something you normally can't do watching on TV. That was especially relevant for this game since both these teams like to use all the space on the field.

The crowd was big but everything seemed to go very smoothly inside and outside the stadium. Some in the row in front of us were rowdy and liked to call out "Bullshit, ref!" at every opportunity ... something my kids are not used to hearing. At the beginning they noticed this and apologized, but after a few more drinks they forgot to apologize anymore :-). But hey, it's a rugby game, so no surprise :-).

Of course the mood of the crowd was helped by the All Blacks winning comfortably. It's quite a remarkable record that they haven't lost at Eden Park since 1994 (and not against Australia there since 1986).

I haven't seen in the media any pictures of the revamped stadium from the outside. It actually looks quite cool, a little bit like the Olympic stadium in Beijing.

One other nice thing is that our route to Eden Park takes us through Dominion Road, which has a large collection of good Chinese restaurants. Our regular place there is Love A Duck but it was full; however, lots of other restaurants in the area had space, even just an hour and a half before the game. We had a fine dinner at some Shanghainese place; Shanghai Dimsum might be its English name, or that might just be descriptive text.



Monday, 8 August 2011

South Island Holiday

Two weeks ago we had a family holiday in the South Island. The idea was just to head down to the mountains and relax in the wintry environment, without skiing or snowboarding because I'm completely useless at those.

We flew to Queenstown and stayed a few nights at the Novotel Lakeside hotel. The room was small but the location was great, right next to Lake Whakatipu and the Queenstown Gardens park. The first full day we walked the Twelve Mile Creek loop track with a side trip to Lake Dispute, about four hours of walking altogether up in the hills north of the lake along the road to Glenorchy. The weather was great and the views over the lake to Walter Peak etc were stunning, as usual. The upper part of the track was above the snow line. Below the snow line, water running in the forest had frozen into icicles and other fantastic ice formations.

On Sunday a "polar blast" blanketed much of the country in snow. It snowed pretty hard in Queenstown at times. We spent some time inside at the Caddyshack mini-golf park, which was well worth it. We started heading up Queenstown Hill but turned back because the snow was just falling too hard, so we spent the rest of the day manipulating snow in the Queenstown Gardens.

The next day we had to drive to Twizel so we got the chains on the rental car and headed out on still-quite-snowy roads. But about halfway to Cromwell we took them off again because ironically it had failed to snow in the Mackenzie Basin, in the heart of the Southern Alps, even while it was snowing down to sea level elsewhere! So we had an uneventful drive to Twizel. We had a nice stop at Lindis Pass, which was beautiful.

We went to Mt Cook village where it was very definitely snowing, and windy and cold too. We managed to make our way to the Kea Point lookout over the frozen lake at the end of Mueller Glacier, but it was hard going for an hour into the teeth of a high wind. All the other tracks in the area were closed due to the conditions, so I was impressed to see three people tramp out of the wilderness; they must have been in the back country when the weather hit.

We visited Lake Tekapo. The big plan that day was to get up Mt John. The direct track (~1 hour) was closed due to "icy conditions" so we set off on the long route, north alongside the lake and then up and south along the ridge to the summit (~2.5 hours). The ridge was exposed to the wind so we were a bit cold by the time we got there, although the views up there made it well worth it. We got up there around 4pm with a bit over an hour of daylight left. My plan had been to run down and bring the car back to the top to pick up the rest of the family, but on our way up they'd closed the road due to high winds. (I'm not sure why; the road did not look precipitous.) So we were up there having hot chocolate at the cafe wondering how we were going to get down, which felt odd. We met a guy who'd come up the direct route, and he told us the ice wasn't really a problem and there were no dropoffs, so we decided to go down that way. We put on our waterproof pants and whenever it got too icy to walk, we just did a glissade. It was actually a lot of fun.

We drove to Dunedin and visited the Cadbury factory. Later we drove out along the Otago Peninsula --- lovely. We went on the Taieri Gorge Railway --- a lot more impressive. The Taieri river looks good for rafting.

This was the first family road trip with a smartphone. Having maps, GPS and Web browsing were very useful. On the other hand, the temptations to surf and read email were hard to resist, so it's a mixed blessing. I'm undecided whether to bring it again.






Blog Update

After years of service from mozillazine.org (thanks Jason!), I moved my blog to robert.ocallahan.org. Having it hosted under my own domain name makes it more future-proof, but since I don't want to run my own server, I'm using Blogger to host it. Unlike mozillazine.org, Blogger doesn't let you upload any old content, but I've worked around that by putting images and other content on www.ocallahan.org hosted by Google Sites. Getting this setup working and my content migrated was a bit of a pain but it seems to be working well now. I worked with Jason to put in redirects from all the old articles to their new locations.

Blogger seems pretty good. Unlike Wordpress, they give me free hosting under my own domain name. The design options are very flexible, and the stats pages are nice. We'll see how it goes, but so far, so good.

Friday, 5 August 2011

Books

I've read a few books lately...

I read Pirate Freedom by Gene Wolfe; it's definitely the best Catholic pirate time-travel fantasy I've ever read. Seriously, it's quite good. Most Wolfe novels I've read have felt a little too obscurantist, like I'm taking a fiendishly difficult reading comprehension test, but this one's just good.

I read a few classic Philip Dick novels: The Man In The High Castle, The Three Sigmata Of Palmer Eldritch and Do Androids Dream Of Electric Sheep?. They're all good, although The Man In The High Castle seemed the best; his depiction of Japanese influence on postwar California (in a future where the Allies lost World War 2) is intriguing. Although I liked the novels, they all gave me an odd mental aftertaste, as if my own grip on reality had been subtly undermined. Perhaps that's what taking drugs is like. It would certainly explain --- or be explained by --- Dick's lifestyle.

I read The Runes Of The Earth, Stephen Donaldson's return to the Thomas Convenant saga that I read the first six books of in the early 90s. It's not bad, although like his previous books I find it grossly overwritten. I'll probably read on, although (spoilers!) I'm rather nervous, since the introduction of time travel into a long-running storyline is usually a sign that you've jumped the shark.

Now I'm working through Harry Potter. Partly it's because I want to be ahead of my kids reading it, but it's not bad at all. It definitely picked up in the third book.

I've bought The Scar by China Mieville. I liked Perdido Street Station a lot, despite the deus ex machina ending, and Iron Council was quite good even though it didn't make any sense. Mieville's world-building is captivating. Like Dick, though, his books leave me with a mental aftertaste. Mieville's world seems wrong in a way that no other fantasy world does to me. There seems to be a fundamental disorder; a feeling that there are no laws or limits or organizing principles, or those that exist are corrupt somehow. I can't explain it very well. It tastes of Hell, maybe.

Rugby Fandom

Many people in New Zealand feel down when the All Blacks lose, especially in the World Cup. It seems silly to me, although I'm sometimes one of them; sports simply aren't that important. Along with that we have an unhealthy tendency to excoriate and repudiate losing teams, which apart from being unfair is also irrational because in most close games, factors outside the team's control could easily have determined the result.

Therefore, I have a plan. Starting tomorrow night (when I will be watching NZ vs Australia at Eden Park) I intend to enjoy watching the game whether the All Blacks win or lose, and encourage others to do the same. We'll see if I make it through the World Cup :-).

Friday, 1 July 2011

Permissions For Web Applications

Summary The permissions model we have evolved for Web applications so far is mostly on the right track; introducing Android-like bundling of permissions with "up front" permission grants would be a mistake.

Traditionally Web applications are untrusted, treated as potentially malicious, and hence "sandboxed" to deny them the ability to affect the user's system or access user data. As we add capabilities to the Web platform, we sometimes encounter situations where legitimate applications want functionality that must be denied to malicious applications. A natural solution is to ask the user whether such requests should be permitted. Long ago we learned that modal requests --- interrupting the application with a warning, and forcing the user to OK/Cancel before proceeding any further --- was ineffective for security, because users quickly learn to OK such warnings without even reading them. An alternative approach is "passive confirmation UI": for example, UI appears requesting a permission, but the user can ignore it and continue using the application, so users who don't read the message are less likely to grant permission by default. Firefox's geolocation permission UI is a good example of this.

The Permissions Bundle Model


That approach seems OK but as apps use more features that require permissions, there is a desire to not bombard users with lots of permission prompts, even passive ones. A number of people have proposed moving to an Android-like model. On stock Android systems, when the user starts using an application for the first time, they are presented with a list of permission requests; the user either grants them all or not. If permissions are denied, the app does not run, otherwise it runs with no further prompts.

Personally I think that is a terrible model. There are two main problems:


  • There is no way for a user to know why an app needs the permissions. For example, Google Music requests, among others, "read phone state and identity", "display system-level alerts", "modify global system settings", and "send sticky broadcast" (whatever that is!). Why should it need to do those things just to play music? I have no idea. Remember that I haven't even been able to run the application yet. As a user, I just don't have the information I need to know whether the request is reasonable or not.
  • If I don't grant the permissions, I can't use the app. Of course I want to use the app, since I downloaded it, so of course I am going to grant the permissions. It's the old OK/Cancel modal dialog all over again.

Someone should do a study where they promote a simple game app which requests absurdly overbroad permissions, and see how many users download the game but reject it at the permissions screen. I'll be amazed if it's more than tiny fraction of users. If so, the permissions screen should be removed, because it provides no security and interferes with usability.

Alternatives


The most important thing we can do is to remove the need to ask for permission to use a feature in the first place. We can often do this by carefully designing our APIs.

Implicit Permission Grants


Good old <input type="file"> accidentally invented an excellent permissions model for file I/O. The app asks the user to choose a file to load, and in the process the user implicitly grants the app permission to read that file! The same sort of approach works in other situations. For example, recently someone proposed that browsers offer permission UI to give an app access to the user's telephone number. Instead, <input type="tel"> could pop up a on-screen keyboard optimized for phone number entry, with access to the user's contacts database and perhaps a "my number" button. Then the application UI would contain "Enter your phone number: []" and the user would easily be able to do so --- or enter another number if they don't want to reveal their own.

Another example of this would be registering Web apps as content-type handlers. Instead of asking for permission to do that, we should simply let any Web app register as any content-type handler. However, when the user downloads content that could be handled by a Web app, then we would prompt the user to choose which app to use, highlighting their last-chosen application and secondarily any newly registered app. (Android does something like this.)

Ask Forgiveness, Not Permission


In many cases the actions of a malicious application can be easily detected and safely undone. For example, the ability to play sound through nearby Bluetooth devices could be abused to annoy the user, but the user will easily be able to detect the problem and the system should offer convenient UI to identify the abusive app and silence it --- per-application volume control. There is no need for a priori permission requests in such cases.

Remember This Decision


I'll assume without further discussion that the system can automatically grant permission to do whatever the app was permitted to do before, unless the user chooses otherwise.

Permissions In Context


Once we've reduced permission requests to the bare minimum, we still have the question of whether to ask for permissions "up front" or while the application is running. I firmly believe it's best for applications to request permission in the context of the user action that requires that permission. For example, it's easy for me to understand that when I click on the "show my location" button in Google Maps, it's going to want my location.

This leaves us with a (smaller) version of the "bombarded by prompts" problem that we started with. Personally I think we should live with it. IMHO the alternative of "bundled permissions" doesn't solve the problem, it simply works around it by teaching users to grant all permissions. "Bombarded by prompts", viewed in a better light, is a process of making informed permission decisions one by one as the user becomes familiar with the app.

Greedy Applications


One wrinkle is that lazy app developers can turn the "permissions in context" model back into the "bundled permissions" model by activating APIs up-front and refusing to let the application proceed until all requests are granted. My hope is that if most apps don't behave that way, users will develop higher expectations and be distrustful of lazy apps.



Saturday, 25 June 2011

Auckland Web Meetup Recap

Yesterday night I went to the Auckland Web Meetup at the Telecom building. Nice building! Apparently there were 180 people there.

The first talk was about a task management Web app that some local people are building; that seems like a hard market to crack.

The second talk was about optimizing CSS, HTML and JS for page load times. It was interesting to see what Web developers are telling each other. Most of the suggestions were good, but there was some advice to put external script loads as late in the HTML page as possible. That probably made sense in older engines -- it let other resources start loading without having to wait for scripts to load. However, modern engines can speculatively parse past a <script src> element before it finishes loading, so these days it's probably best to put external script loads as early in the HTML page as possible.

Then there was a break and a bit of a scrum for pizza :-).

Then it was my turn. My plan was to show a few modern HTML games, explain the technologies behind them, discuss the current browser support landscape and related issues, and then talk about missing features and future APIs that may address those features. I didn't have time to prepare any slides; I only had notes listing topics I wanted to cover. I think it went pretty well anyway.

The demo sites were Pirates Love Daisies, the 2d-canvas version of Angry Birds (the "HD" WebGL version doesn't seem to work in Firefox trunk; I think it may have been broken by the same-origin texture loading restriction), and Quake 3 WebGL. I had no network and one thing that became immediately obvious was that these games don't work well offline :-).

Pirates and Angry Birds use 2D <canvas>, which benefits from acceleration in Firefox and IE9, and there's not much else to say about that for now.

Pirates and Quake 3 use HTML <audio>, and both provide Ogg files. Angry Birds uses MP3 played by Flash, for various reasons but probably partly because Chrome's <audio> support has some problems. I talked a bit about the codec issues, including out-loud wondering why everyone can't at least ship support for Ogg Vorbis, since it's a good format that is very widely used (including even by Microsoft, in some products), without encountering patent problems. JSMad is an interesting approach: decode audio in JS. (This was a good time to talk about how JS engines are so much faster at manipulating binary data now, especially Spidermonkey and JM/TM.) The equivalent of JSMad for Vorbis would be an interesting way to support Vorbis on uncooperative browsers. When MP3 decoding comes out of patent coverage (Real Soon Now), we should implement in Firefox in a hurry, and that will also help developers by providing a decent (albeit not as good as Vorbis) baseline audio codec.

I went on to talk about WebGL and the issues around it. Microsoft seems to be resisting it for now, ostensibly due to security issues but I take that with a grain of salt due to Silverlight supporting a very similar 3D API with the exact same issues. I think WebGL will win and Microsoft once again will have to follow the lead of the other Web browsers. Nigel Parker from Microsoft did point out that Apple are backing off making WebGL fully available on iOS, although why is anybody's guess.

An interesting feature that I didn't know about until recently is that Pirates supports touch events; apparently because IE9 supports touch events on Windows tablets, which I also didn't know until Nigel pointed it out last night. Nigel had been such a good sport while I carped about Microsoft that I had to give Microsoft enthusiastic credit for supporting touch events already.

I went on to talk about future APIs: audio effects APIs, basically repeating what I said on my blog recently; full-screen APIs; and communications APIs --- Websockets and upcoming RTC work.

During this talk I grumbled to the audience about the attention browser vendors get for pushing out half-baked new APIs compared to the attention they get for doing a really good implementation of an API. Web developers tend to be a lot more excited about the former than the latter, which encourages browser developers to do more of the former at the expense of the latter. For the sake of the Web we should all try to change that.

At the end of the talk I asked the audience to give feedback on features they need that we didn't talk about. The first request was "text-overflow:ellipsis", so I was able to answer "landed yesterday; next?" :-). (I'll write more about text-overflow later; it's a good example of the issue in the previous paragraph.) Another request was DRM for Web video. That's a really difficult topic and I didn't have much to say. Someone asked for Mobile Firefox on the iPhone, which is another difficult topic and outside the scope of my talk, but I did give me a chance to rant a little bit :-). OK, to be fair the whole talk was a bit ranty :-).

Anyway, it was a ton of fun and I hope the audience had as much fun as I did, and hopefully learned something too :-).



Tuesday, 21 June 2011

Auckland Web Meetup

This Thursday night (June 23) we'll be giving a talk at the Auckland Web Meetup.

Web standards and browser implementations are rapidly evolving features that enable sophisticated games to be written as standards-based Web apps. These features include fast JS engines, GPU-accelerated 2D and 3D graphics (canvas and WebGL), video/audio, WebSockets, and emerging proposed standards for advanced audio effects, fullscreen APIs, and peer-to-peer real-time communication. We'll show some demos and benchmarks, explain about the technology behind them, describe the browser support landscape, and discuss what happens next. We want feedback from developers to help guide our priorities and technical decisions.

Indeed, as usual I'll be going to as much to get feedback from Web developers as to spread our own message.



Monday, 20 June 2011

CS Unplugged

I've been concerned about the teaching of computing in schools. At my children's primary school, they get a little bit of instruction in using a computer. There are even optional tests, with questions like "what does this icon do?". Unfortunately this seems to convey that a career in computing is all about using Microsoft Word, so anyone with a brain would dismiss such a career from an early age. Anecdotal evidence suggests that this does happen.

Early this year I saw Tim Bell from the University of Canterbury demo a few of the lessons from his site CS Unplugged, techniques for teaching computer science concepts to kids --- without using computers, which he views as often being just a distraction from learning these concepts, and I agree. I was keen to try this myself at my children's school, and last Friday I finally did, in my older child's class, with the help of my wife and another computer science-y couple who also have a child in that class.

We spent an hour and a quarter covering binary numbers, parity-based error correction, and a short discussion of searches using sorting and indexing. (For the latter I had the kids look words up in books with indexes, and discussed how Web searches work similarly. I had also manufactured an unsorted phone book and we compared lookup difficulty with a regular phone book, to try to convey the idea that you can make operations faster by organizing your data the right way.)

It went well. In this class of 9/10-year-olds, all the students could do the activities, Many found them easy but some needed help. This suggests to me that the activities were about the right level. Everyone seemed to enjoy it. I'd definitely do it again!



Thursday, 16 June 2011

Media Processing

A number of proposed features with overlapping spec and implementation requirements are popping up:


  • Advanced audio APIs that allow complex mixing and effects processing (e.g. Mozilla's AudioData, Chrome's AudioNode)
  • Synchronization of multiple HTML media elements (e.g. proposed HTML MediaController)
  • Capture and recording of local audio and video input (e.g. proposed HTML Streams)
  • Peer-to-peer streaming of audio and video streams (e.g. proposed WebRTC and HTML Streams)

At the API level, I want to integrate effects processing with capturing, recording and streaming by building an effects API for HTML Streams. There is a proposal on the Mozilla wiki --- it needs work.

At the implementation level, all these features should be built on a common foundation to ensure that current and future integration requirements can be met; trying to bridge separate implementation models while maintaining global synchronization, high throughput and low latency seems very difficult. My current project is to build that foundation. It's challenging but fun. I've made it my top priority partly because it could block work on the above features, and also because there is an urgent need for practical comparisons of a Stream-based processing API against less-integrated APIs.

I've found that to focus I really need to disconnect from the Internet. So I'm getting into the habit of working online in the first half of the day and then going offline around 2pm or 3pm (NZST) for the rest of the day. It's working. My apologies to anyone trying to find me online during that time.

My current plan is to build the foundation and just enough of the DOM API to support Worker-based audio synthesis and capture of stream graph output; then I'll be able to write tests for the framework. Then we'll hook up actual audio and video output, make the HTML media element implementation use the framework, and flesh out the new features.



Wednesday, 15 June 2011

Some Advice

Never, ever trust anyone who writes "PhD" after their name. Seriously.

To clarify: I have nothing against PhDs. I know lots of great people with PhDs. None of them write "PhD" after their name. People who write "PhD" after their name are trying to impress you.

Wednesday, 8 June 2011

Running

I never used to be able to run. When I was at school I'd be breathless and exhausted after one or two laps of the playing field, and I basically never ran if I could avoid it. On the other hand, I enjoyed walking arbitrary distances at speed.

Then a few years ago, on a whim I started running on the beach near my parents' house, and surprised myself by enjoying it. Since then I've often run short distances (a few kilometres). I was curious about what longer distances would be like, but I haven't had the time to try until a couple of times recently: a few months ago I ran 7km in 45 minutes (twice as long as I'd ever run before) and last weekend I ran 17km in two hours. It didn't seem difficult except that my feet got sore, which is perhaps because I was running barefoot: the problem is that if I run in any kind of footwear I own I quickly start feeling tired. There's probably something I could wear that would be OK, but this is part of my larger problem of not knowing anything about running so I'm probably doing it all wrong :-).



Tuesday, 17 May 2011

Singing

On Sunday I was coordinating the service at our church (Auckland Chinese Presbyterian Church) and I offered a couple of thoughts on worship, in particular, singing.

One bad habit it's easy to fall into is to sing along with everyone else without thinking about the words. If we're going to be honest with God, ourselves, and each other, we need to mean what we sing and sing what we mean. If the words say something we don't agree with, we shouldn't sing them. If I have an atheist friend at church and they don't sing anything, I respect that.

On the other hand, some Christians don't sing because they just don't feel like it --- perhaps they're having a bad day, or a bad year. But worship is not supposed to depend on how we feel; it's something we're commanded to do no matter how we feel. If a song is true then there's no harm and some benefit to singing it. (Of course singing is not the only form of worship, and if someone has taken a vow of silence, I respect that too :-).)



Monday, 9 May 2011

The Pinnacles

Over the weekend I took my kids to the Pinnacles in the Coromandel Ranges. It was my first time going there and the first time in my adult life I've done proper overnight tramping, i.e. carrying all my gear, staying out overnight (in a DOC hut in this case), and carrying it all out again. To make it more challenging I was carrying most of the kids' gear too!

The weather forecast was pretty bad --- rain both days --- but we went anyway, and as it turned out, the weather was fantastic. We drove to the end of the Kauaeranga Valley road, walked up the Webb Creek track for about two and a half hours and got to the hut just after 5pm. The Pinnacles hut is huge, and it was completely full on Saturday night --- not bad considering the weather forecast --- with 80 people. Despite what I wrote on my blog earlier about social interactions with strangers, I enjoyed talking to people at the hut; it helped that I got talking to people due to my kids being younger and cuter than everyone else at the hut! But there also seems to be good camaraderie at tramping huts in general.

There were a number of large groups, in particular about 20 teenagers plus minders on a school trip, and a group of university students who seemed to spend all their time cooking the most amazing food.

Today we got up at 6am to go up the actual Pinnacles, the core of an old volcano whose flanks have eroded away. After that we had breakfast and took the Billy Goat track back down. This took over four hours, partly because the track was very wet and muddy. At the end we had to cross the Kauaeranga stream by wading it, which was quite exciting with the kids! So we were rather wet and muddy, but we all had a fantastic time. We'll do more :-).





Thursday, 28 April 2011

"My Wedding And Other Secrets"

Tonight I went with my wife to see the movie My Wedding And Other Secrets. It has a number of points in common with my life story, starting with "nerdy white guy meets Chinese girl(s) at Auckland University". It was fun watching scenes in Albert Park, Continental Noodle House, and the university Science Building --- exactly where I used to hang out with my Chinese friends. (At least it looked like the science building, but it was dressed up to look like a film school. I'm pretty sure the Auckland uni film school doesn't look like the science building.) It was also fun to spot HP8, the Viaduct, downtown Queen St, and various other Auckland locations.

Although I enjoyed the movie, I was not over-impressed. Like a lot of romantic stories, much of the drama seems to depend on the protagonists being foolish. The dramatic arc of the last section of the movie seems unfathomable. James stakes the whole relationship on insisting Emily "choose him over her family" ... but she already has, because she's confronted her family and they've obtained her parents' permission to marry; his demand is that she move in with him immediately instead of waiting until they're properly married. To me that just seems pointless, petty and self-serving. Partly that's because I'm against sex before marriage, but even aside from that, it doesn't make sense. Plus I don't understand how Emily's mother's opposition to their marriage is motivated or is resolved.

Even more distressing, there is a major error in the Dungeons and Dragons scene. Emily is holding a d4 and announces she'll attack with her dagger, but then the DM tells her off for using the wrong die! But Emily was right, daggers do 1d4 damage!

Lastly, why is it that in movies, nerd girls are always actually gorgeous, just disguised? Is it impossible to cast a woman who's not beautiful, even if you're planning to ugly her up for most of the movie?



Tuesday, 26 April 2011

Gangly, Bespectacled And Cerebral

Unlimited Magazine ran a profile on me a while back. It's finally come out online.

Maybe I should change the title of my blog!

Tuesday, 19 April 2011

Shy

I've always been shy. To this day I find it difficult to talk to strangers face-to-face; over the years I've learned to do it reasonably effectively when necessary, but I'm still queasy in an adversarial situation. Large groups also freak me out, even amongst people I know --- two weeks of constant social contact with Mozillians just about exhausted my social energy. At least with technical people I don't have the confidence issues I have with other groups.

And yet, it doesn't seem to bother me getting into conflict situations on-line, when that's necessary. It's probably the Internet making the other person seem less than real. Overall I think that's a horrible feature, but it works for me here I guess.

I'm just thinking about this as I grapple with my fear of having to interact with car mechanics, while I'm not afraid to take on mega-corporations :-).



Tuesday, 5 April 2011

Movies

I saw four movies on the plane today. For once they were all good:


  • True Grit (the new one) ... really good genre piece. I love the crisp, precise, snappy dialogue, which you usually only see in pre-1970s films. Great characters. Well worth seeing.
  • Black Swan Disturbing but intriguing. Worth seeing, unless you have kids doing ballet in which case, avoid.
  • Winter's Bone Brilliant! At one level it's a noir-ish thriller, at another level it's just a young girl trying to keep her family together against the odds. It exudes gritty realism (the depressed and depressing countryside reminds me of some areas outside Pittsburgh), and has honest yet sympathetic characters played by astonishing actors. Riveting at every level. See it.
  • The King's Speech Formula drama that follows a well-worn pattern: character with defect seeks professional help, gradually builds trust with professional, relationship punctuated by periodic crises staged for dramatic effect, character overcomes defect. Yawn ... no wonder it won the Oscar. The weakest of the four movies, but I'll still call it "good" because what it does, it does very well, and poking fun at class structures warms the heart even if it's obvious and crowd-pleasing.

None of these movies are better than Toy Story 3 though. Rusell Baillie agrees, and so does Christianity Today.



Monday, 21 March 2011

White Island

Last weekend we had the annual Mozilla Auckland outdoor activity. No-one else volunteered to organise it so once again I had us visit a volcano :-). This time, it was White Island in the Bay of Plenty.

We drove down to Whakatane on Friday night and stayed at the White Island Rendezvous Motel, which also runs the guided tour boat trip out to the island. We went out to the island on Saturday morning. Small boats weren't leaving the harbour due to tsunami fears, but our boat was cleared to leave. The boat took an hour and a half to get out to the island, then we had about two hours walking on the island, after which we had lunch on board, circumnavigated the island and returned to Whakatane. The weather was fantastic for the whole trip --- sunny, warm, a slight swell on the bow on the way out but nothing to speak of on the way back.

The island is amazing. It's basically just a large crater with a high rim to the west, north and east. Inside the rim is an astonishing, completely barren landscape. The crater walls are bare rock, much of it loose, heavily eroded by rainfall. The floor is rocks, ash, and mineral formations, plus some fumaroles, boiling water and mud, and the crater lake itself. The lake is amazing: quite large, constantly roiling, hot (currently around 67 degrees C, 152 F), and with a pH of -0.4, considerably stronger than battery acid. The fumaroles and the lake itself constantly belch a mix of steam and sulphuric fumes. The steam plume is huge, constantly visible from Whakatane in clear weather. The fumes shift around in the wind; when they blow your way, you start coughing and spluttering --- fortunately the tour company provides masks with fliters. The place is both hellish and totally fascinating, and beautiful in its own way. (Odd that we would consider something attractive that's so inimical to life!)

The crater area contains the remains of sulphur mining operations. They're a lovely example of industrial ruins, and look older than they actually are due to the spectacularly corrosive environment.

As a nice bonus, we encountered dolphins on the way to the island and on the way back, and a seal too! On the return trip there must have been hundreds of dolphins in the group; for a while, wherever you looked you could see dolphins leaping out of the water at every moment. They loved playing in the wake of the boat, and the boat's skipper was adept at encouraging them. Seeing those magnificent animals just having fun was incredibly exhilarating --- I just wanted to laugh, or cry, with joy.

We capped off the day with a visit to Ohope beach, which was lovely, and dinner at the "Africa" cafe, which was wonderful South African food, although I think they were overwhelmed by the fifteen of us --- me, Chris Double, Chris Pearce, Matthew Gregan, Karl Tomlinson, Daniel Holbert, Jonathan Watt, Cameron McCormack, plus Doug Schepers from the W3C, plus a few partners and family members. All in all, an excellent adventure!