We Need A "Dumb Device" Movement

I'm habitually pessimistic about many things, and this year Snowden reinforced my habit. In the narrow sense of his obligations to the US government, he's a traitor, but to the human race as a whole he's a hero and a role model; he personally is inspiring, but what he revealed is depressing.

I think his most important lesson is that total surveillance is an explicit goal of the US and UK governments (and by extension other governments), and there's no real restraint in how that goal is being pursued, especially for those of us who aren't US citizens. Combine that with the cold truth that we are incapable of securing complex systems, and we're in a very bad situation. We have to start assuming that mass-market computing devices are compromised, or can be compromised at will.

When people talk about the "Internet of things", they're implying the situation is going to get much worse. Every device that is network-accessible and supports updateable software is a surveillance device ... if not all the time, then as soon as someone decides to turn it on. (Let's ignore for now devices that can be programmed to take hostile action against their users!) I am not in favor of the Internet of things in the present climate.

Unfortunately, factors of cost, convenience and cool will keep driving general-purpose, network-accessible computation into every nook and cranny of our world. It may help if a significant subset of customers (I hate the word "consumers", it's demeaning) prefer devices that don't have unnecessary computation jammed into them. I want to buy "dumb devices" --- meaning they are not unnecessarily smart, and don't talk about me behind my back. My refrigerator, clothes, and bicycle do not need network access or upgradeable software, and I don't want them. Of course, if my market segment's population is me, it's not economically viable. Therefore I need a mass movement.

One interesting product segment is cars. The computerization of cars is truly terrifying, and there is some great work detailing how modern cars can be subverted. I would pay a decent premium for a car that lacks any kind of over-the-air communications. A potential problem is that safety regulations require new cars to have sophisticated computers, and sooner or later a computationally secure car may become effectively illegal, if it isn't already.

I don't know what to do from here. Does this movement already exist? If not, I hope someone starts it, since I'm rather busy.

Blood Clot

I'm tagging this post with 'Mozilla' because many Mozilla people travel a lot.

In September, while in California for a week, I developed pain in my right calf. For a few days I thought it was a muscle niggle but after I got back to New Zealand it kept getting worse and my leg was swelling, so I went to my doctor, who diagnosed a blood clot. A scan confirmed that I had one but it was small and non-threatening. I went to a hospital, got a shot of the anticoagulant clexane to stop the clot growing, and then went home. The next day I was put on a schedule of regular rivaroxaban, an oral anticoagulant. Symptoms abated over the next few days and I haven't had a problem since. No side-effects either. I cut myself shaving at the Mozilla Summit and bled for hours, but that's more of an intended effect than a side effect :-).

I had heard of the dreaded DVT, but have never known anyone with it until now, and apparently the same is true for my friends. Fortunately I didn't have a full-blown DVT since the clot did not reach a "deep vein". Still, I can confirm these clots are real and some of the warnings about them are worth paying attention to :-).

I saw a specialist for a followup visit today. Apparently plane travel is not actually such a high risk so in my case it was probably just a contributing factor, along with other factors such as sitting around too much in my hotel room, possibly genetic factors, and probably some bad luck. (Being generally healthy is no sure protection; ironically, I got this clot when I'm fitter than ever before.) So when flying, getting up to walk around, wiggling your toes, and keeping fluid intake up are all worth doing. I used to do none of them :-).

My long term prognosis is completely fine as long as I take the above precautions, wear compressing socks on flights and take anticoagulant before flights. I will be given a blood test to screen for known genetic factors.

I did everything through the public health system and it worked very well. Like health systems everywhere, New Zealand's has its good and bad points, but overall I think it's good. At least it doesn't have the obvious flaws of the USA's system (the only other one I've used). In this case I had basically zero paperwork (signed a couple of forms that staff filled out), personal costs of about $20 (standard GP consultation), good care, reasonable wait times, and modern drugs. I was pleased to see sensible things being done to deliver care efficiently; for example, my treatment program was determined by a specialist nurse, who checked it out with a doctor over the phone. New Zealand has a central drug-buying agency, Pharmac, which is a great system for getting good deals from drug companies (which is why they keep trying to undermine it, via TPP most recently ... of course they managed to make the Pharmac approach illegal in the USA :-(.) Rivaroxaban is relatively new and not yet funded by Pharmac, but Bayer has been basically giving it away to try to encourage Pharmac to fund it.

Overall, blood clots can be nasty but I got off easy. Thanks God!

Why I Don't Worry About Global Warming (Much)

I don't worry about global warming or any other threat whose most important effects are several decades out. Technology is going to change everything by then: either we'll kill ourselves in more immediate ways, or at least destroy most of civilization (which would do a lot to reduce carbon emissions!) --- or we'll make a lot more technological progress, probably developing brain uploading, strong AI, or other game-changing capabilities we can't forsee yet. Probably the former.

People who worry about what will happen when the sun burns out in five billion years are the worst.

One Day The Luddites Will Be Right

Whenever a person proposes that technological advances might reduce human job opportunities in the long term, someone responds with the Luddite Argument: "the Luddites thought the Industrial Revolution would destroy their jobs, and they were wrong, so you're wrong too" [1]. Some go further and explain that the Luddites were wrong because technological productivity improvements are balanced by finding new uses for human labour. Wikipedia has a good summary. However, it seems obvious to me that at some point technological advance will --- or at least could --- be a net destroyer of jobs. All you have to do is imagine a world where robots can do everything a human can do, at lower cost than maintaining a human life. Clearly, there are no economically rational job opportunities for humans in that state [2], so at some point of technological advance short of that state, there's net job destruction.

The only question is whether and when we will reach that point. It seems inevitable we'll reach it unless something halts technological progress or some very strong flavor of Cartesian dualism holds. Economic arguments that human labour will still be worth something at that point are just wrong.

[1] Actually the Luddites were right; the Industrial Revolution did destroy their jobs, and drove them into misery. But they were wrong in that they did not forsee the net benefits to future generations.

[2] There could be sinecures to keep humans occupied, but they would not be economically motivated.

WebRTC And People-Oriented Communications

Tantek had an interesting blog post about making people rather than protocols the organizing principle of communication apps. I like his vision quite a lot. One neat extension of his post would be to introduce WebRTC. With WebRTC it would be relatively easy to have the "Robert O'Callahan" app check if I'm currently logged into the appropriate receiver app, at any WebRTC-capable endpoint, and if I am, establish a voice or voice+video session with me (with peer-to-peer transmission, naturally). If I'm not logged in, WebRTC lets you record a message for later delivery. This would be very cool.

Another Knee-Jerk Reaction To International Rankings

Predictably the OECD "Pisa" report ranking countries' education results has caused a stir in New Zealand. New Zealanders, or at least their news media, love international rankings of all kinds --- especially if they can be portrayed negatively for New Zealand. As often, the latest report, and the discussion around it, has some major problems.

For starters it's interesting to compare the initial NZ Herald story with the more nuanced reporting from the AP wire story. The Herald chose the completely fallacious headline Significant drops in NZ educational achievement --- fallacious because a drop in ranking does not necessarily mean a drop in actual achievement (and in this case, there is no evidence of a drop in achievement). (PPTA president Angela Roberts gets this right here.) The Herald story (and its followups) simply ignores the issues with the Pisa report which are touched on by the AP story, and better explained in Slate (mainly, it's invalid to compare city-states with entire countries, especially including Shanghai but no other part of China!).

Apart from that, some of the reactions to the report are ridiculous. There's this:

But Labour says any drop in the rankings should be sheeted home to an excessive focus by National on "testing" over the past five years. Although National Standards does not actually involve national testing, Labour's education spokesman Chris Hipkins said, "It shows that the last five years' focus on test-taking has been a disaster and it has actually narrowed the focus of our system and it has actually decreased the level of achievement within the education system".

NZ's ranking drop is mainly due to Asian countries increasing theirs. Those countries' education systems are far more focused on testing than NZ's has ever been, which is probably why their ranking is increasing: if you focus on testing (and teach to the test), you do really well on tests, which is of course what studies like Pisa measure, since tests produce data and other educational activities don't. Chris Hipkins, if you really think Pisa is important you should advocate a big increase in national testing.

I actually think national standardized testing is important, but it's not the only important thing and the Asian education systems ranked highly in the Pisa report have massive problems despite producing good test results. My wife and many other people I know went through those systems and describe how they're focused on rote memorization and discourage any kind of learning other than school and after-school coaching on their core subjects. For example, I taught myself computer programming in my copious spare time after school, but for most children in Hong Kong that simply wouldn't have been possible. People talk about how in exams you "give it back to the teacher" --- cram for exams, do well, and then forget it as you prepare for the next one. I think it would be very interesting to re-test children three years after they left school to see what they've retained.

It's really important to avoid over-optimizing for the things we can measure at the cost of the things we can't as easily measure. It's also really important to not overreact to every international ranking report. We have to think critically about these things. I wish the media would.

Does John Banks Only Do Good?

I read this rather unusual statement by John Banks:

"I've spent a lifetime of doing good, a lifetime of trying to balance my family ledger, a lifetime of making a difference for people, and a lifetime of contributing to this country. I only do good. I don't do bad things."

Reporters like to take quotes out of context so maybe Mr Banks didn't really mean to say this, but for the sake of argument I'll assume it's accurate.

What strikes me about this statement is that in the past John Banks has portrayed himself as a Christian, and I don't think a Christian can make that statement or anything like it. A bedrock truth of Christianity is that we are all sinners, and bad ones. We've all done bad things and we all keep doing bad things. We generally don't realize it, because we're good at self-justification and our standards are lower than God's, but we are all desperately in need of forgiveness. This holds no matter where we are on our spiritual journey --- "forgive us our sins", as Jesus taught his disciples to pray.

So, for a self-professed Christian to say "I only do good, I don't do bad things" is an egregious error, no matter how it was meant. I hope that any Christian genuinely practicing their faith would try to steer away from saying anything like it under any circumstances, lest they give a false impressions of what Christians believe.

Wanaka

Last week my wife and I celebrated our wedding anniversary by taking a few days off in the lakeside town of Wanaka in the South Island. The weather was unexpectedly delightful. As is our wont, we spent a lot of time walking during the day and refueling at night.

We arrived on Tuesday. In the evening we walked through the town to Iron Mountain and walked up it (it's small). On Wednesday we ascended Roy's Peak. That walk has a 1200 metre elevation change but the views are definitely worthwhile. On Thursday we drove out to Mt Aspiring National Park through Matukituki, to do the Rob Roy Glacier walk. This is a really beautiful walk, through pasture, into the bush, up the Rob Roy valley, emerging in an alpine meadow at the head of the valley with stunning views up the mountain-face to the Rob Roy glacier --- snow gleaming in the sunshine, and about a dozen meltwater waterfalls streaming down the face. On our last day we drove out to Haast Pass and then back to Queenstown.

The whole area is very beautiful. It's quite special to sit by the lake and watch people sailing, swimming and fishing, surrounded by ice-capped mountains. The Department of Conservation's Mt Aspiring activity centre is in Wanaka and well worth a visit. The number of day walks and multi-day tramps in the region is just staggering; there must be a couple of dozen different huts. No wonder the town is beseiged by tourists :-).

Mozillians At The Tongariro Crossing And Orakei Korako

Last weekend some local and visiting Mozillians (visiting for a Gecko media work week), plus assorted loved ones (such as my children), drove down to Tongariro National Park to do the Tongariro Crossing on Saturday.

The weather was reasonably good --- cloudy, but mostly dry. As often, there was thick fog up on the mountain, clearing as we descended the northern flank. Even though the fog eliminates some of the amazing views, it's still very atmospheric and fun in its own way. It does help conceal the other thousand people doing the hike :-).

I've done the TC several times before but this was the first time I've done it since last year's eruption from Tongariro's Te Maari crater. That has made the TC even more interesting! On the descent past Te Maari you can now see large quantities of steam venting from the crater --- quite impressive. The Ketetahi Hut was hit by boulders from the eruption, and overnight stays are now forbidden there for safety reasons. Two decent-sized holes punched in the hut roof have been preserved and make an excellent argument for following the rules :-). An even better argument is just down the track from the hut: a sizable impact crater, with the impacting boulder mostly buried at the bottom of it. No-one was at the hut on the night of the eruption, but if they had been, it would have been a terrifying experience!

Near the end of the track, it crosses a stream. Lahar activity associated with the Te Maari eruption has blocked the original course of the stream, so the existing bridges are over dry streambeds. Now the stream spreads through the bush in other places and we had to splash across.

We finished the walk in just over seven hours, and a good time was had by all.

On Sunday some of us drove back to Auckland via Orakei Korako. I hadn't been there since I was a child, even though I really enjoy geothermal areas. I very much enjoyed Orakei Korako! It has an excellent mix of features --- geysers, silica terraces, hot pools, boiling mud, and a cave --- all in a lovely bush setting, plus you cross a small lake in a boat to get to it. This is my second favourite geothermal area in New Zealand, after Waimangu Valley. Waimangu's Frying Pan Lake is hard to top :-).

A Tale Of Two Cities

The week before last I was in Paris for the "Web rendering" work week. It went reasonably well. The Paris Mozilla office is as amazing as it looks. Benoit Jacob gave us an excellent walking tour on Sunday. Brian Birtles and I split off and attended Trinity International Church, which seemed really good. It's great to see what God is doing in different places and be a tourist in God's kingdom. The actual work was good. It was a mix of briefing each other on what's going on, brainstorming new directions, and thrashing out problems. It was particularly good to have Gaia and Shumway hackers there to talk about their issues.

Due to some combination of jet lag and God's grace I woke up at 7am sharp every day and went for a run, training for the Auckland half-marathon. It was a lot of fun to run from the hotel down Rue de Louvre to the Seine, along to the Eiffel Tower, back along the Seine, and then on a good day do a lap of Île de la Cité and Île Saint-Louis. However it was a bit strange to be running at 8am with the sun not having risen yet!

I got back to Auckland on Monday and had an extremely hectic few days before I head out to San Francisco today (Sunday). The biggest thing on my mind was the half-marathon, which was today, but I only had time for a couple of 10km training runs. There's a nice 10km circuit from my house to the summit of One Tree Hill and back. This takes me through city streets to the bucolic charm of Cornwall Park and One Tree Hill --- especially charming at this time of year, with cows and calves, sheep and lambs, verdant pastures, blooming trees, and excellent views from the summit of all of these, as well as the Waitemata and Manakau harbours. Entirely different from Paris but to my mind even more delightful!

I really enjoyed the half-marathon today. Not having done one before, I wasn't sure what to expect, but was pretty pleased with my result. I ran faster than I did during my solo runs; I think the crowd helped motivate me, especially towards the end. I didn't want to try it barefoot, at least for the first time, so while in Toronto last month I bought some Vibram foot-glove-type footwear, by asking for the closest thing to barefoot they had. These seem pretty good. Interestingly though, every other person I saw during the race had regular footwear on. In fact several people at the starting area were curious about my footwear and asked me various questions. It's a bit awkward since I don't really know anything about running!

Now I'm heading off to San Francisco for Mozilla's LEAD program. This is the last session so hopefully after this week my travel and the rest of my life will return to relative normality. Though we do have the media work week in Auckland in a couple of weeks, with a trip to the Tongariro Crossing thrown in, and after that a trip for our wedding anniversary, ... so it goes.

Summit Day Zero

It was pretty good. Lots of fun talking to people old and new. A few embarrassing incidents where I didn't recognize someone I should have. I brought down Settlers of Catan from my room --- Rob Arnold won, but only because Doug Sherk kept stealing my road. I also brought down Citadels and some other people played that. I should have brought more games! If we're still short of games tomorrow night we can call in reinforcements from the Toronto office.

Prescriptive Vs Descriptive Frameworks

People love to create taxonomies. Just about the first thing that Adam does in Genesis is to name the animals. Classifying and naming things is valuable because it helps us to understand relationships between entities and communicate with other people about them. However, there is a common error where people come to believe that entities that don't fit easily into the taxonomy either don't or shouldn't exist, or solve problems by searching the taxonomy when that's not the best approach.

Design patterns are a great example. They are a very useful vocabulary for explaining code. However, it's usually a mistake to tackle a programming problem by finding the design pattern that best fits and applying that pattern to the problem. Instead, we should always analyze the problem on its own merits, applying all the information we have about the context, and come up with the best solution specific to that problem. (I'm assuming here that we're interested in producing the very best code for a solution, which is true for something like Firefox but not for throw-away code or for projects with a small audience.) But since design patterns are easy to teach, and selecting a design pattern is easier to systematize than finding the best solution ex nihilo, and it's easier to tell your boss that you're using the Fridge Magnet pattern than to explain a bespoke solution, it's easy to see why a lot of people made design patterns their Procrustean bed.

This observation generalizes beyond programming. In areas as diverse as personality tests and politics, there is a drive to describe complex entities with simple categories. Entities that don't fit easily into a single category are implicitly or explicitly discouraged, to disastrous effect. I find it extremely frustrating.

We need to make a clear distinction between descriptive frameworks, which provide vocabulary to talk about common features (often not mutually exclusive), and prescriptive frameworks, which demand that reality be describable with a fixed vocabulary (generally with mutual exclusion). A taxonomic framework is very useful for describing what exists. It should not be used as the search space for solutions to problems, although it may be useful by inspiring you with options you haven't considered.

Avoiding Burnout

Mozilla has lost some good people due to burnout. I suspect working for Mozilla carries greater risk of burnout that working for "normal" organizations. A lot of people are motivated to work for Mozilla because of our mission. This motivation is powerful and persists through unhappiness. Thus, people drive themselves to work hard at the expense of their happiness and mental health. This is a recipe for burnout.

That is very true for me. There have been long periods of time when I have been frustrated with my work, feeling I would be happier doing some other work, and in fact feeling that I'm struggling to cope. But I've survived in this job a long time, I don't currently have those feelings, and in fact I feel pretty good even though I'm currently going through some travel madness. Here's a few things that have helped.

Acknowledging the ebb and flow. When I'm feeling burned out, I slack off. I think that is OK as long as it's temporary --- and as long as I'm not using it as an excuse to be lazy. For me, it's natural to have periods of time when I'm energized, followed by periods when I'm tired and I need to back off while I rest and rekindle my enthusiasm. Per my earlier post, this is not a sign I need to change jobs.

A sense of perspective. When I feel my job sucks, it's important to take a step back and think about what I'm comparing it to. Objectively, my worst weeks at Mozilla have been worldly bliss compared to most weeks of most people over most of human history. It is critically important to never lose sight of this fact. (There are a few Mozilla people for whom this is not true; you know who you are and I tip my hat to you.) Whenever I feel like a heroic martyr, it's simply ridiculous.

Changing focus. When I'm sick and tired of working on something, it's very helpful to work on something different --- "a change is as good as a rest" (which is mostly but not entirely true). Mozilla does a lot of important things and there are a lot of very different ways to contribute to Mozilla's mission, so it's not a problem to rotate people in and out of particular roles.

Family. My wife and children are a vital anchor for my mental health. My commitments to them --- and their policing of those commitments --- prevent me from going too far overboard with work. Spending time with them, giving and receiving love, is the best tonic I have.

The Sabbath. I often struggle to rest and relax because I feel burdened with work projects and guilty for not working; this contributes to burnout. I find that compulsory downtime, decreed and ideally enforced by an external source, is extremely helpful in dealing with that guilt: I don't need to feel guilty about not working when someone is making me do it. My family is one source of that compulsion, and my Christian convictions about not working on Sunday (not always observed :-( ) are another.

Relinquish responsibility. Feelings of being irreplaceable can be extremely draining and contribute to guilt, anxiety and the difficulty of resting. Therefore it's very important to break those feelings --- forcing oneself to step aside, at least temporarily, and see what happens. It's usually not as bad as you think. We also need to keep in mind that we are only responsible for what we do; as individuals we are not responsible for Mozilla's ultimate success or failure. We have to let go and not worry about what we, as individuals or as a group, do not control. Over the years I've developed a little bit of apathy, which has been valuable.

Remember the mission. Sometimes, when enthusiasm fails, I must ask myself whether there's any work I could be doing that's more important than my work at Mozilla. So far, the honest answer has always been no, and so I know what I must do. As I suggested above, this can be a scourge, but it can also be liberating. There's a certain exhilaration in knowing what one should do, and throwing yourself into it with nothing but grim determination.

Jesus. Sorry to my irreligious readers, but this true and important. Many times I have felt myself to be running on empty, and my only recourse has been to pray to the Lord for strength to do what I know I should do. Many of those times I have been quickly refreshed in a way that feels miraculous. Apart from that immediate relief, my identity in Christ provides logical and emotional support to many of the items above: my commitment to my family, my sense of duty, my sense of perspective, my sense that God is ultimately in control, and the obligation to periodically disengage from work. Had I not been a Christian, I don't think I would have survived.

Travel Madness

I am currently in the middle of the most intense travel burden that I have ever experienced. In the week starting September the 8th I was in California for the LEAD leadership training program. Then I was home for two weeks before flying to Toronto for 8 days for the Mozilla Summit. After that I'll be home for a week and a half before flying to Paris for the "Web rendering" work week. After that I'll be home for a week before flying to California again for LEAD.

I know a lot of people fly more frequently than this, even at Mozilla, but for me this is extraordinary. Also, because I live in New Zealand, the distance of these trips is especially great. For the nine weeks starting September 8, my average speed is over 65 kilometres per hour.

I still like travel! It's fun to visit different places and people. The problem with travel is that it takes me away from my family, my closest friends, and my church. It's not good for the environment either. Also I don't sleep all that well when I'm travelling. I'm not sure why; I think it's not so much jet lag as being away from the comforts of home, and tending to be too busy to do my normal work, so trying to catch up on my normal work at nights.

Fortunately this will come to an end soon. LEAD ends this year so next year I can expect things to return more or less to normal.

The Forge Of Disappointment

I think New Zealanders have reacted to Oracle's stunning victory in the America's Cup very well. There's very little opprobium or anger, but instead a lot of praise for Team New Zealand and respect for Oracle. A lot of people are bitterly disappointed --- hopes were, rightfully, very high when TNZ was 8-1 up --- but I don't sense the doom-and-gloom that descends whenever the All Blacks fail to win the Rugby World Cup. This is probably because TNZ was, in the long run, always the underdog, whereas the All Blacks never are.

Like Rugby World Cups, this event has been unifying for large swathes of New Zealanders. It's gratifying to know that many of your friends and neighbours, and people around you that you don't even know, are sharing this intense experience. Shared enthusiasm and disappointment, joy and despair, all contribute to building our feelings of community. This is old news: historians traditionally see World War I as a key catalyst in the development of New Zealanders' national identity, especially the ANZAC deployment at Gallipolli, which was a horrible disaster militarily.

Being much more cerebral than sportsmanly, I used to see professional sports as ridiculous and wasteful, but now I think I see one way they can be genuinely valuable. We need events, even artificial ones, to rally around. My Canadian pastor observes that Quebecois separatism subsides dramatically while the Canadian hockey team is playing during the Winter Olympics. This sort of thing forges people into nations.

A Tip For The Surivial Of Humanity

Let's assume that at some point in the future we'll be able to build computer systems capable of behaving just like a human brain. There are three obvious ways to get there. One way is to build a system that emulates an actual human brain, bootstrapped by copying the brain of an existing person. Another way is to emulate a human brain, but bootstrap via a learning process as children do. The third way is to build a system that doesn't work like a human brain but reimplements the functionality in whatever way seems most suitable to the hardware we have built. I believe that copying the brain of an existing person is by far the best path; the other options are much greater threats to the survival of humanity, and indeed intelligent life.

Fully reimplemented intelligence would be very hard to predict and control. Reliably engineering for virtue will be extremely difficult. In its early stages the system would likely be narrowly focused on specific goals (perhaps military or corporate), and there is great potential for catastrophic bugs, such as runaway goal seeking that accidentally devastates or exterminates organic humanity. Even if you think a complete transition from organic to machine intelligence would be a good thing, we could easily by accident end up at a dead end for intelligent life, for example if humans die off before all aspects of machine self-reproduction are automated.

On the other hand, if we copy an existing brain, we will know roughly what we're going to get: a disembodied human mind in a machine. We can even choose particularly virtuous people to copy. The process will no doubt be lossy, but people with severe disabilities cope with sensory and motor deprivation without going mad, and our emulated minds probably will too. We can be confident that a benevolent, thoughtful person who cares for the welfare of humanity will still do so after transcription.

The emulation-from-infancy approach falls somewhere in between in terms of risk. The result would probably be more like a human than fully reimplemented intelligence, but it's hard to predict what kind of person you would get, partly because they would grow up in an environment very unlike what we would consider a favourable environment for children.

I'm optimistic that an emulation approach is more likely to succeed technically than a full reimplementation approach at producing a self-aware general-purpose intelligence. Futurists tend to imagine that mere aggregation of computing power will bring on intelligence, but I think they're quite wrong. A great truth of computer science is that hardware scales but software remains hard. Porting existing software is expedient. However, copying an adult brain may be a lot harder than bootstrapping from infancy.

I'm not sure that any of these things will happen in my lifetime, or ever. There is great potential for God or man to frustrate our technological advancement.

Unexpected Visitors

Today I was working at home when two Korean women knocked on my door. They were from a "Bible study group" and wanted to "conduct a survey". They were obviously evangelists so, not wanting to waste their time, I said I was a Christian already. That obviously was covered in their script, since they immediately asked if I knew about "God the Mother", and when I said no, whipped out a Bible, turned to Genesis 1:26-27 and explained that since God created men and women in his image and uses the plural personal pronoun, he must therefore have male and female sides. I explained why "the image of God" clearly cannot mean God is like us in every way, and the plural "us" in verse 26 is a royal plural, and then we went on a merry tour of Bible verses cherry-picked to support various unusual doctrines. They seemed to focus a lot on verses in Revelation mentioning the bride of Christ (something to do with "God the Mother" I suppose), and on verses to do with "living water". (They were interested in the source of it; no idea where they were going with that ... not the filioque controversy at least :-).). Suffice it to say that I did not agree with their interpretations.

One woman was somewhat older than the other, and obviously mentoring the younger, since the younger took the lead initially but when she started to flounder the older woman took over. When she finally tired of me she accused me of following my own ideas and not the text, and took her leave. Before they left I managed to hand them a card for ACPC and invite them to my Bible study at 7:30pm tonight. I genuinely hope they'll show up, though I'll be surprised if they do. (Then again, wouldn't they want the opportunity to convert more than one person at a time?) Funnily enough the only thing that really knocked the older woman off her stride was when she discovered I go to a Chinese church.

A quick search suggests that they belong to the World Mission Society Church of God, a pseudo-Messianic Korea-based, Christian-based cult, er, minority religious group.

In the excitement, there are things I wish I'd done differently:

• Known something about their organization during the conversation
• Pulled out my study Bible to get more background on some of the verses they were using
• Had at my mental fingertips verse references for key concepts such as the church as the bride of Christ
• Made a better case for them to visit our Bible study or our church to explain their ideas more fully
• Gotten some contact details from them

I have immense respect for these women. They probably sincerely believe that their door-to-door evangelism is helping people reach salvation, and are willing to overcome fear, shame, a language barrier and demands on their time in pursuit of that. I think they're wrong about some critical issues, but they get more right about Jesus than the average person who rejects him completely. I'm impressed.

Debugging Transient Rendering Issues With GNOME Shell Screencasts

Debugging transient issues like flickering can be a real pain because it's difficult to see exactly what's happening when it only affects a few rendered frames. Enabling paint flashing in Firefox is useful for detecting unnecessary repainting but again it can be difficult to use when there's a series of consecutive interesting frames --- e.g., when you're trying to figure out why something has repainted many times in a row instead of just once. But now I have found an excellent solution to these problems!

Andrew Overholt recently pointed out to me GNOME Shell's built-in screencasting ability --- just press Shift-Ctrl-Alt-R to start recording, and a WebM file is dumped in ~/Videos. As far as I can tell, this file contains exactly one video frame per frame rendered by the desktop compositor, which is perfect for analyzing rendering issues. (Video frame timestamps are correct, so playing the file normally works as expected.) Loading it up in gmplayer, you can press '.' to advance one frame at a time. The video quality is high, and overall it's absolutely brilliant for frame-by-frame analysis of what was rendered.

Servant Leadership

Every so often the principle of "servant leadership" is mentioned in discussions about management. Sometimes it's portrayed as something new and radical. It's definitely radical, but not at all new. It goes back as least as far as the New Testament, in which Jesus spells out the necessity of servant leadership among his followers. For example:

Matthew 20:

"Jesus called them together and said, 'You know that the rulers of the Gentiles lord it over them, and their high officials exercise authority over them. Not so with you. Instead, whoever wants to become great among you must be your servant, and whoever wants to be first must be your slave — just as the Son of Man did not come to be served, but to serve, and to give his life as a ransom for many.'"

Luke 22:

"A dispute also arose among them as to which of them was considered to be greatest. Jesus said to them, 'The kings of the Gentiles lord it over them; and those who exercise authority over them call themselves Benefactors. But you are not to be like that. Instead, the greatest among you should be like the youngest, and the one who rules like the one who serves. For who is greater, the one who is at the table or the one who serves? Is it not the one who is at the table? But I am among you as one who serves.'"

John 13:

"When he had finished washing their feet, he put on his clothes and returned to his place. 'Do you understand what I have done for you?' he asked them. 'You call me "Teacher" and "Lord," and rightly so, for that is what I am. Now that I, your Lord and Teacher, have washed your feet, you also should wash one another’s feet. I have set you an example that you should do as I have done for you.'" (This is of course the origin of the tradition of the Pope washing people's feet.)

It's not surprising people forget this, since these instructions have been followed poorly over the centuries. In general it seems very difficult to sustain servant leadership as organizations (of all kinds) grow vertically; unity across ranks diminishes, and greed, vanity and ambition take over. It's less of a problem when leaders are driven by shared belief in a mission more than desire for personal status and gain --- as tends to be the case at Mozilla, and the churches I've known. It has also helped that the churches I've attended have all been fairly small, with loose ties (if any) to higher levels of hierarchy.

Indoi

Indoi is a new Indian restaurant on Short St in Newmarket, very close to the Mozilla office. It's my favourite Indian restaurant in Newmarket. The non-vegetarian thali lunch special is $15, which is a very reasonable price for a very nice meal. I've concerned about their longevity though; I almost never see customers there other than our lunch group. It's probably partly the location; a lot of restaurants have come and gone there, Short St doesn't get much foot traffic, and the restaurants on Lumsden Green probably snap up a lot of the customers heading in that direction. My advice is to eat at Indoi before it shuts down!

Creating Screencasts On Linux

I had great difficulty creating a good screencast for the previous blog post. Various tools I tried all had horrible flicker problems in the captured video. I think the problem is related to the GNOME compositing window manager I'm using in Fedora 18. After various tools failed, I solved the problem using vncserver to set up a separate X11 display in which to run the demo app, and using ffmpeg to record the video like so:

ffmpeg -f x11grab -s 1024x768 -i eternity:2 capture.webm
(where 'eternity:2' is the DISPLAY for the vncserver)

Update: Ignore the above solution. Andrew Overholt pointed out that GNOME Shell has screen-casting built in: just press Ctrl-Alt-Shift-R to start and stop, and .webm files are dumped in your Videos directory. It works perfectly for me! Somehow in all my Internet searching I never discovered this feature.

SVG-In-OpenType Progress Update

TypeCon is coming up and there's going to be a discussion there about competing proposals for advanced glyph rendering, especially coloured glyphs. That was an incentive for me to knuckle down over the last couple of days and fix some issues in our implementation of the Mozilla-Adobe proposal that adds an "SVG" OpenType table to render glyphs in SVG format.

I had to update our implementation from our original prototype to use the format defined by the merged Adobe-Mozilla proposal. Then I had to track down a bug in the calculation of glyph extents for SVG glyphs --- this turned out to be a generic bug affecting any font with egregious glyph extents, so I'm glad we've caught it. The last and most interesting part of the Gecko work was to enable animation in SVG glyphs. This is tricky because we need to map from a font whose glyphs have changed back to all documents and text that using that font, so we can ensure the text is properly repainted, and we've never needed this mapping before. gfxFont.cpp is gnarly but the code worked out OK.

Creating fonts with SVG glyphs has required hacky scripts so far, and I was getting tired of that, so before I did the Gecko changes I wrote a small Web app to help: SVG OpenType Workshop. You can open an existing OTF/TTF font file (sorry, no WOFF support yet), create/edit an SVG table document, see the results on (editable) sample text live as you type (generating a new font file at every keystroke), and save the generated font. I found it very useful, and it shows that you can have live font editing in Web apps without an SVG font DOM or other dedicated browser APIs. To render the fonts, the application requires browser support for the SVG-OT draft mentioned above, so for now only experimental Firefox builds will work.

Check out this screencast of animated glyphs in action.

Mozilla Code Reviews Talk On Air Mozilla

In Taipei in May, as a followup to the engineering culture discussion, we had a session about code reviews, covering the topic from both the point of view of the code reviewer as well as the code author. This session is now online at Air Mozilla. Similar to the engineering culture talk, this wasn't originally intended for public consumption, so is rather unpolished, but you get an honest look at what Mozilla developers talk about amongst themselves :-).

Prepare To Stop

Serendipity.

Mozilla Engineering Culture Talk Now Online

At the Mozilla "Web Rendering" work week in Taipei in May, I led a discussion of "Mozilla (really, Gecko) engineering culture". This was partly for the benefit of all the new developers in Taiwan but I think it was interesting for many of the old hands too. The talk is now available on Air Mozilla. It's excruciating to watch myself talk, but people may find it interesting.

It wasn't initially intended for public consumption, but after reviewing it I thought "why not?". So it's a pretty honest snapshot of Mozilla developers explaining what we do.

A Shock Of The Ordinary

Last night I watched a scene in an episode of the second season of Downton Abbey in which two sympathetic characters visit a church together and pray. That's it --- no plot devices, no subtext, no ulterior motives, no mockery; it was just showing what some ordinary people sometimes do. I was shocked, because I basically never see Christians just being Christians in movies or television. Soon afterward I was flipping through the catalogue of the New Zealand International Film Festival, and there it seems one is more likely to see a serial killer being portrayed sympathetically than a Christian. This is not surprising --- "it's 'the world', stupid", as someone once put it to me --- but it's sad nonetheless. No wonder people have such distorted ideas about us.

Happy Days

I gave permission from a writer to use an old wedding photo of ours for a story.

We got married in Sydney because that's where my wife was living at the time --- I was at CMU in Pittsburgh. We had the ceremony in the morning and the reception in the evening so we had several hours in between to just roam around Sydney taking photos and having a good time. I think a lot of weddings are for the benefit of guests and the couple doesn't have such a great time, but we bucked that trend. It was definitely one of the best days of my life.

A lot of things have changed since then. Our marriage has had its issues but I'm very thankful that in most ways it's better than it's ever been, as we keep learning to be better at being married. We'll need to, for the challenges ahead!

Avoiding Copies In Web APIs

The Web platform has matured to the point where avoiding data copying is often an important optimization to reduce running time and memory usage. Avoiding data copies often requires careful API design. The Web has two facilities that are very useful for writing such APIs: Blobs and ArrayBuffer neutering.

A Blob is a chunk of generic immutable data, optionally with a MIME type. Its contents are not directly byte-addressable, enabling browsers to easily offer multiple implementations of Blob under the hood. For example, in Gecko we support Blobs backed by in-memory buffers, Blobs backed by files, and Blobs that reference lists of other Blobs they are the concatenation of. Because Blob contents are immutable, it's easy for multithreaded or multiprocess browser implementations to safely share Blob contents in any situation without copying (including, for example, Blobs being passed between Workers via structured cloning). Any Web API dealing with generic data chunks that don't need to be byte-addressable or easily mutable should probably use Blobs, or at least support Blobs as an option.

ArrayBuffer is part of the Typed Arrays spec; it represents main-memory data that is directly addressable and mutable. Because preventing in-memory data races is an important principle of the Web platform, ArrayBuffers cannot be directly shared between Workers. Therefore we added to the Web the ability to transfer ownership of an ArrayBuffer's data between workers (and the main thread) via the Transferables abstraction. An ArrayBuffer is normally represented as a small JS object containing an internal pointer to a (usually much larger) buffer containing its elements. When an ArrayBuffer is transferred from one thread to another, the ArrayBuffer on the origin thread is neutered --- i.e. its length becomes zero; its element buffer is detached and ownership transferred to the destination thread; and on the destination thread a new ArrayBuffer object is created wrapping the transferred element buffer. The contents of the element buffer are not copied.

ArrayBuffer neutering is potentially useful for more than just transferring ownership between Workers. Some Web API methods take ArrayBuffers or typed arrays as parameters, and have to copy the data internally for later use. For example, WebGLRenderingContext.bufferData has to validate its ArrayBuffer parameter's contents and copy the data for later use --- otherwise the caller could modify the ArrayBuffer after the bufferData call has returned, and defeat any validation that was performed. If it's helpful for performance, we can add a version of bufferData that neuters its ArrayBuffer parameter and takes ownership of the data instead of copying it.

Web Audio is an interesting case. Web Audio AudioBuffers contain ArrayBuffers containing audio samples. AudioBuffer samples are made available to an audio processing thread. The question is, what happens if those sample ArrayBuffers are modified by the main thread while the audio processing thread is using them? If the memory is naively shared, the answer is "data races", which are unpopular with many people (including me). One way to fix this without adding copying is to allow an AudioBuffer to neuter its ArrayBuffers, taking ownership of their element buffers and making them immutable so no copying is needed but there's also no possibility of data races.

When designing new Web APIs, if copy avoidance is important, we should keep ArrayBuffer neutering and Blobs in mind as possible solutions.

Movie Picoreviews

I've just returned home from another Bay Area trip for the LEAD program. I also met up with Dirk Schultze and some other Adobe developers for some work on SVG/CSS filters.

Because of the Asiana crash at SFO, my flight to SF was delayed by ten hours and became a daylight flight, so I watched five movies on the way there. I watched two on the way back. The movie haul this month was pretty good actually.

• Life Of Pi: Brilliantly executed but not all that interesting.
• Seven Psychopaths: Slightly above average. A lot more interesting than the title would suggest.
• Stoker: Above average thriller.
• Argo: Pretty good, apart from the gratuitous slander against New Zealand.
• Trance: Somewhat absurd but interesting thriller.
• Side Effects: Pretty good mystery/thriller.
• Dangerous Liasons (2012 Chinese version): Pretty good, but I'm a sucker for Chinese period drama, not to mention Zhang Ziyi and Cecilia Cheung.

Contributing Advocacy

Sometimes we make changes that are strongly justified but inevitably annoy some of our users. The Firefox 22 change to be aware of UI scaling on Windows is a good example. Some of the most helpful contributions I've see from community members recently are patient explanations to complaining users why we made changes and what they can do about it. I'm particularly thankful because interacting with upset users is draining for me --- I need to save my emotional energy for interactions on Web standards mailing lists :-).

The same goes for our volunteers who work with Web developers. Thanks to all of you. Please keep it up!

Gradual Changes Afoot

I've been a manager at Mozilla for several years now. I became a de facto manager by necessity, when we established the Auckland office, and became a de jure manager a little later when the increasing size of the platform team meant we couldn't all report to the same manager. (It's hard now to remember what those days were like!) I've never been passionate about being a people manager, and I've always looked to reduce or eliminate that part of my role, but our pace of hiring always outstripped my ability to send my direct reports to other managers. Lately that's changed, partly because our hiring has slowed in the areas I touch, but also because we have more and better managers in the platform team to take up load. So, the people-management side of my role is finally shrinking. I think this is good for me and Mozilla --- I think my time and attention best serve Mozilla when used for tasks other than people-management.

This means if you're looking for someone responsible for running a Gecko platform team, it's probably not me anymore. For layout, it's Jet and David Baron, for graphics it's Jet and Milan, for media it's Anthony Jones, and for WebRTC it's Maire. Those areas also have their own technical decision-makers.

My decreasing management responsibilities have been offset by increasing responsibilities for leadership in other ways. As Mozilla has grown in size and scope, and my tenure lengthens, I've been playing the "senior figure" role more and more. Sometimes this is nothing more than repeating what a lot of engineers know, but putting my status behind it. Sometimes it's about drawing on broad knowledge of Mozilla and the Web platform to instruct people. Sometimes it means trying to modify or review code no-one else will touch :-). This is all necessary and important, but I'm not completely comfortable in it yet. Sometimes I'm hit by imposter syndrome. Most of the time I'd rather be writing code :-).

Developer Parallelism

In Gecko we've often had N large projects be implemented by just one person each. This has some benefits; in particular it can be quite efficient, since coordination costs are low. It's also a good way to reduce the amount of code learning developers have to do. Another approach is to put N people to work on one large project, and hopefully get it done in roughly 1/N of the time. I think the latter is probably a better way to work. I think we get better reviews, since at least for me, I do much better reviews in code I've actually hacked on than in code I've never touched. It means that if someone gets hit by a bus --- or worse still, leaves Mozilla --- we've got people who we know can work on the code. Projects getting completed faster means there's less bitrot to deal with. It makes us more agile (I hate that word though). Perhaps most importantly, it's more fun.

Most of our projects go through a startup/prototyping phase where we write some new code and get things basically working. Then there's a grind phase where we get the code working on all platforms, iron out the corner cases, fix all the regressions, fix all the fuzz bugs, write tests, and get it landed. The latter phase is usually a lot more work than the startup phase, but it's also the phase that parallelizes across developers fairly easily. So I think we should consciously try to work this way: when a project enters the grind phase, we should shift developers onto it until it's over the line. We've done some of this recently with fixing rooting hazards in JS, and in the layers refactoring.

In cases like OMTC there's some flexibility since getting the project "over the line" isn't all-or-nothing. Having people individually bring up platforms one-by-one works OK and brings incremental benefits. Same goes for converting code from XPIDL to WebIDL. So for those, the benefit of having a lot of people pile on is reduced (especially, in a conversion project, if we know that for some reason or another we will never be able to completely eliminate the stuff we're converting from).

In any case, I hope that from now on, for any project of significant size we have at least two people working on it during the grind phase, to capture the benefits I mentioned above. If someone sees this not happening, let me know and we'll fix it :-).

Meeting Absenteeism

I have a bad habit of missing meetings that I should (and want to) attend. This is inexcusable given that my meeting load is actually very light.

I have identified a few contributing factors. Many of these meetings are outside normal work hours (especially early in the morning, given my timezone), so sometimes I fail to check my calendar before I go to bed and wake up after the meeting. Sometimes I want to wake up but forget to set my alarm, or set it incorrectly. Sometimes the alarm just fails to wake me up. Sometimes I know about the meeting and I'm awake and working, but I get caught up in something and don't realize that the time has arrived. I use Google Calendar and GMail, and when people send time change notification emails Google Calendar adds them as new meetings instead of replacing the existing one, which sometimes confuses me about when the meeting is. I quite often quit and restart my browser, which means my Google Calendar tab often isn't really loaded and can't produce notifications. Most of my meetings are teleconferences of some kind, and sometimes I don't plan ahead and when I try to join the meeting, I find that Vidyo isn't working or some other technical problem prevents me from joining the meeting immediately. None of these issues are excuses, since this isn't a very hard problem.

I could try setting up the FxOS calendar app, but I'm not very vigilant about keeping my phone charged so that might not help much. Maybe I should do that and also ensure that I have a desktop calendar app capable of producing audible notifications, that launches on startup. To some extent I may just have to be trained by pain and humiliation.

Until that happens, consider this blog post an apology for the meetings I've missed, and an apology in advance for the meetings I miss in the future. Also, consider this a blanket permission and indeed recommendation to contact me by whatever means necessary when I've agreed to a meeting but I'm not there --- IRC, email, cellphone, even my home phone. The numbers are in the Mozilla phonebook.

Blink, PNaCl, And Standards

When the Blink fork was announced, it came with an admirable commitment to good citizenship when adding new Web-exposed features. Unfortunately, Google has recently announced that they will expose Portable Native Client (PNaCl) --- and by extension, the Pepper APIs --- to all Web pages, a move which clearly runs counter to Blink's commitments.

PNaCl is an execution framework that lets Web developers compile code in various languages using LLVM-based compilers to LLVM "bitcode", which is then served to clients (i.e. Chrome) which compile it down into native code. To be useful, code needs APIs to interact with the outside world (e.g. graphics APIs), so Google created the "Pepper" APIs for NaCl/PNaCl applications to use. Let's see how PNaCl and Pepper stack up against the Blink principles:

• In practice, we strive to ensure that the features we ship by default have open standards.

  PNaCl and Pepper are not open standards, and there are not even any proposals on the table to standardize them in any forum. They have documentation, but for the details one must defer to the large bundle of Chrome code that implements them. Other vendors wishing to support PNaCl or Pepper would have to adopt or reverse-engineer this code.

• Factors that decrease compatibility risk (in rough order of importance):

  • Other vendors shipping compatible implementations
  • A mature specification in the relevant standards body
  • Positive signals from other browser vendors
  • A small API footprint

  PNaCl and Pepper satisfy none of these. In particular, there are no positive signals from other browser vendors; in fact all the signals I've detected have been hostile.

• The following tiers are good rules of thumb to know that the feature is on the right track (ordered by increasing risk to compatibility and therefore decreasing order of desirability):

  • Two other browser engines already ship roughly interoperable implementations in stable or experimental channels. In this situation, the feature is already a de facto standard. If a de jure standard does not yet exist, we should help create one.
  • One other browser engine ships a roughly interoperable implementation in a stable or experimental channel, we believe the feature to be stable, and we’ve consulted with the appropriate standards body.
  • The appropriate standards body considers the feature ready for implementation. For example, the W3C issuing a Call for Implementations or publishing a Candidate Recommendation of the feature would meet this guideline.
  • The specification for the feature has been accepted by the appropriate standards working group (e.g., a First Public Working Draft in the W3C) and we’ve received positive feedback from other browser engines about the feature’s feasibility and value.

  Again, PNaCl and Pepper satisfy none of these.

• In extremely rare cases, we may enable a feature by default before the feature's compatibility risk is as low as we'd like. In such cases, we will meet the following requirements:

  • We will propose an editor’s draft (or equivalent) to the relevant standards group.
  • We will discuss the feature publicly with implementers of other browser engines.

  Further, we will take on an active commitment to shepherd the feature through the standards process, accepting the burden of possible API changes.

  Again, none of this is happening for PNaCl and Pepper.

• [Regarding the Blink "Feature Dashboard"] We associate each value with a shade of red or green, corresponding to how the value reflects our web citizenship. For example, “opposition from another browser vendor” is red and “a similar implementation in another browser” is green. Viewed in aggregate, these colors provide a quick snapshot of the project’s overall web citizenship.

  PNaCl and Pepper aren't on that dashboard, but if they were, they'd be a very, very bright shade of red.

The disconnect is alarming. Unfortunately it appears Blink's principles only apply to Blink, not Chrome as a whole. I'm sad, because this seriously undermines the value of the Blink team's good intentions; a Google team that doesn't want to be a good Web citizen can probably find a way to be "not Blink" and run roughshod over the Blink team's good work :-(.

Taiwan Travelogue

In Taipei we had probably the best Gecko work week I've ever been to. It was also, for me, the most draining. Apart from getting the regular Gecko teams together for work and fun, we also got to meet many of the new Mozilla developers in Taipei. This was very helpful and a lot of fun. However, as a senior Mozilla figure I felt a lot of responsibility to educate and generally facilitate the interactions of all our teams, 24/7, and since I'm not naturally gregarious, my social skills were plain worn out by the end of the week. It was exacerbated by the sheer number of people, and the cultural and language barriers. I also have to struggle against the urge to strut before Chinese women. Fortunately I have considerable experience dealing with these issues, otherwise I probably would have imploded.

That aside, it was all very very good. The food was amazing. The breakfast at the Grand Hyatt is the best hotel breakfast I've ever seen, by some distance. The Chinese banquet dinners were over-the-top great (bar the shark's fin --- gah). The mango dessert in the Taipei 101 basement food court --- mango chunks, shaved ice, condensed milk, brown sugar syrup, and mango ice cream --- was so good I had it twice. The extracurricular activities included visiting the National Palace museum, board games at night, and two trips into the hills. The day after I arrived, Sunday morning, I completed my first ever organized running event, a 10km trail run in the hills on the north side of the city, with a few other Mozilla staff. It was hot and humid, but the field was pretty weak and mostly walked up the steep bits, which worked to my advantage since I have long legs and walking fast uphill is my specialty, so I came in 18th out of 88 male-over-40 finishers. On Saturday morning, just before we left, a group of Mozilla people went for a hike to and beyond Elephant Mountain. That was lots of fun.

During the week, I led two sessions not related to specific technical issues: one was on "Mozilla culture" --- what we do, why we do it, how we should change it, specifically targeted at people relatively new to Mozilla, especially those from companies with very different cultures. Part of that was about code review, and stimulated another session specifically about code review, and how to do it better. I have videos of those sessions, and once I've scrubbed them to remove one or two unfortunate comments I'll put them online :-).

It turns out that currently the only place you can buy official Mozilla merchandise is the Taiwan office's online store. Anticipating this, I brought a half-empty suitcase and placed the biggest order the store had ever seen. Now I have a stockpile of gifts for friends and family that should last a while :-). We desperately need an international Mozilla gear store, though.

Since I'd seen most of the good Air New Zealand May movies earlier this month, the plane movie selection was a bit random...

• The Last Stand: Schwarznegger vehicle, so I had low expectations, but they were exceeded. Not a great movie, but not a bad one.
• The Assassins: Borderline incomprehensible Chinese movie. I missed being able to pause the movie and ask my wife what on earth is going on.
• Taken 2: Such a bad movie. Utterly boring and predictable, none of it makes any sense at all, and embraces a number of common movie failures: classic combination of the bad guys being both supremely brilliant and completely stupid; classic assassin-idolization main character who kills people without feeling but is also a really great guy; classic divorced couple who are so right for each other it's incomprehensible why they got divorced in the first place. Avoid.
• Remembering 1942: Really really good. Got mixed reviews, but I think it's a great movie. It's a litany of human suffering, set in the Chinese province of Henan, trapped in the vice of war and famine. Watching movies like this periodically helps me keep a sense of perspective.
• Ripper Street (series): Victorian police procedural. I had low expectations, but was actually quite impressed. Conveys a real sense of the times --- and they were interesting times indeed.

It was a great decision to have the work week in Taiwan, and I hope we get back there soon.

Travel

Last week I was in California. It was my first time in the Mozilla SF office --- lovely view of the Bay from the roof. I always enjoy the free snacks and I'm always glad we don't have them in Auckland. I spent quality time with some of the people I know and love at Mozilla, and that's always exciting.

On Wednesday and Thursday I was at Half Moon Bay doing LEAD training. It was fun, but thinking about "soft skills" for two days straight is quite draining for me; my social skills are learned, not innate.

This cohort is different from previous cohorts --- most members are relatively new to Mozilla; of our cohort, Vlad and I have been at Mozilla the longest, by far. This gives me the honor and duty of representing the Mozilla old guard. I feel the power of the narrative that has me in the "crusty old engineer, harping about the old days and resisting change" role ... and I do my best to reject it :-).

One of the results of LEAD so far is that I perceive my relationships with other Mozilla staff to be warmer and stronger than they perceive them, on average. I suspect this may be related to the difficulty of maintaining deep relationships with remote employees I see a few times a year at best. I'm still trying to figure that out.

Plane movies:

• Gangster Squad: Genre flick. OK.
• Zero Dark Thirty: Pretty good. Not exactly entertaining, but interesting.
• Live And Let Die: Some kind of cross between a Bond movie, a blaxploitation flick, and the Dukes Of Hazzard. Odd.
• The Town: Genre flick. Slightly better than average.
• I, Anna: Sort of noir-ish psychological thriller. OK.
• Les Miserable: The movie of the musical. Pretty good. I need to read the book sometime.

Interestingly, Air New Zealand lets you see what movies they're showing on their routes. This Web interface is a pretty faithful mockup of the actual in-seat interface (which is pretty bad ... it would be great to be able to see more than one movie title at a time).

On Friday I leave for Taiwan for a week at the Mozilla office, a "Web rendering" work week. This should be even more fun than last week.

The Direct Route

Over time I've become increasingly impressed with the broad applicability of Matthew 18:15-17:

If your brother or sister sins, go and point out their fault, just between the two of you. If they listen to you, you have won them over. But if they will not listen, take one or two others along, so that ‘every matter may be established by the testimony of two or three witnesses.’ If they still refuse to listen, tell it to the church; and if they refuse to listen even to the church, treat them as you would a pagan or a tax collector.

The first step is often difficult but crucial. The path of least resistance can be to go behind your antagonist's back --- to your friends, or their friends, or their manager. I've seen all kinds of negative consequences from following that path --- hurt, distrust, unnecessary escalation, confusion and fear. I feel my integrity depends on people knowing that whatever I say about them to others, they will not be surprised by because they've already heard it from me.

This applies in the other direction too, when people complain about third parties to me. If the third party is unaware of the issue, I don't want to know --- go away and talk to them first.

There are rare exceptions, usually involving time-critical emergencies or complex secrecy requirements.

Web Audio Progress

Our Web Audio implementation is making great progress. This is mainly due to the efforts of Ehsan Akhgari, who is, astoundingly, cranking out one or two features per day. Paul Adenot and I are spending hours every day just reviewing his code. I think this is partly due to Ehsan and I laying down some pretty good infrastructure at the outset.

Our current goal is to have a basically complete implementation for Firefox 24, which branches from trunk in about eight weeks. There are a few things we need to do to get there:

• Complete the feature set. At this point that mainly means adding all the node types that aren't implemented yet: MediaStreamAudioDestinationNode, MediaStreamAudioSourceNode, MediaElementAudioSourceNode, ConvolverNode, OscillatorNode, and WaveShaperNode. The first three are all related and shouldn't be too hard since we designed Web Audio from the start to share infrastructure with MediaStreams (which are already integrated into media elements) --- internally, a Web Audio node is just a special kind of MediaStream. We still need to implement HRTF and soundfield panning modes for PannerNode. We need to implement OfflineAudioContext. For some of the audio algorithms that aren't very well specified, we're borrowing code from Blink. This is suboptimal but there's ongoing discussion about what level of detail we should specify the audio algorithms at.
• Work on latency. Right now audio output has pretty bad latency, especially on Android, FirefoxOS and Windows; the biggest problems are intrinsic issues with the platform APIs we're using. On older versions of Android and Windows XP it may be a lost cause, because good APIs simply aren't available. For Windows Vista and up we're writing a new audio output backend using WASAPI. On FirefoxOS we may rip out the Android code we're using and replace it with PulseAudio. There is additional work to do to better integrate the MediaStreamGraph (that drives MediaStream and AudioNode processing) with our libcubeb audio backends for lower latency and better tracking of the audio clock. This latency work is desperately needed for WebRTC as well as Web Audio.
• Work on throughput. Right now we're focusing on having a good clean design and functional correctness. For example, all communication and synchronization with the MediaStreamGraph real-time processing thread is asynchronous, using message passing. Updates to Web Audio and MediaStream graphs are batched so all changes performed by a script happen atomically on the real-time thread. But we haven't done any profiling, tuning, or optimization of the actual processing code. In particular we'll clearly need SIMD implementations of basic audio primitives such as mixing to get near-maximum performance, especially on mobile.
• Test and fix bugs, needless to say.

Contributors welcome! The Web Audio bug (779297) has a lot of dependencies to choose from :-).

One interesting issue that Jean-Marc Valin brought up recently is the prospect of a loudness war on the Web. Some areas, such as Sony's Playstation products and some broadcast TV regions, are trying to mitigate the loudness war by standardizing acceptable dB levels for all content. It might be a good idea to do this on the Web too, and have browsers automatically limit the volume of content that exceeds those levels. We're still thinking about whether and how this should be done, and talking about it on public-audio.

Rotorua

We had a lovely few days away. On the drive to Rotorua we stopped at Waihi and rode the old railway up to Waikino in an open carriage. Later we stopped at Mount Manganui and walked to the top --- always a great view.

We stayed by the lake at Waiteti, just out of town, at Waiteti Lakefront Motel. I think that was a pretty good decision; very peaceful, great view, relatively cheap, and free use of kayaks which we took advantage of on Saturday morning, paddling up the Waiteti Stream a bit.

On Friday morning we did the gondola and luge ... especially the luge, which everyone enjoyed. After that we visited Kuirau Park, which I always enjoy a lot --- it's so great to have a geothermal park right in the city. Then we went to Waimangu Valley, which I still think is the best geothermal attraction in the area because of its unique size and features, and its brilliant bush and lake setting. It's changed noticeably since I was last there; Warbrick Terrace is growing quite rapidly.

After kayaking on Saturday we went to the Redwoods for a short walk. I hadn't been there before and it's really great --- not as picturesque as some other places near Rotorua, but large and lovely, especially I imagine for running and biking. Then we went to Lake Okareka and did the new walkway there. Brilliant.

On the way back to Auckland yesterday we stopped near Putaruru and did the Te Waihou walkway. The Waihou river is stunningly clear and beautiful due to having flowed underground through filtering rock in the Kaimai Ranges (for fifty years, supposedly). We saw a lot of trout, but a couple of guys fishing complained they hadn't caught any because the clarity of the water made it too hard to estimate depth!

All in all, another wonderful central North Island getaway. I love this place.

Hunuas Tramp

We drove out to the Upper Mangatawhiri Dam on Sunday afternoon and took a couple of hours to get to Piggotts campsite. It's a pretty small patch of flat grass, so I don't know how you'd get 20 people camping there, but whatever. There's also a small hut which I haven't seen mentioned anywhere. In fact we passed by it, thinking that couldn't be the place, but quickly turned back when it became clear that it was, in fact, the place.

A thunderstorm passed over during the night. It was the first time any of us have been in a tent during a thunderstorm, so that was exciting for everyone.

The next day we got up early and hit the trail about 8:15am. Based on the warnings about central Hunua tracks being "for experienced trampers only", and that we hadn't ever hiked a long distance carrying all our gear, I wanted to give us the maximum possible time to get to Adams campsite. As it turned out even the warning-est track (Upper Mangatawhiri) was just fine. Less fine was that the Adams campsite was a lot further along the track than where my map shows it, and when we finally found it at about 3pm, it looked pretty awful. The ground was waterlogged and, while there was a sign saying "toilet", there wasn't one to be seen. We decided to skip it and push on to our final destination --- Waharau Regional Park. (Now that I've looked a bit closer online, it seems Thousand Acres campsite is a better bet ... but it's not even on my map. Lesson: use up-to-date maps and do more research!)

So our three-day tramp turned into a two-day tramp and we arrived at Waharau about 5:15pm after nine hours of tramping, covering about 18km of reasonably rugged terrain with all our gear. Everyone was tired but I'm proud of my kids for handling it very well.

Upcoming Travel Plans

If the weather forecast isn't too horrible I'll be away the coming Sunday afternoon to Tuesday morning tramping in the Hunua hills southeast of Auckland.

Thursday is Anzac Day, a public holiday, and I'll be taking Friday off to go on a holiday with my family. So I'll probably be in the office/online only on Wednesday next week (that's Tuesday for those of you on the wrong side of the date line).

About a week later, Sunday May 5, I have a flight to San Francisco. I'll be in the Bay Area on Monday and Tuesday, and at LEAD training on Wednesday and Thursday.

About week after that, Friday May 17, I'm flying to Taiwan for a Mozilla layout/graphics/media work week.

Fun times!

Google Vs New Zealand

The head of "Google NZ", Tony Keusgen, has been talking about New Zealand's shortage of "IT experts". And apparently, unlike most people who complain about it, he's actually tried to do something to help. That's great.

However, what's not so great, and of course not mentioned at all in the article, is that Google itself is contributing to the problem in a significant way by aggressively recruiting for NZ developers to move to their Sydney engineering office --- since they don't have an engineering office in New Zealand. That works for Google, for their recruits, and even for me and Mozilla, since there's basically no competition to Mozilla for NZ developers who want to do the sort of platform engineering work we do. It sucks for New Zealand though.

It's too bad the writer, Ben Chapman-Smith, didn't know about or declined to mention this. And if no-one called Tony Keusgen on it, that's bad too.

I would love to see a real Google office here. Some pretty high-profile Googlers are NZers. Make it happen!

The Chris Problem

I just sent a message to public-audio replying to Chris Rogers, Chris Pike and Chris Lowis. Fortunately Chris Wilson wasn't involved in the thread, although he could join at any moment.

It reminded me of a Mozilla meeting discussing multi-process video playback, which I attended with Chris Double, Chris Pearce, Chris Jones and Chris Blizzard.

Something is deeply wrong here.

The Future Of Cheating

I went to an orienteering event on Saturday. Orienteering is a race through a set of waypoints on a map; what sets it apart from a regular running race is the need to navigate via the map. Obviously it would be easy to cheat with a smartphone. Currently it's not hard to ensure people don't carry or use smartphones during the race. However, it's going to get increasingly difficult to detect cheating as technology progresses into smartphone watches, Google Glasses, and ultimately body implants. It doesn't seem practical to use X-rays or MRI scans on every participant, even if that would work.

Similar cheating problems are already happening in chess, and in written exams. Casinos grapple with these issues too.

It seems likely that any competitive event where real-time computer assistance (including communication with other humans) would be useful will sooner or later become practically impossible to secure against cheating. This will have profound effects. Even when people choose not to cheat, not knowing whether the game is fair is corrosive. Enjoy competitive games while you can.

This is going to be a problem for job interviews too.

Blink

I blogged last time the browser engine count changed, so I think to be fair I should blog about this one too.

I think this is good news for the Web. When we lost an engine, that was bad for engine diversity and the open Web. Now we're gaining one, and that's good for engine diversity and the open Web. It's not a direct reverse of the Opera situation though. Compared to Presto, Blink will have much more market share, and therefore a bigger impact. On the other hand, Blink is a promise to deliver a different engine over time, not a new engine right now, and even in the long term one presumes it will share a lot of code with Webkit --- especially if people port code between the two projects. So it contributes less diversity than Presto did.

One immediate benefit for the open Web is that today it's difficult for mobile Web developers to advocate "just coding to Webkit" as the way forward.

One issue we'll need to figure out in standards groups is when a Webkit implementation and a Blink implementation of a feature count as two independent implementations to allow a spec to proceed to REC. It's tough; even if the implementations are developed independently, the shared context (both code and architecture) will make the implementations less independent than they should be. We may need to address this on a case-by-case basis.

I am a bit worried about how this will impact Apple's support for the open Web. Safari has lagged behind other browsers in various ways, but they were also dragged along by Google to some extent in supporting new features whether they cared or not. Now they're falling completely off that train and we'll see a true test of whether Apple is willing to step up their investment to keep up with the progress of the open Web. Apple has obvious incentives to hobble Web apps on iOS, so I worry.

Of lesser importance, this is probably good for Mozilla. Even if it means Chrome moves faster, the fracturing of the Webkit community benefits us. It's not something to gloat over though.

Back to work!

Update One other thought occurred to me: now, a company adopting Webkit has to choose between a fork dominated by Apple and a fork dominated by Google. That's going to be less appealing than when Apple and Google were providing some counterweight to each other.

Mitigating Control-Flow Exploits With x86 ISA Extensions

A lot of exploit styles (e.g. "return-oriented programming") rely on jumping or calling into code that was never meant to be executed "stand-alone" --- i.e., jumping to an instruction that was only ever supposed to be executed by falling through from the previous instruction or via a branch within its function. On x86 this includes destination "instructions" that are actually part of another multi-byte instruction. It seems to me these exploits could be made much harder by somehow marking the start of "valid" branch/call/return targets and faulting when control is transferred to other instructions inappropriately.

Here's a more specific proposal for x86:

• Add a new flag bit for code pages to enable "destination checking" on a per-page basis.
• An indirect control transfer is any return instruction that pops EIP off the stack, or any jump or call instruction whose operand is not a constant. When an indirect control transfer jumps to a page marked for destination checking, and the instruction at EIP is not 0x90 (NOP), fault.
• A toolchain would take advantage of this feature by marking code pages for destination checking, avoiding use of 0x90 for regular NOPs, and placing a 0x90 at every function entry point and after every call instruction. For bonus points, avoid generating 0x90 bytes inside other instructions.

Obviously there are a lot of ways to tweak this. For example "PUSH EBP" is very often the first instruction of a function, so you could whitelist that instruction as a valid function entry point. You could avoid having to place a NOP after a direct CALL instruction by checking if the instruction at EIP-5 is 0x9A (direct CALL). You could make false returns even harder by extending that special case to require the address we're returning from to be "close to" the destination of the direct CALL instruction.

This is obvious enough that I presume someone's worked on it already.

RIP Crazy Noodle

For several years Crazy Noodle Bar was one of our favourite restaurants in Newmarket. It served cheap tasty Hong Kong cafe food --- a mix of Chinese dishes and Chinese-style Western dishes. Last year it went on hiatus for six months and moved to another building a few blocks away. When it reopened, the menu quickly evolved away from its previous incarnation although there was still a resemblance and it was still pretty good.

A more disturbing change was the new decor, whose distinguishing feature was retro posters of Chairman Mao. He's an iconic figure in China, but he's also one of the top three most murderous dictators of the 20th century, so not someone I want to see celebrated. I put up with it.

It got worse. The staff showed up with Chinese military-ish uniforms, green with red epaulettes. Then last week, the name changed to Red Guard Noodle Bar. This is really the last straw. The Red Guards were Mao's vanguard of the Cultural Revolution, which was an awful and disastrous episode of Chinese history. I don't know what the proprietor of the cafe is thinking, but I'm not going back.

As an aside, I don't know why this isn't a scandal. Imagine the outcry if someone opened a "Stormtrooper Cafe" decorated with swastikas and pictures of Hitler and Röhm. I suppose it's because Mao mostly restricted his brutality to his own citizens.

Getting The Facts

I'd feel worse about the decline of traditional media if it wasn't so disappointing when they write about issues I have first-hand knowledge of.

Today's example is a story about the lack of pedestrian access from Gilles Ave to Broadway under the new Newmarket Viaduct. The article records complaints from Cameron Brewer and Andy Smith about how the project has neglected pedestrians. Reading it, you'd think a walkway would be a real boon for pedestrians accessing Newmarket. But a glance at a map shows the truth: every route along Gilles Ave to the under-Viaduct area must pass Alpers Ave or Mortimer Pass, both of which also provide direct access to Broadway. Therefore a walkway under the Viaduct would shorten the journey to and from Newmarket for approximately zero pedestrians. (I say this as someone who walks to work in Newmarket every day, often along Gilles Ave, and could actually use this walkway regularly.)

I'm ambivalent about whether this walkway should be built. On one hand it would be nice to have and I've been looking forward to it. On the other hand perhaps our money could build a new pedestrian route that's actually useful. But the quotes from Brewer and Smith, and the tone of the article, are overblown and misleading, and the introduction of a few geographic facts would have cleared things up. I won't mourn the loss of journalists who can't or won't do this.

Why I Work

Recently Pascal Finette quoted Tom Chi as saying:

When you come home from your job in the evening and pretty much all you want to do is slouch on the sofa, watch a movie and have a beer: Quit your job.

There have been many times when that's all I've wanted to do. Does that mean I should quit my job? No.

Before I worked full-time on Mozilla code, my job was computer science research. I'd have to say that was more fun and intellectually stimulating than my job at Mozilla. Does that mean I should quit my job? No.

I work at Mozilla because I think God gave me talent to use for the maximum good of all, not for self-gratification. I work at Mozilla because it's the most important work I can imagine doing. And it's not just me; one of the great things about Mozilla is that many, if not most, of its people have a similar kind of motivation. Mozilla is full of very smart people who have spent years dealing with difficult and frustrating problems, often turning down lucrative offers to do other things, because they believe in what we do. I love them for it.

It would be nice if important tasks were always exciting, energizing and exhilarating. But real life just isn't always like that. Making Flash less painful for our users, or tracking down bizarre graphics driver bugs, or struggling to convince megacorporations to do the right thing, is just draining.

However, it's important to keep a sense of perspective. Most of us are paid ridiculously well to do a job which is generally less stressful or challenging than many other kinds of work. If we make a few small sacrifices, no-one should be overimpressed.

Seeking Relevance

In the media scrum around the papal election, a lot of commentators called for the Catholic church to "modernize" to "stay relevant". As a Protestant I agree my Catholic brothers and sisters could make changes for the better, but "modernizing to stay relevant" has often been a siren song that has, and still does, lead many Christian groups to heresy, torpor, and --- ironically --- irrelevance.

The problem is that calls for "relevance" tend to promote the erosion or rejection of tenets of the historic Christian faith --- such as belief in miracles, divine judgement, the authority of scripture, or even the existence of God himself --- in an effort to make Christianity more palatable to modern people. Many Protestant denominations have wholly or partly gone down this path, including New Zealand Presbyterianism of which I am a member. The effects have been disastrous along many axes. Many congregations have ceased to be Christian in all but name, and are incapable of carrying out God's work in the world. By refusing to officially apostasize, they create vast confusion over what Christianity means and (further) undermine the unity that Jesus' followers should have. And instead of making Christianity more appealing, this drive for "relevance" ultimately kills the congregations that follow it. It's easy to see empirically: unbelieving congregations tend to age and dwindle, and healthy and growing congregations tend to be more orthodox. These trends make perfect sense: if a congregation loses its "saltiness" and becomes a secular club with Christian trappings, why would the average person prefer it over another secular club without the tomfoolery? Not to mention that there can be no evangelism, no spiritual regeneration, and no working of God beyond common grace.

The institutional hegemony of the Roman Catholic church has a lot of downsides, but it has enabled them to mostly avoid going down this path --- for which I am grateful. I pray that they'll continue to ignore misguided calls to change course.

For the church to stay relevant, we must keep teaching the classic gospel of Jesus and keep making disciples in his name. To increase our relevance we need do nothing new, but do the old things better: more Christian unity, more evangelism, more faithful living, more prayer, more love. This won't happen with sweeping pronouncements, election of Popes, or even writing of blogs, but as individual Christians quietly choose to put Jesus ahead of themselves every hour of every day.