Stepping Stones

Last Sunday I spoke to our church congregation about a few things that had helped me grow spiritually. Many of those things were books. Here's the list:

• The NIV Study Bible. This is the Bible I bought when I first became a Christian and I've mostly relied on it since then. The notes helped me understand a lot, especially in the beginning. The notes cover a lot of ground: translation issues, interpretations, and controversies. They're pretty good about presenting multiple sides of controversial issues in a fair way.
  
• Basic Christianity by John Stott. This was one of the first Christian books I read. It's a particularly good introduction to what Christianity is all about, for non-Christians and Christians alike. It focuses strongly on Jesus: who he was (and is), why he came, and what that means for us. Highly recommended.
  
• The Screwtape Letters by C.S. Lewis. A senior demon advises a minor demon on how to handle his “patient” --- a new Christian whom the demon has been assigned to corrupt. This is a very fun read, with lots of useful insights into the Christian life. Phillip Pullman likes it, so it must be good.
  
• The Great Divorce by C.S. Lewis. An imaginary tour of heaven and hell --- not at all intended to be taken literally, but to stimulate one's thinking.
  
• The Knowledge Of The Holy by A. W. Tozer. Much more dry than the above, but lays out key truths about God and strongly encourages the reader to take them seriously. Tozer emphasizes God's ineffability, and acknowledges the tension this creates for his own book!
  
• Jesus by A. N. Wilson. A portrait of Jesus written from a skeptical/secular point of view. I read it quite soon after I became a Christian. It's interesting because it's clear from the book Wilson was fascinated by Jesus --- maybe even entranced --- even though he didn't really believe in him. I wasn't all that surprised when Wilson recently become a believer.

Bye Bye HWNDs

As Firefox 4 races towards completion, we are discovering that quite a few extensions, pointing-device drivers and accessibility tools have been broken by the fact that Firefox no longer creates child HWNDs for its internal use.

These child HWNDs were been removed as a necessary step towards providing the best possible graphics acceleration performance in Firefox 4. Supporting child HWNDs was also architecturally painful; removing them has let us simplify or remove a lot of code, and we'll be able to remove a lot more later. It also fixed many bugs, and helped us to smoothly support new UI features such as Firefox UI element layered over Web page content. These HWNDs are not coming back.

It's unfortunate that various third party software components were messing with our internal HWNDs, since they were never supported API and we've been gradually removing them over a long period of time. But I suppose being molested is the price of popularity...

We're doing a few different things to deal with the fallout. We have some hacks in place to keep ATs working (fortunately ATs mostly use interfaces that do not depend on HWNDs), plus we've worked with AT vendors directly to fix issues. We have some more hacks in place to deal with pointing-device drivers, and we'll likely have to add even more hacks for them :-(. For extensions, we probably won't do anything, since the things they've been doing --- e.g. inserting a native child window under the HWND for our sidebar, or even Web content --- are mostly not things we want to support directly.

If you maintain a Firefox extension that is broken on Firefox 4 because it has native components relying on HWNDs that no longer exist, your best solution depends on what your extension actually does. But there are two main approaches you can take:

1. Give up using native HWNDs and use Web platform features instead. This is the most future-proof, most portable and often the most performant route. Firefox 4 has many new features to handle use-cases the browser couldn't support before: audio synthesis, GPU-accelerated 2D and 3D graphics, touch input, etc.
   
2. Wrap your HWND-using code in an NPAPI plugin. If you have large amounts of legacy HWND-dependent code that you can't rewrite, or there's platform functionality that you still can't get without an HWND, you may be able to wrap an HWND up as an NPAPI plugin, then you can insert a plugin instance into Web content or browser UI. NPAPI plugins are a bit of a pain to work with, but this might be a practical approach for some situations.
   

Good luck!

Hopefully now that we're down to a single HWND (most of the time), we'll see less breakage in future releases simply because there's little left to abuse or change...

Distrusting Research

Here's an interesting article about bias in medical research. Almost everything they say about medical research applies to computer science too.

So you have to be very skeptical of empirical results in computer science, especially "we tried this and it worked" results. Fortunately --- unlike biology, I suspect --- at the heart of many computer science papers is just a clever idea or observation someone came up with. You can take that idea and try it for yourself. But don't bet the company on it just because some researcher said it worked great.

Of course, the situation is still pretty bad overall. The sick research culture that only rewards positive results doesn't just create selection bias in reported results; it also deprives us of interesting data about ideas that don't work. After all, people only try things they think will work, so failures on average should be more surprising --- and therefore more interesting --- than successes.

At PLDI this year not a single paper reported a negative result. However, during the short talks session one group from Microsoft presented a cute piece of work applying an automated reasoning tool to perform compilation; nice idea, but it didn't really work. I was so excited I ran up to the microphone just to say "thank you for presenting a negative result!". Then I added "We need more results like this", and everyone laughed...

Mitigating Dangling Pointer Bugs Using Frame Poisoning

The most important data structure in Gecko layout is the frame tree. Roughly speaking each frame object corresponds to a CSS box. The lifetimes of these frame objects are explicitly managed, and the structures get fairly complicated, so we sometimes have bugs where there is a dangling pointer to a frame object after it has been deleted. These bugs often used to be exploitable security bugs, since making a virtual call through a pointer to a deleted object can often be used to take control of the process using heap spraying. We fixed these bugs as fast as we could, but even after fixing all the ones we know about, experience suggests there are additional dangling frame pointer bugs we don't know about yet. So last year we implemented a mitigation technique that makes it impossible to exploit almost all bugs involving dangling pointers to frames. We call this technique frame poisoning. It incorporates some ideas that I first saw presented in a paper Memory Safety Without Runtime Checks or Garbage Collection. Most of the work was done by Zack Weinberg, who is now a PhD student working with Collin Jackson at CMU West.

Exploiting dangling pointers is essentially about exploiting type errors. An attacker writes an attacker-controlled data value to some memory location L in a deallocated object, typically by causing a new object to be allocated there. This data value is invalid for the type T the program expects when it accesses L through a pointer to the deallocated object. For example, the program may expect L to be a pointer to a C++ vtable, but the attacker has overwritten L with an integer --- which when interpreted as a vtable pointer, lets the attacker trigger arbitrary behavior.

Our approach is to prevent the first phase of the attack by making it impossible to overwrite fields of deallocated objects with values of the wrong type. We do this by ensuring that whenever the memory used by a deallocated frame object is reallocated to a new object, the new object must always be exactly the same type as the old object and at exactly the same address. Thus whenever code writes a value to a field of the new object, the value must be valid for the type T the program expects when it access that location through a pointer to the new object *or* the old object. Thus, dangling-frame-pointer attacks cannot get started.

This is implemented by keeping a "free list" of deallocated frames, one freelist per frame type. When a frame is deallocated, its memory is added to the freelist for its type. When a new frame is allocated, we check the freelist for its type and reuse memory from that freelist if the freelist is not empty, otherwise we allocate new memory for the frame.

One complication is that we actually keep one set of freelists per document (i.e., Web page). When a document goes away, all frames are destroyed and all frame memory is returned to the general application heap. This is safe because almost all persistent pointers to frames are from data structures associated with that document, data structures which are torn down when the document goes away. Thus it is very unlikely we will see bugs involving dangling pointers to frames whose document has gone away (experience bears this out).

An additional line of defense is that when frames are deallocated we fill the memory with a "poison value" --- a nonsense value which, when interpreted as a pointer, always points into a large region of invalid memory. This ensures that if the program loads a value from the memory of a deallocated object and dereferences that value as a pointer, it will almost certainly crash in a way that cannot be exploited. In practice this means when someone finds a bug involving a dangling frame pointer, the browser usually crashes immediately before doing anything an attacker would find interesting. "Frame poisoning crashes" are easy to identify because they're accesses through a pointer that is (or is close to) the "frame poison value". For more details see TestPoisonArea.

We didn't use NULL (zero) for the poison value because a lot of code tests to see if a pointer is non-null, and if so dereferences it. If that code operates on a deallocated object, we want it to crash immediately, not carry on trying to use the deallocated object in some other way.

We shipped frame poisoning in Firefox 3.6. In the end there was no significant performance impact, although we had to be careful because some of our earlier attempts did hurt performance! In particular, we optimize by not writing the poison value over deallocated frame objects when the entire document is going away.

Like all mitigations, frame poisoning is not ideal. We still fix dangling frame pointer bugs on trunk as quickly as we can. However, we know of no way to bypass frame poisoning to exploit dangling frame pointer bugs, so we may choose to not fix certain bugs on a maintenance branch where we believe frame poisoning blocks exploitation and a proper fix would be very destabilizing.

South Island Trip

I'm well and truly back again! Our holiday around the northern end of the South Island was wonderful. One of the best things about living in New Zealand is that it's easy to take holidays around New Zealand.

We flew to Wellington, took the ferry to Picton for a night, then took the TranzCoastal train to Christchurch via Kaikoura --- amazing views of the Kaikoura Ranges and the coast. We crossed over large cracks in the ground near Kaiapoi showing exactly where the recent earthquake had hit. We had a couple of nights in Christchurch; during our full day we walked from Cashmere up into the Port Hills, then east along the Crater Rim track for a few hours before descending into Lyttleton on the Bridle Path --- fantastic 5-6 hour walk with amazing views west to the Southern Alps and south into Lyttleton Harbour and Banks Peninsula.

From Christchurch we took the TranzAlpine train up to Arthur's Pass, where we spent a couple of days. The first day was wet so we walked up the Devil's Punchbowl falls, which were a raging, spraying torrent. Awesome! The next day our big walk took us from the village along the Bridal Veil track, then up into Bealey Valley below Mount Rolleston, then back to the road and along the road to the Dobson Nature Walk before returning back down the road to the village. We saw lots of keas --- wonderful alpine parrots --- and the alpine/bush environment is stunning with snow reaching nearly down to the village.

We took the train the rest of the way to Greymouth, picked up a rental car and drove to Murchison via the Pancake Rocks at Paekakariki and the Buller Gorge (both spectacular). Our day trip from Murchison was to Mount Robert at Lake Rotoiti in Nelson Lakes National Park; another 5-6 hours up, across the top (with lots of snow), down, and back to the carpark. The weather was perfect, the view staggering; we could see at least 100km in most directions. We could easily see all the way to Motueka on the coast. Bush, snow, sun, lake, staggering views --- amazing.

From Murchison we drove via Motueka to Takaka for the last three nights of our trip. The highlight of this section was the limestone country and exploring the caves at the Aorere goldfields and Rawhiti Cave. (Note: documentation says that you can drive a car up a "farm track" to a parking area for the Aorere goldfields walk, but the "road" is very rough. I suggest parking at the first gate and walking the rest of the way.) The initial descent into Stafford's Cave was slightly tricky, given I have no real climbing/caving experience and had children to assist, but then (having a torch) it was easy to make our way through a series of chambers to an area with glow-worms. Wow! We also drove up to the base of Farewell Spit and did a short 1.5 hour walk there; it looks worth spending more time there.

On our last day we drove from Takaka to Nelson airport to fly back to Auckland. Along the way we visited the Stoneridge Cafe and had great fun in their hedge maze.

It was an amazing, wonderful trip. We had half a day of rain in the eleven days we were away, which is completely unreasonable for this time of year. Consequently we did more walking than I had expected, but we made up for it with more eating! Everyone raves about the South Island's scenery, but the superlatives are fully deserved. As a bonus, the areas we covered are very compact; apart from the TranzCoastal train, each step of our trip was just two to three hours of travel.

Accommodations and food were great. All the motels were more than adequate, but Golden Bay Motel in Takaka was exceptional. Even where there were only a few options to buy food, the food was excellent (albeit sometimes expensive).

Cellphone coverage was mysteriously ubiquitous! In particular I made calls from the depths of Bealey Valley and the peak of Mount Robert.

Contrary to the predictions of some, I did not miss my computer one bit, although I did a lot of thinking --- and praying! With no distractions I was able to sleep eight to ten hours a night, which I think was excellent for my health. All in all, it was a great time of physical, emotional, mental, and spiritual refreshment.

Now I've dropped straight back into the front lines ... ready for battle again!

Raising Awareness

I want to ride a tricycle from Vladivostok to Glasgow to raise awareness of the critical shortage of human attention.

Seriously, "raising awareness" campaigns make no sense to me in a world awash with campaigns on all sorts of issues. Instead of spending money on these global jaunts, let's spend it on actually trying to fix problems.