Saturday, 27 December 2008

Ship's Blog

We just got back from our Christmas boat trip with extended family. It was fantastic. I've created a custom Google Map showing the places we visited. The weather was mostly very good although on Tuesday it rained a fair bit (especially Tuesday night) and strong south-easterly winds kept us holed up in Bon Accord Harbour. Christmas Day and Boxing Day, on the other hand, were sheer magic. (Visibility was amazing; from Sullivan's Bay we could clearly see the tip of Castle Rock above the horizon --- 44 nautical miles away (that's 50 miles, 80 km). A Christmas Eve cruise up the Mahurangi River right up to Warkworth was another highlight. We did lots of walks, ate lots of food, swam, fished, rowed, mucked around on the beach, and played games. I hardly thought about Web browsers at all.


weka

The grounds of Mansion House are have a lot of a fairly tame wekas and peacocks.


Beehive Island

Beehive Island seen from Kawau with the storm gathering behind it.


Castle Rock

Taken from Sullivan's Bay in the evening of Christmas Day, Castle Rock is the peak jutting out above the horizon to the right of Pudding Island in the foreground.


Sullivan's Bay

Sullivan's Bay on Boxing Day --- there were lots of day trippers on Christmas Day, but it was a lot quieter the next day, although there were still a lot of campers about.

Saturday, 20 December 2008

Offline

I'm going to be offline until December 27. If you need anything from me, rent a boat and search for me somewhere in Kawau Bay.

From then I plan to be sporadically online until January 5, when I should be back operating at full power.



Sunday, 14 December 2008

New Game, Old Rules

This summary is not available. Please click here to view the post.

Monday, 8 December 2008

Reftests

Before I get into what this post is really about, let me heap praise on reftests. David Baron came up with the idea of writing automated tests for layout and rendering where each test comprises two pages and the test asserts that the renderings of the two pages are identical. This works much better than comparing test pages to reference images (although you can use an image as a reference if you want), because you can easily write tests that work no matter what fonts are present, or what the platform form controls look like, or what platform antialiasing behaviour is, and so on. There are almost always many ways to achieve a particular rendering effect in a Web page, so it's very easy to write reftests for parsing, layout, and many rendering effects. There are also tricks we've learned to overcome some problems; for example, if there are a few pixels in the page whose rendering is allowed to vary, you can exclude them from being tested just by placing a "censoring" element over them in both the test and reference pages. In dire circumstances we can even use SVG filters to pixel-process test output to avoid spurious failures. Sometimes when there are test failures that aren't visible to the naked eye (e.g. tiny differences in color channel values), it's tempting to introduce some kind of tolerance threshold to the reftest framework, but so far we've always been able to tweak the tests to avoid those problems and so I strongly resist adding tolerances. They should not be needed, and adding them would open a big can of worms.

The reftest approach may seem obvious but it definitely isn't, because other browsers don't seem to use it. I don't know why. Comparing against reference images only makes sense in very limited circumstances. Dumping an internal representation of the page layout and comparing that against a reference makes life difficult if you want to change your internal representation, and skips testing a lot of the rendering path. Reftests don't even depend on which engine you're using --- you can run most of our reftests in any browser. One argument that has been made against reftests is that someone might introduce a bug that breaks the test and the reference in the same way, so tests pass. That is possible, but if feature X regresses and all reftests pass, that just means you should have had a test specifically for feature X (where the test page uses feature X but the reference doesn't).

Anyway, the problem at hand: quite frequently we fix bugs where a particular page is triggering pathological performance problems. For example, we might switch to a slightly more complex algorithm with lower asymptotic performance bounds, or we might be a little more careful about caching or invalidating some data. Unfortunately we don't have any good way to create automated tests for such fixes. Our major performance test frameworks are not suitable for this because these pages are not so important that we will refuse to accept any regression; we just want to make sure they don't get "too bad". Also we don't want to have hundreds of numbers that must be manually compared, so it's not clear how to automatically choose baseline numbers for comparison.

One crazy idea I've had is performance reftests. So you create a test page and a reference, and the test asserts that the test page execution time is within some constant factor of the reference's execution time. In this case we would definitely need to introduce a tolerance threshold. One problem is that a slow machine getting temporarily stuck (e.g. in pageout) would easily cause a spurious failure. So perhaps we could measure metrics other than wall-clock execution time. For example, we could measure user-mode CPU times and assert they match within a constant factor. We could instrument the memory allocator and assert that memory footprint is within a constant factor. I'm not really sure if this would work, but it would be an interesting project for someone to experiment on, perhaps in an academic setting.



Places I've Been


Google's Street View is cool, especially now that it covers New Zealand.

Places I lived:


  1. 5612 Fair Oaks St, Pittsburgh (first 2 years of grad school --- with Andrew, David, Scott and Herbie)
  2. 350 South Highland Ave, Pittsburgh (next 2.5 years of grad school --- with Andrew, David, Scott, Herbie, Ted, and David M!)
  3. 5726 Beacon St, Pittsburgh (remaining 2.5 years of grad school --- my wife didn't want to move in with those guys)
  4. 89 N Broadway, White Plains (first 6 months at IBM)
  5. 81 Holland Ave, White Plains (3 more years at IBM)

Where I live now was actually missed by Google's Street View pass over Auckland. Fine with me.

Places I worked:


  1. Wean Hall, Carnegie Mellon University, Pittsburgh (visible at the end of the driveway)
  2. IBM Research, 81 Skyline Drive, Hawthorne, NY (Street View hasn't covered this yet)
  3. Novell, 39 Market Place, Viaduct, Auckland (pan right for the incomparable Sunshine Chinese restaurant)
  4. Mozilla, 83 Nelson St, Auckland
  5. Mozilla, 8 Kent St, Newmarket, Auckland (current location)


Mozilla NZ

The last couple of weeks were quite exciting in the Mozilla Auckland office: interns Michael Ventnor and Brian Birtles started, and new hire Jonathan Kew was visiting. Michael has been working on bugs related to text-shadow and box-shadow, and is looking for more GTK integration projects. Brian will be working on SMIL. Jonathan is working on text and fonts; his current project is implementing Core Text integration for Mac while fixing some blocker bugs on the side. Everyone else is busy with their usual stuff; Karl just landed downloadable fonts support for GTK/Pango on trunk.

On Friday we went out for a Christmas-y dinner bash at Al Dente.


Mozilla NZ dinner


West Coast

After getting back from Waiheke we picked up Jonathan Kew and dragged him out west to the Piha area. I can't let people visit Auckland and miss out on the spectacular wild west coast. We strolled around on the beach, which was predictably crowded by NZ standards, being the first really warm Saturday of the summer. But we also did the Mercer Bay loop, a favourite 1-hour walk of mine along the clifftops just south of Piha --- no-one else in sight. You just have to be careful taking photos, since last year (or so) someone ill-advisedly stepped outside the fence and fell off the cliff.


View over the ocean from the Mercer Bay track


Waiheke

A little over a week ago my wife and I had a few days on Waiheke to celebrate our tenth wedding anniversary. We rented a car on the island and stayed in an apartment at Onetangi. The weather was fabulous and we had a great time. We didn't do much, but one day we walked from Cowes Bay Road to Waikopua Bay and then around the rocks to Man-O-War Bay and looped back along the road. The view from the eastern end of Waiheke across to Coromandel was fantastic. The second day we walked from Oneroa around the coast past Island Bay to Matiatia and then back to Oneroa. The first part of that walk is all vineyards, most of the rest is spectacular rocky coastline. That loop would be a good day trip since you can start and end at the Matiatia ferry and stop in Oneroa for refreshments. We ate out at Charley Farley's in Onetangi (good), Te Motu Vineyard (good, but surprisingly we were the only guests the whole time we were there), and the Lazy Lounge in Oneroa (also good).

View of Man-O-War Bay from the Cowes Bay Road



View of the 'island' in Island Bay