Eyes Above The Waves

Robert O'Callahan. Christian. Repatriate Kiwi. Hacker.

Saturday 30 December 2017

Mixed Blessings Of Greenfield Software Development

The biggest software project I have ever worked on, and hopefully ever will work on, was Gecko. I was not one of its original architects, so my work on Gecko was initially very tightly constrained by design decisions made by others. Over the years some of those decisions were rescinded, and some of the new decisions were mine, but I always felt frustrated at being locked into designs that I didn't choose and (with the benefit of hindsight, at least) would not have chosen. "Wouldn't it be great", I often thought, "to build something entirely new from scratch, hopefully getting things right, or at least having no-one other than myself to blame for my mistakes!" I guess a lot of programmers feel this, and that's why we see more project duplication than the world really needs.

I was lucky enough at Mozilla to work on a project that gave me a bit of an outlet for this — rr. I didn't actually do much coding in rr at the beginning — Albert Noll, Nimrod Partush and Chris Jones got it underway — but I did participate in the design decisions.

Over the last two years Kyle Huey and I have been working on a new project which is broader in scope and more complicated than rr. The two of us have designed everything (with some feedback from some wonderful people, who know who they are) and implemented everything. We've been able to make our design choices quite freely — still constrained by external realities (e.g. the vagaries of the x86 instruction set), but not by legacy code. It has been exhilarating.

However, with freedom comes responsibility. My decision-making has constantly been haunted by the fear of being "That Person" — the one whom, years from now, developers will curse as they work around and within That Person's mistakes. I've also come to realize that being unencumbered by legacy design decisions can make design more difficult because the space is so unconstrained. This is exacerbated by the kind of project we're undertaking: it's a first-of-a-kind, so there aren't patterns to follow, and our system has novel, powerful and flexible technical capabilities, so we can dream up all sorts of features and implement them in different ways. It's scary. We have to accept that mistakes will be made and just hope that none of them are crippling. After all, if developers are cursing our design decisions years from now, that means we succeeded!

One of our biggest challenges is constantly trying to strike the right balance between forward planning and expediency. It's always tempting to respond to uncertainty by planning for all possibilities, by building abstractions that make it easier to change decisions later. Another temptation is to obsessively study issues to reduce the uncertainty. All that represents delay for our minimum viable product, and therefore must be minimized. My general approach here is to try to think about the future, but not code for it; to tell stories about how we would handle possible future scenarios, but not invest in them yet.

I think we're doing OK. Being nearly two years in is long enough to have regrets, and we don't have many. It will be very interesting, and humbling, to review our scorecard over the longer term.


Ted Mielczarek
I've thought a lot about this in recent years. I've seen lots of people make wrong decisions in both directions--not spending enough time on design, which makes future work harder, and also spending way too much time on design, which means it takes longer to get the thing you need right now done, and also almost always winds up making future work harder anyway! The best design Maxim I've been able to come up with is "design only for what you need right now plus the immediate next thing you want to do". Software is always in flux, so building quick hacks that don't leave any room for change will always make you unhappy. However, it's virtually impossible to predict what your needs will be on any sort of long-term time frame, so trying to build a perfect architecture is doomed to fail by not anticipating actual future needs, resulting in a design that makes it hard to do the things you actually need to do then. Unfortunately figuring out how much design is enough is the hard part. :) I think Rust is well-positioned here to market itself as a language that makes refactoring easier. If you know you're going to have to change your code anyway, you might as well have tools that help you do it!