Tuesday 17 June 2008
Eager Backouts
David Baron has argued that if someone checks in a patch and we later discover a serious regression from the patch, one that was not caught by tests --- we should behave the same as if there had been tests for that regression before the patch was checked in.
The logic is compelling and I think I mostly agree. My only caveat is that once a patch has been landed for a while, the cost of backing it out --- due to other changes that have built on the patch --- may rise to make this impractical. But for most changes, big and small, there's a reasonably large window where that won't be the case.
In practice it would mean backing out big patches a lot more often. That's probably a good thing, if we take seriously the goal of having the trunk remain in a shippable state at all times. I think it also lowers risk quite a bit ... compare the histories
Check in A Back out A Check in A+B Back out A+B Check in A+B+C | Check in A Check in B Check in C |
With the backout approach we would be much less likely to get into sticky situations where we don't know what has to be backed out to get to a good place. And because we periodically get the trunk back into a good state, it would be easier to distinguish the effects of A+B+C development from other ongoing trunk development.
Distributed version control makes this approach more practical because it's now much easier to maintain A+B+C on a local branch until it finally sticks.
People may argue that with long-lived experimental branches, tryserver builds, and improving test infrastructure, we won't have belatedly-discovered regressions, but I'm not that optimistic :-).
Comments
If there is nobody able and willing to fix such a regression (in a reasonable amount of time), and if the regression is deemed to be worse than the bugs fixed by the patch, then that would seem to make sense.
Of course the cost of removing the patch, reintroducing the bugs it fixed and working out and landing a new fix should be weight against the cost of accepting the regression for a while and working on a fix for the regression, but I'm sure we'll do that sensibly on a case by case basis.