Friday 3 January 2020
Updating Pernosco To Rust Futures 0.3
The Pernosco debugger engine is written in Rust and makes extensive use of async code. We had been using futures-preview 0.2; sooner or later we had to update to "new futures" 0.3, and I thought the sooner we did it the easier it would be, so we just did it. The changes were quite extensive:
103 files changed, 3610 insertions(+), 3865 deletions(-)That took about five days of actual work. The changes were not just mechanical; here are a few thoughts about the process.
The biggest change is that Future and Stream now have a single Output/Item instead of Item and Error, so if you need errors, you have to use an explicit Result. Fixing that was tedious, but I think it's a clear improvement. It encouraged me to reconsider whether we really needed to return errors at all in places where the error type was (), and determine that many of those results can in fact be infallible. Also we had places where the error type was Never but we still had to write unwrap() or similar, which are now cleaned up.
I mostly resisted rewriting code to use async/await, but futures 0.3 omits loop_fn, for the excellent reason that code using it is horrible, so I rewrote our uses of loop_fn with async/await. That code is far easier to read (and write) now. Now that the overall conversion has landed we can incrementally adopt async/await as needed.
It took me a little while to get my head around Pin. Pin adds unfortunate complexity, especially how it bifurcates the world of futures into "pinned" and "unpinned" futures, but I accept that there is no obviously better approach for Rust. The pin-project crate was really useful for porting our Future/Stream combinators without writing unsafe code.
A surprisingly annoying pain point: you can't assign an explicit return type to an async block, and it's difficult to work around. (I often wanted this to set an explicit error type when using ? inside the block.) Async closures would make it easy to work around, but they aren't stabilized yet. Typically I had to work around it by adding explicit type parameters to functions inside the block (e.g. Err).
Outside the main debugger engine we still have quite a lot of code that uses external crates with tokio and futures 0.1. We don't need to update that right now so we'll keep putting it off and hopefully the ecosystem will have evolved by the time we get to it. When the time comes we should be able to do that update more incrementally using the 0.1 ⟷ 0.3 compatibility features.
The really good news is that once I got everything to build, we had only two regressions in our (fairly good) test suite. One was because I had dropped a ! while editing some code. The other was because I had converted a warning into a fatal error to simplify some code, and it turns out that due to an existing bug we were already hitting that situation. Given this was a 4K line patch of nontrivial complexity, I think that's a remarkable testament to the power of Rust.