Thursday 14 November 2019
Supercharging Gdb With Pernosco
At Pernosco we don't believe that gdb (or any other traditional debugger) has the ideal debugging interface, but users familiar with gdb can get started with Pernosco more easily if gdb is available. Also, gdb has many useful features and it will be a long time before Pernosco matches every single one of those features. Therefore we've integrated gdb into Pernosco. Moreover, leveraging Pernosco's infrastructure makes our gdb experience clearly the best gdb experience ever — in my opinion!
The best feature of Pernosco-gdb is speed. We built a gdbserver that extracts information from the Pernosco omniscient database, so forward or reverse execution in gdb is simply a matter of updating an internal timestamp and getting results for that time whenever gdb asks for information. Our users routinely skip forward or backward through minutes of application execution with a second or two delay in the UI. Achieving this is harder than you might think; gdb uses internal breakpoints to observe, e.g., loading and unloading of shared libraries, so we need tricks to avoid unnecessary internal stops.
I'm also proud of the synchronization we achieved between gdb and the rest of Pernosco. Our gdb sessions track changes to the current moment and stack frame made by other parts of Pernosco (including other gdb sessions!), and likewise gdb "execution" and changes to the current stack frame are reflected in the rest of Pernosco. (Pernosco stack walking is considerably more robust than gdb's, so synchronization can fail when gdb disagrees about what stack frames exist.) Starting a gdb session attaches it to the current process, so you can have multiple gdb sessions open, for the same process and/or different processes, all synchronized as much as possible.
Pernosco enables some nice improvements to watchpoints and breakpoints, as described here.
Deploying debugging as a cloud service enables some other improvements in the gdb experience. Gdb sessions run on our servers and users can run arbitrary code in them, so by necessity we put each session in a very tight sandbox. This means we don't have to worry about potentially malicious gdbinit scripts, DWARF debuginfo, etc; we can just configure gdb to trust everything.
Another benefit of the Pernosco model is that we take responsibility for configuring gdb optimally and keeping it up to date. For example we ensure .gdb_index files are always built.
We achieve all this with minimal changes to upstream gdb. Currently our only change is to disable gdb's JIT debugging support, because that creates a lot of usually-unnecessary internal breakpoint stops. We publish the sources for our build of gdb (though technically we don't have to, as gdb is not AGPL).