Saturday 19 June 2021
Spectre Mitigations Murder *Userspace* Performance In The Presence Of Frequent Syscalls
I just made a performance improvement to the (single-threaded) rr sources command to cache the results of access system calls checking for directory existence. It's a simple and very effective optimization on my Skylake, Linux 5.12 laptop:
Before: [roc@localhost code]$ time rr sources ~/pernosco/main/test-tmp/basics-demo >& ~/tmp/output2 real 3m19.648s user 1m9.157s sys 2m9.416s After: [roc@localhost code]$ time rr sources ~/pernosco/main/test-tmp/basics-demo >& ~/tmp/output2 real 0m36.160s user 0m36.009s sys 0m0.053s
One interesting thing is that we cut the userspace execution time in half even though we're executing more userspace instructions than before. Frequent system calls actually slow down code execution in userspace. I assumed this was at least partly due to Spectre mitigations so I turned those off (with mitigations=off) and reran the test:
Before: [roc@localhost code]$ time rr sources ~/pernosco/main/test-tmp/basics-demo >& ~/tmp/output2 real 2m5.776s user 0m33.052s sys 1m32.280s After: [roc@localhost code]$ time rr sources ~/pernosco/main/test-tmp/basics-demo >& ~/tmp/output2 real 0m33.422s user 0m32.934s sys 0m0.110sSo those Spectre mitigations make pre-optimization userspace run