Sunday, 19 February 2017

"New Scientist" And The Meaning Of Life

A recent issue of New Scientist has a headline article on "The Meaning Of Life". It describes research showing correlations between a sense of purpose and various positive attributes such as health and happiness. It then touches on a few ways people can try to enhance their sense of purpose.

Unfortunately the second paragraph gives the game away:

As human beings, it is hard for us to shake the idea that our existence must have significance beyond the here and now. Life begins and ends, yes, but surely there is a greater meaning. The trouble is, these stories we tell ourselves do nothing to soften the harsh reality: as far as the universe is concerned, we are nothing but fleeting and randomly assembled collections of energy and matter. One day, we will all be dust.

Quite so --- assuming a secular worldview. In that context, the question "what is my purpose?" has no answer. The best you can hope for is to grab hold of some idea, make it your purpose and refrain from asking whether you have chosen correctly.

To me, even before I became a Christian, that approach seemed unsatisfactory --- too much like intentional self-delusion. To conclude "there is no greater meaning" and carry on as if there is didn't seem honest.

Later I came to believe that Christianity is objectively true. That means God has created us for a specific (yet broad) purpose --- "to glorify God and enjoy him forever", in the words of the Westminster Shorter Catechism. The question has an answer. We'll spend eternity unpacking it, though.

Monday, 6 February 2017

What Rust Can Do That Other Languages Can't, In Six Short Lines

struct X {
  y: Y
impl X {
  fn y(&self) -> &Y { &self.y }

This defines an aggregate type containing a field y of type Y directly (not as a separate heap object). Then we define a getter method that returns the field by reference. Crucially, the Rust compiler will verify that all callers of the getter prevent the returned reference from outliving the X object. It compiles down to what you'd expect a C compiler to produce (pointer addition) with no space or time overhead.

As far as I know, no other remotely mainstream language can express this, for various reasons. C/C++ can't do it because the compiler does not perform the lifetime checks. (The C++ Core Guidelines lifetime checking proposal proposes such checks, but it's incomplete and hasn't received any updates for over a year.) Most other languages simply prevent you from giving away an interior reference, or require y to refer to a distinct heap object from the X.

This is my go-to example every time someone suggests that modern C++ is just as suitable for safe systems programming as Rust.

Update As I half-expected, people did turn up a couple of non-toy languages that can handle this example. D has some special-casing for this case, though its existing safety checks are limited and easily circumvented accidentally or deliberately. (Those checks are in the process of being expanded, though it's hard to say where D will end up.) Go can also do this, because its GC supports interior pointers. That's nice, though you're still buying into the fundamental tradeoffs of GC: some combination of increased memory usage, reduced throughput, and pauses. Plus, relying on GC means it's nearly impossible to handle the "replace a C library" use case.

Saturday, 4 February 2017

rr 4.5.0 Released

We just released rr 4.5.0. Massive thanks to Keno Fischer who probably contributed more to this release than anyone else.

This is another incremental release. rr is already pretty good at what it does --- single-core recording, replaying, and debugging with gdb and reverse execution. The incremental nature of recent releases reflects that. It's still a significant amount of work to keep updating rr to fix bugs and cope with new kernel and hardware features as they start getting used by applications. Notably, this release adds support for Intel Knights Landing and Kaby Lake CPUs, contains a workaround for a regression caused by a Linux kernel security fix, and adds support for recording software using Hardware Lock Elision (and includes detection of a KVM bug that breaks our HLE support). This release also contains a fix that enables recording of Go programs.

We have some encouraging reports of rr adoption and rr users pretty much ceasing use of standalone gdb. However, rr definitely needs more users. When hardware and OS issues crop up, the bigger the user-base, the easier it will be to get the attention of the people whose cooperation we need. So, if you're a happy rr user and you want to help make sure rr doesn't stop working one day, please spread the word :-).

Let me say a tiny little bit about future plans. I plan to keep maintaining rr --- with the help of Keno and Kyle and other people in the community. I don't plan to add major new user-facing functionality to rr (i.e. going beyond gdb), but rather build such things as separate projects on top of rr. rr has a stable-in-practice C++ API that's amenable to supporting alternative debugger implementations, and I'd be happy to help people build on it. I've built but not yet released a framework that uses parts of DynamoRio to enable low-overhead binary instrumentation of rr replays; I plan to keep that closed at least until we have money coming in. The one major improvement that we expect we'll add to rr some time this year is reworking trace storage to enable easy trace portability and fix some existing issues.

I Really Admire Jehovah's Witnesses

... and other door-to-door evangelists.

I just had a couple of JWs visit me. I always cut through the script by asking door-to-door evangelists who they're with. Anyone who won't tell me straight-up I'd shoo off immediately, but people generally are open about it. On hearing they were JWs, I confirmed that they don't believe Jesus is God and asked them what they make of John chapter 1. We fenced amicably about those issues a little bit and I think they decided I wasn't worth their time.

Of course I think JWs are totally wrong about this key doctrine. However, I really admire the zeal and fortitude of anyone who does honest door-to-door evangelism, because I've done just enough of other kinds of evangelism to know how hard it would be. I believe that these people are at least partly motivated by genuine love for the lost. As Penn Jillette observed, if you think you've come across truth that's life-changing for everyone, only a coward or a criminal would fail to evangelize --- though door-to-door may not be the most effective approach.

Regardless, bravo!

Tuesday, 31 January 2017

A Followup About AV Test Reports

Well, my post certainly got a lot of attention. I probably would have put a bit more thought into it had I known it was going to go viral (150K views and counting). I did update it a bit but I see no reason to change its main thrust.

A respondent asked me to comment on the results of some AV product comparison tests that show Microsoft's product, the one I recommended, in a poor light. I had a look at one of the reports they recommended and I think my comments are worth reproducing here.

In that test, MS got 97% (1570 out of 1619) ... lower than most of the other products, but the actual difference is very small.

The major problem with tests like this is that they are designed to fit the strengths of AV products and avoid their weaknesses. The report doesn't say how they acquire their malware samples, but I guess they get them from the same sources AV vendors do. (They're not writing their own since they say they independently verify that their malware samples "work" on an unprotected machine.) So what they're testing here is the ability of AV software to recognize malware that's been in the wild long enough to be recognized and added to someone's database. That's exactly the malware that AV systems detect. But in the real world users will often encounter malware that hasn't had time to be classified yet, and possibly (e.g. in spear-phishing cases) will never be classified. A more realistic test would include malware like that. Testers should cover that by generating a whole lot of their own malware that's not in any database, and see how AV products perform on that. My guess is that that detection rate would be around 0% if they do it realistically, partly because in the real world, malware authors can iterate on their malware until they're sure the major AV products won't detect it. (And no, it doesn't really matter if the AV classifier uses heuristics or neural nets or whatever, except that using neural nets makes it devilishly hard to understand false positives and negatives.)

So for the sake of argument let's suppose 20% of attacks in the real world use new unclassified malware and 80% use old malware and none of the 20% are detected by AV products. In this report, that would be 405 additional malware samples not detected by any product. Now Microsoft scores 77.6% and the best (F-Secure in this case) scores 79.9%. That difference doesn't look as important now.

The other major issue with this whole approach is that it takes no account of the downsides of AV products. If a product slows down your system massively (even more than other products), that doesn't show up here. If a product blocks all kinds of valid content, that doesn't show up here. If a product introduces huge security vulnerabilities --- even if they're broadly known --- that doesn't show up here. If a product spams the user incessantly with annoying messages (that teach them to ignore security warnings altogether, poisoning the human ecosystem), that doesn't show up here.

This limited approach to testing probably does more harm than good, because to the extent AV vendors care about these test results, they'll optimize for them at the expense of those other important factors that aren't being measured.

There are also some issues with this particular test report. For example they say how important it is to test up-to-date software, and then test "Microsoft Windows 7 Home Premium SP1 64-Bit, with updates as of 1st July 2016" and Firefox version 43.0.4. But on 1st July 2016, the latest versions were Windows 10 and Firefox 47.0.1. Another issue I have with this particular report's product comparisons is that I suspect all it's really measuring is how closely their malware sample import pipeline matches the pipelines of other vendors. Maybe F-Secure won that benchmark because they happen to get their malware samples from exactly the same sources as AV-Comparatives and the other products use slightly different sources. The source of malware samples is critical here and I can't find anywhere they say what it is.

Of course there may be research that's better than this report. This just happens to be one recommended to me by an AV sympathizer.

Update Bruce points out that page 6 of the report, second paragraph, does describe a bit more about how they acquired their samples, and they say they scan the Internet themselves for malware samples. I don't know how I missed that! But there's still critical information missing like the lead time between a sample being scanned and then tested. I think the issues I wrote about above still apply.

Monday, 30 January 2017

Tripling Down Against USA Conference Hosting

Well, that escalated quickly!

I wrote "Really, Please Stop Booking International Conferences In The USA" and then the very next day chaos erupted as Trump's executive order on the "seven Muslim countries" appeared and went into effect.

I'm less open-borders than a lot of the people currently freaking out --- but regardless, the order is cruel and capricious, especially how it shut out people with valid visas and even green cards without warning. And I think this reinforces my point about conferences: how can you book a conference with international attendees in a country where this happens?

Administration staff are already talking about other disturbing changes:

Miller also noted on Saturday that Trump administration officials are discussing the possibility of asking foreign visitors to disclose all websites and social media sites they visit, and to share the contacts in their cell phones. If the foreign visitor declines to share such information, he or she could be denied entry. Sources told CNN that the idea is just in the preliminary discussion level.

No thanks :-(.

It has been pointed out that Trump's EO might actually force some organizations to hold more conferences in the USA because some US residents may not be able to return home if they go abroad. That makes some sense but it feels wrong.

On another note, around now the top US universities start their annual drive to recruit foreign grad students. I wonder how that's going to go. The professoriat not being exactly Trump country, they'll have to reconcile their fears with a message that foreign students are going to be OK. Careers depend on it.

Friday, 27 January 2017

Rustbelt Is Hiring

Rustbelt is looking for postdocs.

Most academic CS research projects strike me as somewhere between "could be useful" and "hahahaha no". Rustbelt is one of the very few in the category of "oh please hurry up we need those results yesterday". That's exactly the sort of project researchers should be flocking to.

The long-term goal here is to specify formal semantics for Rust's safe and unsafe code and have a practical, tool-supported methodology for verifying that modules containing unsafe code uphold Rust's safety guarantees. This would lead to an ecosystem where most developers mostly write safe code and that's relatively easy to do, but some developers sometimes write unsafe code and that's a lot harder to do because you have to formally verify that it's OK. In exchange, you know that your unsafe code isn't undermining the benefits of Rust; it's "actually safe". I think this would be a great situation to have. We need the results as soon as possible because they will suggest changes to Rust and/or what is considered "valid unsafe code", and the sooner they happen, the easier that would be.