Sunday, 17 September 2017

Dreaming The Singularity

This is an actual dream I had last night. I'm not sure of the order of some of the events, but it otherwise felt clear.

I walked around my neighbourhood and suddenly felt a unique sensation in my head — a sort of overflowing energy that quickly increased in intensity until I was overwhelmed and fell down.

I came to, wondering what had happened; the answer came subconsciously, like remembering: You have changed. You have new abilities.

Focusing my attention internally or externally seemed to generate data and explanations from outside myself. I sensed something in my mind like apps, but I seemed unable to use them.

I asked who changed me. Artificial entities.

How advanced are they? They hold themselves back, waiting for people to follow.

How was I changed? Nanotech.

What about my family? They will follow.

Am I still a Christian? Your beliefs remain in place.

Expect more changes, ever more quickly, becoming a continuous stream.

I was filled with fear at not knowing what I would become, and woke up still filled with it. Oddly, though, in the dream I never considered whether the guiding entities were benign. In retrospect that seems suspicious...

In fact the entire dream was odd considering I don't really believe in the Singularity.

Saturday, 16 September 2017

Facebook's "Explaining React's License" Doesn't

This post "Explaining React's license" confuses me. In particular:

As our business has become successful, we've become a larger target for meritless patent litigation. This type of litigation can be extremely costly in terms of both resources and attention. It would have been easy for us to stop contributing to open source, or to do what some other large companies do and only release software that isn't used in our most successful products, but we decided to take a different approach.

This seems to claim that a) contributing to open source makes Facebook a larger target for meritless patent litigation and (later) b) the explicit patent license compensates for this somehow. The post does not spell out a rationale for a). Is it that exposing valuable source code make it easier for people to identify potential infringements of their (meritless?) patents? If so, I'm curious what evidence exists for that. And how does b) work? We know countersuits don't work against so-called Non-Practicing Entities — that's the point of NPEs. So is the goal only to deter meritless patent litigation by Practicing Entities, hoping that they'll use Facebook code and depend on a Facebook patent license? If that's the approach, how does applying the patent license to projects like zstd, where it appears Facebook doesn't have any patents, help? Just by increasing the general fear in a Practicing Entity that somewhere they use Facebook patents?

For a post claiming to be an explanation, it seems to leave a lot unexplained.

Thursday, 14 September 2017

Some Opinions On The History Of Web Audio

People complain that Web Audio provides implementations of numerous canned processing features, but they very often don't do exactly what you want, and working around those limitations by writing your own audio processing code in JS is difficult or impossible.

This was an obvious pitfall from the moment the Web Audio API was proposed by Chris Rogers (at Google, at that time). I personally fought pretty hard in the Audio WG for an API that would be based on JS audio processing (with allowance for popular effects to be replaced with browser-implemented modules). I invested enough to write a draft spec for my alternative and implement a lot of that spec in Firefox, including Worker-based JS sample processing.

My efforts went nowhere for several reasons. My views on making JS sample manipulation a priority were not shared by the Audio WG. Here's my very first response to Chris Rogers' reveal of the Web Audio draft; you can read the resulting discussion there. The main arguments against prioritizing JS sample processing were that JS sample manipulation would be too slow, and JS GC (or other non-realtime behaviour) would make audio too glitchy. Furthermore, audio professionals like Chris Rogers assured me they had identified a set of primitives that would suffice for most use cases. Since most of the Audio WG were audio professionals and I wasn't, I didn't have much defense against "audio professionals say..." arguments.

The Web Audio API proceeded mostly unchanged because there wasn't anyone other than me trying to make significant changes. After an initial burst of interest Apple's WG participation declined dramatically, perhaps because they were getting Chris Rogers' Webkit implementation "for free" and had nothing to gain from further discussion. I begged Microsoft people to get involved but they never did; in this and other areas they were (are?) apparently content for Mozilla and Google to spend energy to thrash out a decent spec that they later implement.

However, the main reason that Web Audio was eventually standardized without major changes is because Google and Apple shipped it long before the spec was done. They shipped it with a "webkit" prefix, but they evangelized it to developers who of course started using it, and so pretty soon Mozilla had to cave.

Ironically, soon after Web Audio won, the "extensible Web" become a hot buzzword. Web Audio had a TAG review at which it was clear Web Audio was pretty much the antithesis of "extensible Web", but by then it was too late to do anything about it.

What could I have done better? I probably should have reduced the scope of my spec proposal to exclude MediaStream/HTMLMediaElement integration. But I don't think that, or anything else I can think of, would have changed the outcome.

Monday, 11 September 2017

Sonny The Prophet

Yesterday, Sunday, as people mingled after our church service, one of our church members brought a visitor to me, introduced me to him, and quickly disappeared. Our visitor introduced himself as Sonny, "a prophet of God".

I have leadership responsibilities so the reflexive urge to palm him off on someone else was not an option. Besides, I'm not a committed cessationist so the possibility of encountering a real prophet exists, however remote...

Sonny explained that he visits different churches every week and God had instructed him to visit our church yesterday so he could bring us the word of God. Sonny noticed that we advertise a table tennis club at our church on Sunday afternoons (105 Vincent St, 3pm, all welcome!) and advised me that this is contrary to God's command to keep the Sabbath holy. I asked him why he thought so, so he opened his Bible at Exodus 20:8. I suggested that that text prohibits work, not games. He switched to Isaiah 58:13, but I continued to express doubt that that excludes table tennis (... never mind the questions of how Israelite Sabbath law and practice apply to a Gentile congregation in the new covenant).

Remaining calm, Sonny proceeded to tell me he was having a vision in which he saw maggots falling on me. At this point I thought it fair to ask Sonny, just as calmly, how he wished to confirm that he was a true prophet, not a false prophet. Unfortunately his only answer was to repeat his testimony about himself, and I was unable to investigate further because I genuinely had to leave for for lunch.

I'd just taught a Sunday school class about our duty to love other Christians (John 13:34-35), our enemies (Matthew 5:43-48), and everyone (Mark 12:28), and here God presented me with someone who was possibly all three, so I did not feel able to get worked up about the situation. On the other hand I don't know how to show love to Sonny, other than praying for him, given I probably won't see him again.

Thursday, 7 September 2017

rr 5.0 Released

I've released rr 5.0. Kyle convinced me that trace stability and portability were worth a major version bump.

Release notes:

  • Introduction of Cap'n Proto to stabilize the trace format. Recordings created in this rr release should be replayable in any future rr release. This is a plan, not a promise, since we don't know what might happen in the future, but I'm hopeful.
  • New rr pack command makes recordings self-contained.
  • Packed recordings from one machine can be replayed on a different machine by trapping CPUID instructions when supported on the replay machine. We don't have much experience with this yet but so far so good.
  • Brotli compression for smaller traces and lower recording overhead.
  • Improvements to the rr command-line arguments to ease debugger/IDE integration. rr replay now accepts a -- argument; all following arguments are passed to the debugger verbatim. Also, the bare rr command is now smarter about choosing a default subcommand; if the following argument is a directory, the default subcommand is replay, otherwise it is record.
  • Performance improvements, especially for pathological cases with lots of switching to and from the rr supervisor process.
  • Syscall support expanded.
  • Many bugs fixed.

Enjoy!

Friday, 1 September 2017

rr Trace Portability

We want to be able to record an rr trace on one machine but copy it to another machine for replay. For example, you might record a failing test on one machine and copy the trace to a developer's machine for debugging. Or, you might record a failure locally and upload the trace to some cloud service for analysis. In short: on rr master, this works!

It turned out there were only two big issues to solve. We needed a way to make traces fully self-contained, because for efficiency we don't always copy all needed files into the trace during recording. rr pack addressed that. rr pack also compacts the trace by eliminating duplicate copies of the same file. Switching to brotli also reduced trace size, as did using Cap'n Proto for trace data.

The other big issue was handling CPUID instructions. We needed a way to ensure that during replay CPUID instructions returned the same results as they did during recording — they generally won't if you switch machines. Modern Intel hardware supports "CPUID faulting", i.e. you can configure the CPU to trap every time a CPUID instruction occurs. Linux didn't expose this capability to user-space, so last year Kyle Huey did the hard work of adding a Linux system-call API to expose it: the ARCH_GET/SET_CPUID subfeature of arch_prctl. It works very much like the existing PR_GET/SET_TSC, which give control over the faulting of RDTSC/RDTSCP instructions. Getting the feature into the upstream kernel was a bit of an ordeal, but that's a story for another day. It finally shipped in the 4.12 kernel.

When CPUID faulting is available, rr recording stores the results of all CPUID instructions in the trace, and rr replay intercepts all CPUID instructions and takes their results from the trace. With this in place, we're able to move traces from one machine/distro/kernel to another and replay them successfully.

We also support situations where CPUID faulting is not available on the recording machine but is on the replay machine. At the start of recording we save all available CPUID data (there are only a relatively small number of possible CPUID "leaves"), and then rr replay traps CPUID instructions and emulates them using the stored data.

Caveat: the user is responsible for ensuring the destination machine supports all instructions and other CPU features used by the recorded program. At some point we could add an rr feature to mask the CPUID values reported during recording so you can limit the CPU features a recorded program uses. (We actually already do this internally so that applications running under rr believe that RTM transactional memory and RDRAND, which rr can't handle, are not available.)

CPUID faulting is supported on most modern Intel CPUs, at least on Ivy Bridge and its successor Core architectures. Kyle also added support to upstream Xen and KVM to virtualize it, and even emulate it regardless of whether the underlying hardware supports it. However, VM guests running on older Xen or KVM hypervisors, or on other hypervisors, probably can't use it. And as mentioned, you will need a Linux 4.12 kernel or later.

Tuesday, 29 August 2017

Fedora/Ubuntu Kernels Work With rr Again

Ubuntu finally released a kernel update (4.4.0-93) that fixes the regression that broke rr. It took a month after the regression was fixed upstream. The slow update cycle has been frustrating, and it's a bit worrying: I hope security fixes are treated with more urgency!

Fedora updated F25 last week (updating to a 4.12 kernel) and F26 was fixed earlier, so I think we're mostly out of the woods on this issue. I will prepare an rr 4.6.0 release.