Thursday, 4 March 2021

On-Premises Pernosco Now Available; Reflecting On Application Confinement

In November we announced Pernosco availability for individual developers via our debugging-as-a-service platform. That product requires developers to share binary code, debug symbols and test data with us (but not necessarily source code), and we recognize that many potential customers are not comfortable with that. Therefore we are making Pernosco available to run on-premises. Contact us for a free trial! On-prem pricing is negotiable, but we prefer to charge a fixed amount per month for unlimited usage by a given team. Keep in mind Pernosco's current limitations: applications that work with rr (Linux, x86-64), C/C++/Rust/Ada/V8.

An on-premises customer says:

One of the key takeaways for me in our evaluation is that users keep coming back to pernosco without me pushing for it, and really like it — I have rarely seen such a high adoption rate in a new tool.

To deploy Pernosco on-premises we package it into two containers: the database builder and the application server. You collect an rr trace, run the database builder, then run the application server to export a Web interface to the database. Both steps, especially database building, require a reasonably powerful machine; our hosted service uses c5d.9xlarge instances for database building and a smaller shared i3.4xlarge instance for the application servers. If you want to run your own private shared service you are responsible for any authentication, authorization, public routing to the Web interface, wrangling database storage, etc.

To help our customers feel comfortable using Pernosco on-premises, all our closed-source code is bundled into those two containers, and we provide an open-source Python wrapper which runs those containers with sandboxing to confine it. Our goal is that you should not have to trust anything other than that easily-audited wrapper script (and rr of course, which is open source, but not so easily audited, though lots of people have looked into it). Our database builder container simply reads one subtree of the filesystem and drops a database into it, and the application server reads that subtree and exposes a Web server on a private subnet. Both containers should be incapable of leaking data to the outside world, even if the contents were malicious.

This seems like a natural approach to deploying closed-source software — no-one wants to be the next Solarwinds. In fact even if you receive the entire product's source code from your vendor, you still want confinement because you're not really going to audit it effectively. Therefore I'm surprised to find that this use-case doesn't seem to be well-supported by infrastructure!

For example, Docker provides an embedded DNS server to containers that cannot be disabled. We don't need it, and in theory crafted DNS queries could leak information to the outside world, so I'd like to disable it. But apparently our confinement goal is not considered a valid use-case by Docker. (I guess you can work around this by limiting the DNS service on the container host somehow, but that sucks more.)

Another interesting issue is preventing leaks through our Web application. For example when a user loads our application, it could send messages to public Internet hosts that leak information. We try to prevent this by having our application send CSP headers that deny all non-same-origin access. The problem is, if our application was malicious it could simply change the CSP it sends. Preventing that would seem to require a proxy outside the container that checks or modifies the CSP headers we send.

I'm surprised these problems haven't already been solved, or at least that the solutions aren't widely known and deployed.

What Would Jesus Do ... About Vaccination?

My thoughts about COVID19 vaccination as a Christian are pretty simple (assuming the Pfizer vaccine or something similar):

  • Is it safe for me?
    Yes. I'm not known to be allergic to vaccine components or immunologically compromised, and the safety data is solid.
  • If I get exposed to COVID19, will the vaccine stop me from getting infected and passing it on to people around me?
    Yes, almost certainly per the data.
  • Knowing this, if I chose to not get vaccinated, caught COVID19, and infected people around me with COVID19, would I have disobeyed Jesus' command to love my neighbour?
  • Is there any other way I can ensure I won't catch COVID19 and infect others?
  • Are there any countervailing ethical issues with taking the vaccine?
    No. None of the vaccines on offer are closely connected to abortion.

Thus it is pretty clearly God's will for me to be vaccinated.

Monday, 22 February 2021

Mercer Bay

West of Auckland, between Karekare beach and Piha beach, there is a small bay called Mercer Bay. It's surrounded by cliffs and there is no marked track to get down to it, but I've known for a long time that there is an unmarked route down the cliff. Last year I met someone who knows the route and she kindly agreed to guide a group of us down it on Saturday morning.

We started from Karekare up the Cowan track to the turnoff. The cliffs look precipitious and I'm not good with heights, but the route is actually not very difficult, and we got down quite easily. Mercer Bay is very pretty. At dead low tide you can walk around the north end of the beach to an inlet with three large sea-caves. The largest goes a significant distance into the hill to a blowhole where the roof has collapsed. It's incredibly impressive.

Climbing back up the cliff is the sensible but strenuous way out, but at dead low tide you can walk south around the rocks back to Karekare beach, and that's what we did. There is some nontrivial (for me) climbing involved — a few narrow ledges, a few overhangs — but the barnacle-covered conglomerate rocks provide excellent footholds and handholds if you have gloves or don't mind a few scrapes. Unfortunately we did not time our walk well and the tide was coming in, so we had to hurry. In a few places we had to cross little inlets with waves surging in and out, and later on we just plunged in when it wasn't too deep. I don't know how close we came to being trapped, but I certainly would have preferred a larger safety margin. Lesson learned! (I was carrying my personal locator beacon and we may even have had cellphone coverage, so I think we would have been rescued had we taken refuge on the higher rocks, but how embarrassing!)

Anyway, now that I know the way I'm already looking forward to going back :-). If we go around the rocks again I'll make sure we start well before low tide.

Saturday, 23 January 2021


Dehydrated food is great for tramping trips (saves weight and is less perishable) but the variety and cost in our local shops is not great, so although I don't like to accumulate many gadgets I bought a dehydrator over Christmas — a Biochef "Arizona" 6-tray unit. I've only used it a few times so far, but I'm very happy with it.

I've dehydrated sliced fruits: apples, pears, peaches, bananas and plums. Different people prefer different fruits but all of them have been well received. The pears are so sweet I feel guilty eating them. The unit can dehydrate at least 12 apples at a time, taking about 8 hours at 63C.

I've made beef and lamb jerky based on this recipe (using some Jack Daniels BBQ sauce I had around instead of "liquid smoke"). The unit can process about 2kg of meat in one run, taking about 6 hours at 70C with 6-7mm thick slices of meat. Longer dehydration times or smaller slices makes the jerky crunchy which is fine but not to everyone's taste. (The strips shown in the photo below are really too thin, because I bought a "stir fry" package from the supermarket with thinner strips than I expected, but they still taste good.)

I was surprised by how easy the process is. I thought it would take some practice to get good results but just about everything I've tried has turned out well.

There are a lot more experiments I want to do. In particular I want to investigate dehydrating vegetables for cooking meals while tramping. Fun!

Thursday, 21 January 2021

Tongariro Northern Circuit 2021

Yesterday I got back from a walk on the Tongariro Northern Circuit. Unfortunately things didn't go quite according to plan!

We had intended to walk the circuit over three days, clockwise. On Monday we would walk from Mangatepopo over Mt Tongariro, via the Tongariro Crossing, to Emerald Lakes, where we would turn off and carry on down to Oturere Hut. On Tuesday we'd walk south to Waihohonu Hut, and on Wednesday walk west to Whakapapa to complete most of the circuit. However on Monday high winds were forecast at the top of Tongariro so the Department of Conservation issued a "bad weather" forecast, so the shuttle we had booked would not take us to Mangatepopo, so we couldn't walk from there even if we thought it was safe, which we did not. We talked to staff at the DoC visitor's centre in Whakapapa and eventually decided to walk from Whakapapa east across the Tama Saddle to Waihohonu Hut and then north to Oturere — i.e. do half the circuit anticlockwise from Whakapapa in our first day. On Tuesday we walked south to Waihohonu as planned. On Wednesday we walked out to the Desert Rd and caught a shuttle back to Whakapapa instead of walking back to Whakapapa as planned, because the weather forecast was still poor and there didn't seem much point in re-traversing the saddle into wind and rain.

I thought the first day might be a bit gruelling — about eight hours of walking, on paper, with significant wind and rain forecast. It actually turned out pretty well. Only a little rain fell on us — we seemed to be moving east just ahead of it — and the wind was mostly at our backs. The sun even broke out a few times. No-one complained about the length of the walk and I felt pretty good myself. We reached Oturere Hut after about eight hours but that included our lunch break, a lengthy stop at Waihohonu Hut for a rest and hot drinks, and a side trip to Lower Tama Lake, so we were actually quite fast. One upside of the weather was that Oturere Hut, which is rather small, would have been packed with twenty-six people in good weather but only our group of ten and two other women actually showed up, so it was very comfortable.

On Tuesday the weather was similar — westerly wind and rain — but some of us wanted to do a "side trip" up Oturere Valley to Emerald Lakes if possible, before moving on to Waihohonu Hut — an hour and a half each way. Six of us (out of ten) did it, but it was a bit brutal! It wasn't too bad in the valley — wind and some drizzle in our faces, the spectacular volcanic desert landscape obscured by drifting fog — but the track then climbs steeply up to the saddle with the lakes, and there it was colder and the wind was much stronger. Scrambling up the last, steep part of the path into strong winds and driving rain was no fun at all! (The strong smell of sulphurous gases from the volcano added an extra frisson!) We had a quick look around the lake and the Tongariro Crossing junction and then scrambled right back down again. The walk back down the valley with the wind at our backs was positively pleasant. One good thing about this side trip is that it confirmed we had made the right choice in not risking crossing Tongariro on Monday!

We were able to have lunch in the cosy Oturere Hut and then it was just an easy two and a half hour walk to Waihohonu. Actually it was only mostly easy; in a few especially exposed places we got some very big wind gusts, probably the windiest conditions I've ever walked in. I had to lean hard into the wind to not be blown over, and some in our group just had to squat down and wait for the gusts to pass. I guess it was probably blowing a hundred kilometres an hour.

Once again we had the hut mostly to ourselves — and I think Waihohonu is still the best hut in New Zealand! The two women from Oturere stayed there, and there was also a French woman who was in NZ to work for a few years (arriving eighteen months ago — that was good timing). We had great afternoon and evening — a fire, games, good food and fellowship — and then on Wednesday morning, an early start and a short seventy-five minute walk to the car park to get picked up and returned to Whakapapa.

The weather was certainly disappointing. Ruapehu and Ngauruhoe are beatiful mountains and were entirely covered by cloud the entire time we were there. Of course it was disappointing we couldn't cross Tongariro. On the other hand, I think most or even all people in our group of ten had a good time and have much to be thankful for. If you tramp regularly you have to accept that the weather won't always be good; if you enjoy yourself even in the bad weather, you've got the Right Stuff for tramping :-).