Thursday 30 July 2026
Thoughts on 'Pacing The Frontier'
I signed the Pacing The Frontier petition for a very simple reason: I agree with the statement.
There’s a certain set of people who are determined to assign ulterior motives to anyone advocating for any kind of AI regulation or cooperative safety measures. (E.g., “they want regulatory capture!” or “they’re hyping the capabilities of their products!”.) But I know some of the other people who signed the petition, and I believe that everyone I know who signed it is sincere. I suspect those ulterior-motive objections are partly rooted in fear: if people really believed things are this serious, that would be disturbing. Well, we do and it is disturbing.
Another reaction I see is “trying to cooperate on something like this is impossible because of rogue labs and/or China.” That’s certainly a challenge, but it’s a challenge we have to tackle, not run away from. The CCP (which I generally despise) actually talks a good game about AI safety and control. It is obviously in their interests just as much as it is in ours, so it is not necessary to jump to the conclusion that they’re insincere. Obviously there need to be verification protocols to build trust on both sides, and building a trustworthy agreement will be very difficult, but we have to try. I don’t think “it’s really hard and probably won’t work, let’s all just die” is a reasonable approach. Also, the leading Chinese AI companies’ current strategy is open source and open weight models; if they stop doing that, we’ll obviously know. In that sense, it’s easier for us to trust them than it is for them to trust us.
Another reaction is, “we need to just stamp on the accelerator and ‘win the AI race’.” This is nonsense. To such people, I ask: what does “winning the AI race” actually mean? What is the finish line? Is it when human unemployment reaches 100%? Is it when some super-AI that you hope you still control seizes absolute power over the entire world? What experience do you have with controlling something vastly more intelligent than you, to make you think that’s easy?