Most policymakers cannot feel a higher sense of urgency

To make a difference, we need to make ourselves useful, not exert more pressure

Yesterday, I talked to somebody who thought transformative AI is obviously approaching so rapidly that it's now a priority to communicate this sense of urgency. Publicly and loudly, from all sides.

Their model: action happens when it's pressing. But I'm afraid this makes the mistake of modeling fairly homogenous individuals, and actively backfires for diverse groups.

My interlocutor told me to use recent scenario explorations—from AI 2027 to Superintelligence Strategy—and authoritative letters—like the CAIS and FLI Statements—to explain to diplomats that:

• humanity could be over in 2-3 years;

• unless we achieve international agreement x, national regulation y, and standard z;

• global priority #1 is AGI preparedness.

But decision-makers are presented with urgency and appeals to authority all the time. There are thousands of shiny science people advocating for dozens of causes, using detailed forecasts of global catastrophe if actions x, y, and z aren't taken in the next 2-4 years.

Humans are scope-insensitive, so a lot of big policy issues look roughly the same: serious and complex. As a result of deep uncertainty, policymakers’ priorities are set by the beliefs of the closest 2-5 confidants, or maybe 15-50 team members. And these teams have strong memetic immune systems for good reasons.

Healthy teams, organizations & communities have to believe in at least two things to persist:

• they're doing important things for their members; and

• non-members cannot achieve this value (as easily).

As a result, healthy teams assume that outsiders are missing something. An established team then either has to

• have extraordinary norms to maintain its ability to respond to a changing environment; or

• experience a harsh shock to coordinate updates.

Most teams & institutions are the latter type (needs a shock), as it’s hard to maintain exceptional organizational hygiene.

If team members easily caved to outside pressure, the team would not persist. Thus, the most senior members are often also the most resilient to outside calls for urgent updates. It doesn't mean they will not listen to outsiders. But they will not just eat up a good scenario forecast signed off on by a few Nobel and Turing prize winners.

The problem is, in some sense, that there is too much urgency. People are grinding themselves to burnout. Performance metrics don't reward stepping back. Being a civil servant morally demands discipline and focus. Perpetual resource constraints and the resulting politics massively blur the bigger picture.

Giving people a sense of the bigger picture requires careful listening, discarding one's narrative, and then immersing in the other person’s paradigm. As a visitor in their land, one might occasionally earn the right to kindly point out gaps and opportunities. But this takes time and patience.¹

In an environment of abundant urgency, advocates have to advance tailored solutions to their top problem. Otherwise, nobody has an incentive to immediately deviate from The Plan. To design tailored solutions, you need context. And urgency usually doesn't get you enough access to create well-tailored solutions.

Thus, in a world that requires urgent action, most people are genuinely trying their hardest. They work on what they think is top priority, and they often have good reasons to think so.

The people who matter might change frequently and be hard to access but it’s those who need tailored solutions. I don’t mean solution as in “an international treaty”, but the immediate next step to get approval for next week. That’s usually more like “I need help with something vaguely related to what you care about”.

The limiting factor to AI regulation and international agreements is not that people aren’t worried. It’s that a lot of key people don’t have clear models of threats and solutions. Detailed scenarios are needed, but urgency isn’t helpful. The hard-earned trust that creates opportunity for in-depth exchange comes through patience and immediate usefulness.

1. Feel the AGI and urge those close to you! But doing so as a guest frequently backfires. Most people (think they) already give 150%. As a guest, I’d only urge whenever it’s ok to risk a burnt bridge. It may unearth unlikely allies but most often it achieves nothing.