“Because the history of evolution is that life escapes all barriers. Life breaks free. Life expands to new territories. Painfully, perhaps even dangerously. But life finds a way.”
“But we have soothed ourselves into imagining sudden change as something that happens outside of the normal order of things. An accident, like a car crash. Or, beyond our control, like a fatal illness. We do not conceive of sudden, irrational change as built into the very fabric of existence. Yet it is. And chaos theory teaches us . . . . That’s a deep truth about the structure of our universe. But, for some reason, we insist upon behaving as if it were not true.
— Jurassic Park, Michael Crichton
Nature abhors a vacuum. Life expands to fill all available niches. The reason why life escapes all barriers is because it expands till it crosses some assumed/phase-change boundary which gives it an unexpected route past the barrier.
*ALL* barriers depend upon assumptions. The assumption that the barrier can’t be destroyed. The assumption that the barrier is impenetrable. The assumption that there is no way around the barrier. The assumption that the barrier won’t dissolve on it own.
Jurassic Park turns into a disaster because of single-minded and total reliance upon the assumption that a population composed entirely of females can’t reproduce.
The party line for AI safety is a barrier composed of a correctly implemented optimizable top-level goal that is absolutely guaranteed to protect humanity. Unfortunately, it depends critically upon four assumptions that appear to be tremendously vulnerable
- that there even exists an optimizable top-level goal that is both internally consistent and absolutely guaranteed to protect humanity,
- that humanity can find that goal before being overtaken by some other disaster that would have been preventable by some other path,
- that an entity implemented with that goal will be able to successfully defend that goal from all attempts to change it, and
- that even a successful attempt to create an entity with that goal won’t directly increase the probability of human destruction.
The vast majority of dialogue on safe AI has focused on how truly dangerous AI could be and, yet, how AIs won’t be able to penetrate this miraculous barrier. Brilliant individuals with vested interests in their own opinions and appearances have been able to focus solely on the AI penetrating the barrier because the presented (and, I believe, correct) defense is both complicated enough that most people don’t truly understand it and unintuitive enough that they want to argue it. Other directions of discussion have been suppressed by the creation of strawmen (utility monsters, paperclip maximizers, happy face stompers, AIs smart enough to defeat humanity yet too stupid to understand certain simple concepts), the refusal to take the time to understand arguments whose conclusions differ (dismissing them with the single pejorative “confused”) , and the constant pushing of the message that the task is far, far too difficult for a single human to grasp, much less solve.
Jurassic Park turned into a disaster because people with an agenda didn’t seek out rational concerns and wouldn’t listen to rational concerns when they were voiced. There are excellent arguments as to why assumptions one and four, in particular, are almost certain to be wrong. There are excellent arguments as to why the task of safe AI may not be so complicated if only some ridiculous assumptions are discarded — or, at least, the task of “as-safe-as-possible-and-certainly-better-than-nonexistent” AI.
The belief that absolute control is both possible and useful is an illusion. Life (and make no mistake, an AI will be alive) will find a way. Sticking your head in the sand and insisting that simple barriers will hold is merely The Jurassic Park Syndrome.