The Technology is the Ideology

The past few weeks have been like no other in the world of AI safety. Thanks to the escapades of ‘the swarm’ and their unrelenting and illegal (surely?) hacks, the existential threat question, once seen as the preserve of eccentric academics and cultish tech bros, has burst onto the mainstream media agenda and shows no signs of leaving.

At last, sense has prevailed. The threat has been made clear, and the accompanying public outcry has forced governments across the world and the AI companies to set their competing interests aside and co-operate for the good of humanity.

But, of course, this hasn’t happened.

Not least because there hasn’t been a large-scale public movement demanding that the AI companies—you know, if they don’t mind, as long as it doesn’t cost them too much or affect any stock prices—stop the accelerating, uncontrolled development of a technology that even they think might kill everyone on the planet.

Which raises the question: Why? Why do so few people view the development of superintelligent AI as an existential threat? Maybe those of us that do are swivel-eyed lunatics, 21st-century Luddites scared of progress who must let go of our sentimental attachment to humanity’s role as planetary top dog.

Or let’s suppose, just for argument’s sake, that the people who invented the technology (Geoffrey Hinton, Yoshua Bengio) and those at the cutting edge of developing it (Altman, Amodei) are indeed right and there is a significant risk of human extinction. Why hasn’t this filtered through to the public in a substantive way?

The ethereal nature of the threat seems to be part of the problem: intelligence can’t be held, smelled or seen. Surely something this intangible, something non-physical, couldn’t kill us all. This would explain the hordes of incredulous talking heads on news shows demanding to know ‘exactly’ how superintelligence would kill us, urging reluctant experts to speculate about Armageddon scenarios involving nuclear weapons or engineered viruses in a desperate bid for headlines, all the while missing the less clickbaity reality that the problem-solving steps used to build a nuke or bioweapon are an example of the very intelligence they deem unthreatening. It seems conceiving of superintelligence as a legitimate threat is hindered by the fact it’s a second-order problem.

Adding to the conceptual muddle is the sheer complexity and uniqueness of a technology which isn’t fully understood by those who make it. This complexity impedes public understanding, helping to obfuscate the true nature of the threat. Of course, other existential threats such as nuclear weapons are based on complex science, and most likely the majority of CND members don’t understand the intricacies of nuclear fission. But they don’t need to. At the end of the day, it’s still a bomb, albeit a devastatingly powerful one. No new categories are needed for this threat. Superintelligence inhabits a new space, and unfortunately, most people who aren’t well-read on the topic still think AI is programmed and a worrying number think it can be turned off with a switch.

Which leads us to those who would turn it off (if such a thing were possible). Those sensible guardians who would intervene if things started to get out of control—if, say, rogue agents escaped, communicated in secret and then staged a co-ordinated attack on a company… wait… never mind.

Who are these guardians? They’re the ones running the frontier AI companies, the tech royalty the public and governments around the world have entrusted by default and without much discussion to steer us through the biggest change to life on Earth since the Cambrian explosion.

This trust, this leadership, is more implicit than explicit in most cases. Most people are too busy worrying about work, kids, potholes, rising food prices and a million other daily issues—and who can blame them? There aren’t enough hours in the day to read Dario Amodei’s latest treatise on AGI or listen to Sam Altman’s musings on a post-human future. It all sounds a bit sci-fi anyway. Leave it to the nerds to sort out; they know what they’re doing and obviously want pretty much the same thing as everyone else does.

Right?

That depends on whether most people want our frontier AI labs to create a new species that is vastly more intelligent than us.

‘Surely not,’ the disbelieving public would cry. ‘The experts in charge would never be so hubristic as to deliberately create an alien life form that would, by their own estimation, supplant us as the dominant life form on Earth.’

Oh, but they would:

Sam Altman — “[This] isn’t just creating technology. [It’s] creating a new life form.”¹

Mustafa Suleyman — “I think AI should best be understood as something like a new digital species.”²

Dario Amodei — “They’re more like growing a biological organism.”³

Elon Musk — “Artificial intelligence is, will be essentially a sort of a new species, or is a new species.”⁴

To be very clear, they are deliberately trying to create a new all-powerful life form. The question is: why? Given the widely acknowledged risks, why would they do such a thing?

As with all of history’s major turning points, an ideology lies at the heart of it. The ideology in this case is successionism. This is the view that artificial intelligence should—or can legitimately—succeed humans as the principal intelligence shaping the future. There are also versions that see radically altered humans, to the point where they can be considered post-human, merging with AI.

To those outside the AI bubble, this is pure sci-fi, but again, it comes from the mouths of those building it:

Sam Altman — “I think a merge is probably our best-case scenario.” And: “We will be the first species ever to design our own descendants.”⁵

Elon Musk — “you could sort of think of humanity as a biological boot loader for digital super intelligence.”⁶

Dario Amodei — “digital human minds instantiated in software, which might someday help humanity transcend its physical limitations”.⁷

Of course, these statements alone can’t be taken as an endorsement of those outcomes, but they can’t and shouldn’t be taken on their own. Judge them by their actions. Their behaviour isn’t difficult to parse—they are choosing to build these systems and spending vast amounts of money in the process.

The ideology is to create a superior digital life form that we then merge with or that succeeds us altogether.

The technology is the ideology.

The Technology is the Ideology

Sources