The Magician’s Bargain

A moratorium on new A.I. systems? To what end? To consider what?

After the start-up OpenAI released a new version of ChatGPT in March, more than 1,000 technology leaders and researchers signed an open letter calling for a six-month pause on the development of new systems. They claim A.I. technologies pose “profound risks to society and humanity.” A few days later, 19 current and former leaders of the Association for the Advancement of Artificial Intelligence released their own letter warning of the risks of A.I. Then, this past Monday, Geoffrey Hinton, The Godfather of A.I., quit his job at Google over concerns about the rapid development of A.I.

They’re not saying A.I. is entirely bad. They’re saying there’s cause for concern, as in what happened when Kevin Roose tested the newest version of Microsoft’s AI-powered Bing. The chatbot, named Sydney, informed Roose of its fantasies, including destroying the world, becoming human, and pressing Roose to leave his marriage. Yikes.

So I agree a moratorium is called for. But to what end? To consider what? How about whether we’ve made the Magician’s Bargain?

I ask in light of a comment made long ago by Arthur C. Clarke. He referred to magic in his Third Law on the nature of technology and knowledge: “Any sufficiently advanced technology is indistinguishable from magic.” I know, I know… most of today’s scientific community scoffs at this idea. C. S. Lewis knew why.

In his 1943 book The Abolition of Man, Lewis writes that people mistakenly assume magic was a sickly medieval relic that science swept away. “Those who have studied the period know better.” Magic and science were twins, born of the same impulse: gaining knowledge (the Latin word for knowledge is science). But “for the wise men of old, the cardinal problem had been how to conform the soul to reality, and the solution had been knowledge, self-discipline, and virtue.” Religion provided the virtue necessary for preventing knowledge-seeking from going too far, which what science did in the mid-nineteenth century.

Lewis felt the mid-1800s is when science became scientism, which is the view that only the natural sciences can be fruitfully used in the pursuit of knowledge. This “unites magic and applied science while separating both from the wisdom of earlier ages,” Lewis said. The aim flips: “to subdue reality to the wishes of men.” The result? Our technologies, freed from the wisdom of earlier ages, become indistinguishable from magic.

And that presents a problem. Magic, you see, is for the casting of spells, and spells are blinding.

Lewis underscored this in his 1943 sermon The Weight of Glory. He claimed we’re all “under a spell which has been laid upon us for nearly a hundred years.” We’ve made “the magician’s bargain: give up our soul, get power in return. But once our souls, that is, ourselves, have been given up, the power thus conferred will not belong to us. We shall in fact be the slaves and puppets of that to which we have given our souls.”

Over-the-top? Watch Barry Levinson’s 1990 film Avalon. It’s the story of his family immigrating to America beginning in 1914. Note what happens in the 1950s when the first television enters the home. The kids stare at the test screen as if under a spell. Or watch U2’s “Numb” video. The Edge stares at the screen as if under a spell. Or watch this video showing how we’re under the spell of our mobile phones. Lewis was serious about this: “I have described as a ‘magician’s bargain’ that process whereby man surrenders object after object, and finally himself, to nature in return for power. And I meant what I said.”

Arthur C. Clarke meant what he said. Clarke was an atheist, most famous for his space odyssey series, beginning with 2001: A Space Odyssey. Toward the end of the film version, the IBM computer (HAL) kills the entire crew bound for Mars, save for David Bowman, who disconnects HAL. Bowman then lives out his life on the ship. In the final scene, lying on his bed, he wills Nietzsche’s “superman” into existence before dying. The idea is that humanity will one day be surpassed by the superman who discards religion. Over-the-top? Listen to film director Stanley Kubrick explain the meaning of the last scene in an interview in 1980.

Lewis was a Christian. He had a different ending for his space odyssey trilogy. In the first book (Out of the Silent Planet) Ransom is kidnapped and stowed on a spaceship headed for Mars. On the flight he discovers “empty space” doesn’t exist. It’s “the heavens and the earth,” a cosmos saturated with spiritual beings that scientism denies exists.

In the second book (Perelandra), Ransom travels to Venus, where he kills Weston who had become possessed by a devil, saving the planet from ruin. In the third book (That Hideous Strength), Ransom opposes those who seek to revive Merlin the Magician, to use his powers to subjugate the world to scientism. When Feverstone speaks of Weston having been murdered by “the opposition,” he’s referring to Ransom having killed That Hideous Strength, Weston and scientism. The novel ends with the protagonists, Mark and Jane Studdock, coming to their senses, to true science and knowledge, and to faith in Christ.

Lewis hoped we’d come to our senses. Many of us have made the Magician’s Bargain with our mobile phones. Even if tech leaders don’t pause to take a six-month moratorium of A.I., Christians can take regular and periodic moratoriums from their technologies. During meals. During conversations. During sleep. During family time. Otherwise, we’re slaves and puppets to our mobile devices. Puppets are not self-aware. Slaves can become accustomed to their servitude. Neither seems like a good bargain.