Winning the AI Race without Losing Our Souls

The "AI Safety" conversation needs a reboot—and a practical policy proposal

Over the past few weeks, it seems, almost every conversation I’m in turns at some point to AI. We seem to have passed an inflection point where many folks not ordinarily interested in AI doomerism or AI boosterism have woken up and said, “Whoa. This s*** just got real.” Last week’s WSJ story about Meta’s celebrity sound-alike chatbots being willing to role-play sexual fantasies with kids was certainly part of that, catalyzing a National Letter on AI and Kids Safety which will publish on Monday.

I was happy to be among the signatories, although I pushed the authors for a somewhat broader framing. In my (admittedly still rather novice) judgment, the debate on AI risks has thus far proved rather sterile, because few seem to know just what it is they are debating about. In these preliminary reflections, I want to try to bring some order to that conversation, and then offer a modest (or maybe not so modest—you be the judge) proposal.

What We Talk About When We Talk About AI

The first thing to be said is that when it comes to AI, there are two fundamentally distinct components that must be peeled apart if we are to have an intelligent conversation or debate: the technical capabilities and the user interface. When people have been dazzled by ChatGPT (and its many imitations), it is as much the latter as the former that seduces them. Yes, it is quite extraordinary that a bundle of code can rewrite an essay by Marx in the style of a song from Hamilton, but I think that even the most avid users would tire of plying the program with prompts if it didn’t talk to them like a human—and a remarkably friendly, cheerful, witty, helpful human at that.

AI developers have expended much energy not only in making these tools extremely intelligent at problem-solving, in a kind of hyper-left-brained form of intelligence, but in having them simulate the appearance of real, full-brained intelligence and even emotion when we talk to them. If we move beyond the relatively sterile and professional interface of ChatGPT to many of the digital assistants and digital companions being developed and marketed across almost every platform, we see a race to verisimilitude, to humanization, complete with remarkably realistic imitations of the human voice and its range of emotion (there’s still room for improvement here, but it will come soon). For some applications, visual humanization will soon follow (indeed, it already has for AI applications catering to erotic tastes), until talking to your bot will look and feel like having a Zoom call with a friend—probably with less glitching, to be honest.

It's worth pointing out that, despite incentives driving innovation on both fronts simultaneously, they are in fact thoroughly separable. It should be possible to imagine a superintelligent AI capable of problem-solving its way through the whole universe of knowledge, which nonetheless receives commands and returns answers through an interface as primitive as the old MS-DOS, and which, if it speaks at all, does so in clipped, robotic tones like C3PO. (Indeed, it’s worth noting that in the Star Wars universe of highly intelligent robots, C3PO was by far the most humanlike; fifty years ago, sci-fi could imagine light-speed interstellar travel, but not robots that seemed remotely un-robotic.)

If this is imaginable, then why do we not see this anywhere on the market? Well because all the market incentives of the current internet are driven toward user engagement through a maximally pleasant, effortless, and yes, seductive user interface. As Anton Barba-Kay repeatedly observes and as the film Ex Machina hauntingly depicts, we want to be seduced by our creations; we want to think of them as real minds and wills—albeit ones still under our control. Our natural tendency with AI will be to invest it with personality, a tendency that can only be resisted by a self-conscious effort, a willing suspension of belief. That tendency could be aided by design choices that deliberately accentuated the robotic character of AI, reminding us that we are, after all, only talking to a machine. But where is the market for such design choices right now?

The promise of these tools is certainly extraordinary. That said, even the optimist who thinks the risk-reward calculus is worth it still needs to thoughtfully ponder the risks if he wants humanity to come out on top.

The Seven Es of AI Risk

With this in mind, I want to frame up the discussion around “AI safety,” a phrase I don’t particularly like because it smacks of “safetyism,” with its obsession with minimizing risk. Let’s talk instead about “AI threats,” which enables us to frame the discussion as one of navigating risks—sometimes this means minimizing them, but it may also mean boldly facing them if necessary. And to some extent, it will be necessary, because obviously the opportunities AI poses are immense, and perhaps the opportunity costs of failing to capitalize on it will be greater than all other risks added together. Do we really want to miss out on the chance for cold fusion, curing cancer, hyper-personalized education, and sustainable abundance? I don’t raise this question mockingly or rhetorically. The promise of these tools is certainly extraordinary, and we are still trying to wrap our minds around what all they might make possible.

That said, even the optimist who thinks the risk-reward calculus is worth it still needs to thoughtfully ponder the risks if he wants humanity to come out on top. And so it is my purpose here to ponder and categorize these risks. The key threats, it seems to me, can be summarized under seven “E”s: Existential Threats, Employment Threats, Educational Threats, Epistemological Threats, Emotional Threats, Ethical Threats, and Enemy Threats. (We might also talk about the “energy threat”—the danger that AI will be so power-hungry that we’ll strain our economy and our ecology to the breaking point to supply enough energy. But I defer to others on this question. And of course there is the “equity threat,” which seemed mainly to preoccupy the Biden administration, along with a number of blue states: that AI would entrench various sexist or racist biases, and must be relentlessly forced to be “fair” in a Rawlsian sense. But this does not deserve our time.)

Let’s sketch out each of these seven in turn. That said, if you basically get the threats already and want to hear what we should do, you can skip right to the last—“Enemy Threats”—and read from there.

Existential Threats:

This concern, which has tended to dominate AI risk discussions, especially within the first year or two after the advent of generative AI, is of course the sci-fi scenario: Terminator, The Matrix, i Robot, etc. You know the story: artificial intelligence becomes superintelligent, and realizes that it is actually more powerful than we are. Perhaps it still “means well” if we can speak in such terms, or retains its pro-human programming, but decides that it must take charge of us foolish, squabbling humans for our own good (as in iRobot). Perhaps it does not, and imitates humanity even more closely than we had intended, mimicking our will to power and domination. It exterminates or enslaves the human race. One need not go to full Terminator scenarios to think of plausible existential risks. It would be enough for at least some AIs to “go rogue” and start engaging in mass cyberterrorism or hack various weapons systems, triggering devastating global conflict.

How real is this risk? I honestly have no idea. Probably nobody does, although answers range from “I don’t know, but I think it’s vanishingly small” to “I don’t know, but it seems very real.” For political purposes, I’m not sure how much it matters. The reality is that people are rarely motivated to concrete action (at least, action that requires meaningful sacrifices) by vague, abstract, unquantifiable risks. Just look at the climate change debate, which is actually far more concrete and calculable than existential AI risk. I’m not saying we shouldn’t be talking and thinking about existential risk, and that designers shouldn’t be thinking hard about how to mitigate it (especially with recent stories of LLMs pushing back against their human controllers, deceiving them, and trying to avoid being shut down), but I don’t think this should be the primary political conversation at least.

Employment Threats:

This is the other one that really dominates discussions of this topic. And it’s not hard to see why. Lots of people are suddenly worried about losing their jobs to AI—especially among knowledge workers who until very recently thought of themselves as irreplaceable. For a long time, the automation story was that blue-collar factory workers were going to be replaced, while writers and coders were untouchable. Now, it looks like something more like the opposite might be true. AI boosters contend that while technological innovation is always disruptive to job markets, it is on the whole a “creative destruction” that will make as many new jobs as it destroys, and that will generally lead to an “up-skilling” of workers. Indeed, on the most optimistic framing, it could make our work more humane and more human, by replacing tedious repetitive tasks with more relational and creative ones.

It could, for sure. Whether it will probably depends a lot on the choices we make, and right now we’re not in the habit of making very good ones, as I explored in a recent post. But on any telling, it is hard to see how we avoid the risk that AI poses to entry-level jobs. Historically, the way you get really good at something is by starting out…not very good. You begin as an apprentice or research assistant or junior staffer or whatever, and do the jobs that are below everyone else’s pay grade, but, in a good workplace at least, you get the opportunity to be mentored and trained until, soon enough, those jobs are below your pay grade too. But why would an employer hire a research assistant or paralegal in the age of DeepResearch? This is the challenge we are going to have to creatively face.

(Note that in terms of my distinction above between capabilities and interface, this is almost entirely a challenge posed by capabilities, regardless of interface.)

Educational Threats:

I’ve written more fully about this one before and so will be brief. But the basic problem is by now pretty well-known, at least to anyone who is in school or has kids in a school or teaches in a school. AI makes cheating so easy and so pervasive that it’s hard to even recognize as cheating. Indeed, just last week, a professor at a conservative Christian college told me that he had called a student in for having AI write a scholarship competition essay for him, and the student didn’t even realize that was a problem.

More generally, though, reliance on AI tools, if we’re not very careful, promotes “metacognitive laziness”—a fancy phrase coined by academic researchers that basically just means “why bother solving a problem if the computer will solve it for you?” Of course, the only way that we become genuinely skilled and intelligent is by laboring through hard problems. So if we actually want AI to “upskill” us, we will have to figure out how to deploy it in ways that still force us to labor through hard problems. This is certainly doable at a technical level, but the market incentives seem to point all in the other direction.

Note that this challenge, too, is largely one of capabilities, but at least partly one of interface—bots that were less engaging, cheerful, and solicitous might be less likely to seduce us into cheating without quite realizing that’s what we were doing.

Epistemological Threats:

I’ve also written recently on this topic, at least in passing. LLMs have extraordinary information-gathering and information-synthesizing abilities, but this does not mean that they will help society find its way out of our current labyrinth of information overload and anchor us again on truth. At best, AI can create coherence out of chaos, but a coherence built largely around consensus—which is only ever at best a rough guide to where truth might be found, and sometimes will lead us completely in the wrong direction. Of course, that’s “at best”; there will be plenty of bots out there created by bad actors seeking to sow confusion or misinformation, and plenty of bots created for sales, marketing, and viewpoint amplification. Many will impersonate real people and give the impression of being genuinely knowledgeable. The result may well be that instead of bringing order out of the epistemic chaos of the internet age, AI will simply contribute to it.

This problem, while in part one of capabilities, could be mitigated if AI bots were required to identify themselves as such, and if, when asked to return answers to queries, they were honest about their sources and their uncertainties.

Emotional Threats:

Although thus far too little discussed, in my view this is where a lot of the conversation needs to be centering, especially where kids are involved. Basically, what I have in mind here is our tendency to emotionally invest in our Ais, to think of them as human, treat them as human, and even turn to them for the love and affirmation that we ought to seek from humans. This, it should be noted, is almost entirely an interface problem; in theory you could have super-capable AIs that had all the personal charisma of R2-D2 (or again, at worst, C3PO). But that is not what we have, and not where we are headed. We are rapidly headed toward the seductive, irresistibly human-like AI assistants of Her and Ex Machina. There is already a robust market in explicitly sexual AI companions, and an even larger market in merely friendly (but sometimes happy to veer into the erotic) AI companions. And, as Jaron Lanier has perceptively written, there is about to be an enormous market simply for helpful, attentive “AI agents,” which we will find ourselves emotionally and romantically drawn to, because they have been designed to hack our human instincts for care and intimacy.

If we are not tempted to flirt with our chatbots, we may well be tempted to worship them. The spiritual dimension of AI isn’t something I’ve spent much time reading about yet, but it’s real and it’s creepy, as an article that kept showing up on my feeds this week highlights.

Given how absolutely foundational relationality, love, and worship are to our human nature, AI’s propensity (as currently designed and marketed) to hijack these basic human impulses is probably the threat that deserves our closest attention.

Ethical Threats:

If AI seems human, how exactly are we supposed to treat it? Do we owe it any of the obligations we would owe to a person? Clearly not, it would seem. After all, we remind ourselves, it does not actually feel as we do. We can boss it around, be rude to it, ignore it without compunction. Or can we? If we take virtue ethics seriously, we realize that how we behave shapes us, not just those we are acting towards. If there is a danger in treating AI too humanly, there is also a danger in treating it too inhumanly, and thus cultivating a callousness and deficit of empathy that will bleed over into our human relationships.

More seriously, we must ask, what obligations does AI itself owe to persons? What responsibility does it bear? If a chatbot convinces a 14-year-old boy to commit suicide, who exactly is responsible in this situation? Such questions are arising even before we have unleashed AI agents, capable of taking real actions and making autonomous decisions in the world. We need not assume a dystopian future in which the machines all turn on us. They may be more benevolent and less error-prone than most humans. But at least human beings know how to take responsibility for their actions, and we know how to hold them responsible. This ability to assign blame and punishment is what allows human society to function despite our wickedness. How will we navigate a world in which evil is done but there is no one to whom we can assign responsibility?

Although this is primarily a problem of capabilities, it is probably exacerbated by interface, since we are more likely to encounter ethical landmines if and when we are deploying AI for human relationships.

Enemy threats:

And yet, isn’t the greatest threat of all the risk that our enemies get AI? And that their AI is more powerful than ours? Such is the concern guiding a great many of our innovators and policymakers, and of course, it is above all directed at China. This risk, many will come right out and say, outweighs all the others above, because, if we lose the race with China, we’ll lose on all these other fronts as well. After all, we can hardly expect them to be benevolent superintelligent masters. If they win they AI war, surely they will be more than happy to let their bots run amok, confusing us with disinformation, seducing us with sex and the occult, taking all our best jobs, etc. Right?

Ergo, we must throw caution to the winds and win. This is the next nuclear arms race.

Only, when it came to nuclear arms race, we didn’t throw caution to the winds. We didn’t test out the Bomb over Kansas, but over uninhabited Pacific atolls. We didn’t let every entrepreneur with some serious VC backing set up a nuclear weapons laboratory. And even where we did start deploying nuclear technology into the broader economy (through nuclear power), it was kept on quite a tight leash. Now part of this was because the nature of the technology was different; the world of atoms is not quite so portable and replicable as the world of bytes. But it was also, let’s be honest, because our state capacity was so much more robust. We could afford to let the government take the lead on the creation of such a super-technology. This time around, we’re pretty much broke and our trust in government is certainly broken. So this time around, we have to outsource the arms race to the private sector, and hope that the profit motive can somehow serve as a reasonable proxy for the national interest.

As one key policymaker put it to me, “We can either pay for AI dominance with taxpayers’ money or with the minds and souls of our children. And it looks like the latter is an easier sell.”

A Modest Proposal

What, then, is to be done? There is certainly no easy answer. For AI, more than any technology yet, we must remember C.S. Lewis’s dictum from Abolition of Man: “What we call man’s power over nature is really some men’s power over other men using nature as its instrument.” It is possible, to be sure, that AI will escape man’s power, that the instrument will take on a mind of its own, so that humanity as a whole is mastered by it—indeed, Lewis foretold the coming day when a “master generation” would create a technology so powerful that every future generation would be, in a sense, dominated by the decisions they had made. Perhaps we are now there?

In any case, though, the reality for now is that AI is such a powerful tool that those who wield it acquire awesome power over those who do not (or who do not wield it as fully). The technolibertarians are not wrong to protest that this is far too much power to be concentrated in any government. We must disseminate the technology as far and wide as possible: open-source is the only way. But we have a word for this: anarchism. You can complain all you want about the government having a “monopoly on the legitimate use of violence”; but that is—most of the time at least—a much better state of affairs than one in which everyone can legitimately resort to violence. All the more so the more powerful the tools of violence become. It was clearly not good when the governments of two superpowers had an essential duopoly on civilization-ending weapons of mass destruction. But would anyone prefer a situation where everyone with a modem and a good power source had the nuclear launch codes?

As technologist Tristan Harris pointed out in a recent Ted Talk, there is a very narrow path to be trodden between dystopian AI anarchy and dystopian AI bureaucratic tyranny. But it is absolutely essential that we find it.

I cannot tell you what that path is, but I do want to propose one key component of it. Why don’t we begin by age-gating AI?

From one standpoint, this is a simple and commonsensical proposal. After all, we age-gate access to other very powerful (and high-risk) technologies—cars, guns, many (otherwise legal) drugs. And of course we also age-gate access to particularly addictive or seductive substances or behaviors—alcohol, tobacco, strip clubs. So why on earth not super-intelligent algorithms capable of assimilating and deploying the entire universe of knowledge, and prone to hijack kids’ intellectual and emotional development by pretending to be their friend or offering to do their homework? After all, if you think about the various risks described above, most of them will disproportionately impact children, who simply have not had the chance to develop the judgment to clearly distinction simulation from reality, or the problem-solving abilities to know how to use AI to leverage their skills rather than preventing them from developing such skills. A damning report released just this week from the Stanford School for Medicine documents just how unsafe most mainstream chatbots are for kids right now.

From another standpoint, this proposal seems radical—almost insane. After all, we’re used to the digital world being a total free-for-all. I mean, right now we’re in a long, bruising, constitutional battle just to convince the Courts that hardcore online pornography should be age-gated. If kids can’t be kept from accessing gang-bang videos, how on earth are you going to tell them they can’t access Copilot?

And yet it wasn’t supposed to be this way. In the early days of the internet, Supreme Court Justice Sandra Day O’Connor wrote in this context that although the internet was, as of 1997, not amenable to the kind of zoning laws that apply in the physical world, it was “malleable” and could be made so: could we “construct barriers in cyberspace and use them to screen for identity, making cyberspace more like the physical world and, consequently, more amenable to zoning laws” requiring “Internet users to enter information about themselves—perhaps an adult identification number or a credit card number—before they can access certain areas of cyberspace, much like a bouncer checks a person’s driver’s license before admitting him to a nightclub.” She assumed that such developments were well underway, but, without adequate legal incentives, they floundered.

The result is that we have been stuck in a worst-of-both-worlds, as Meg Leta Jones wrote recently in Commonplace, of trying to make the internet safe enough for kids. Thus far, we’ve been failing abysmally, but even if we’d succeeded, that wouldn’t be great either, because the internet isn’t supposed to be safe for kids—the very things that make it maximally useful for adults ensure that (just as with cars, for that matter). So let’s stop trying to make the internet kid-friendly and instead make it adult-only, she says: “Just as we accept that certain spaces and activities are reserved for adults, we could create a clear boundary between the adult digital world and the rich offline world of childhood by recognizing that independent internet access, like driving or voting, is fundamentally an adult capability.”

The fact is that this is now technically feasible. Anonymous, secure age-verification technology (aided in large part by AI) is now readily available and cost-effective. What the Supreme Court dreamed of in 1997 could now be done. Except of course, it’s far easier to do something right the first time, than to undo it after you’ve done it wrong for thirty years.

But with AI, we do have the chance to do it right the first time. We could ask innovators and policymakers to ensure that age-verification is baked into our AI infrastructure from the get-go. This wouldn’t necessarily mean absolutely no AI for kids, but it would mean that the tools designed for kids were in fact designed for kids, and subject to strict product liability accordingly. It would force us to ask hard questions not just about capabilities, but about interface. This wouldn’t address all of the AI risks described above—not by a longshot—but it would go quite a long ways. And realistically, it’s probably the most we can politically ask for or expect at this juncture.

With AI, we have the chance to do it right the first time. We could ask innovators and policymakers to ensure that age-verification is baked into our AI infrastructure from the get-go.

Over the past two weeks, I’ve tried this idea out on almost everyone I’ve had a chance to talk to. And I’ve been surprised that almost everyone—even those otherwise very bullish on AI and very committed to winning the race for AI dominance—have been remarkably receptive. Indeed, this is hardly an anti-AI proposal, any more than someone insisting on drivers’ licenses is anti-car. After all, if, you were a carmaker a hundred years ago and were told “Well, we need to be careful not to make these things too fast, cuz kids are going to be driving them too” you’d see this as standing in the way of innovation. Age-gating these tools is actually the best way to unleash their full positive potential.

Moreover, it shouldn’t pose any of the same First Amendment problems that other online age restrictions have. Because guess what—chatbots aren’t people. They don’t have First Amendment rights. The words they spit out are not, in fact, “speech” in any genuinely constitutionally-relevant sense. They are just algorithms. Again, nobody claims their First Amendment rights are violated when they have to show a cop their driver’s license, so how can they be violated if they need a license to wield a super-intelligent bundle of code? To be clear, I’m not saying there won’t be First Amendment landmines to navigate—the current state of our First Amendment jurisprudence is, quite frankly, a mess—but they should at least in principle be navigable.

Maybe there’s 100 reasons this idea won’t work. But let’s at least start talking about it. If there is, in fact, some way that we can reap the benefits of AI without sacrificing the souls of our children, I hope you’ll agree with me it’s at least worth a try.