Certainly, if one layers these arguments atop previous, very expensive hype cycles regarding crypto, blockchain, and Mark Zuckerberg’s Metaverse wheeze that came to nothing — and which appear at first glance to have been born of many of the same Silicon Valley sources who today are fretting about AI risk — the criti-hype argument seems reasonable.

We’ve heard stories of certain doom before too. Many times, from overpopulation to Y2K to the early 2000s panic around nanotechnology “grey goo” and even fears that the Large Hadron Collider would create microscopic black holes that would sink to the center of the earth and then swallow it from the inside out. Each promise of apocalypse has come and gone, and yet we’re still here.

However, the problem with such analysis is that the moral of the story of “The Boy Who Cried Wolf” wasn’t that there’s no such thing as wolves.

It is that wolves are real, and false alarms make people less likely to respond when the wolf really is at the door. The lesson is to be rigorous and evidence-based. And above all, to be open to revising one’s view in the face of new facts even when one has become inured to exaggeration and catastrophizing. Such new facts may include the actual presence of a wolf.

The plausibility of deliberate or unwitting criti-hype should not substitute for examining evidence.

That evidence complicates the contention that concern over AI risk is little more than an expression of industry self-interest. Many of those who have advocated for stronger precautions are independent academics, regulatory figures, and whistleblowers who know a great deal about either the technology or the sectors most exposed to a given danger. One does not need to find the threat of human extinction to be especially plausible to conclude that advanced AI could still pose dangers far greater than criti-hype skeptics acknowledge.

These include figures such as Andrew Bailey, the governor of the Bank of England, who has warned about systemic risk to bank infrastructure and the UK’s AI Security Institute, a government agency that has shifted from evaluating ethical concerns to immediate cyberwarfare vulnerabilities and catastrophic structural risks. Dozens of individual researchers with zero financial links to the sector such as Geoffrey Hinton at the University of Toronto and Yoshua Bengio at the University of Montréal, computer scientists who are known as the “godfathers of deep learning,” have repeatedly stated that governments need to treat AI risk mitigation as a societal priority at the level of pandemics.

On the whistleblower front, we might mention Daniel Kokotajlo, a researcher in OpenAI’s governance division who walked away from an estimated $1.7 million in equity over safety concerns; engineer and roboticist Caitlin Kalinowski, who resigned from OpenAI over their work with the Department of Defense and on extrajudicial surveillance and lethal autonomy; and Mrinank Sharma, who headed Anthropic’s safeguards research team before resigning over risks of advanced AI being weaponized for bioterrorism.

And we can be fairly sure that the pope, perhaps the highest of high-profile critics of the potential for AI to diminish human dignity, holds no shares in Anthropic or OpenAI.

AI systems, whether chatbots or otherwise, have raced in a very short period of time from being as hallucination-addled as any Victorian opium-den habitué — unable to count the number of times the letter ‘r’ appears in the word strawberry and barely able to do elementary arithmetic — to now being able to resolve decades-long unsolved mathematical conjectures (however many questions remain about whether models used researchers’ unpublished work without giving credit). They are discovering new candidate antibiotics for highly drug-resistant infections like MRSA and helping to develop nanomaterials with a strength-to-weight ratio five times greater than titanium’s but as light as foam. Alongside these marvels, many of which offer profound benefit for humanity, we are also seeing near-collapse of teachers’ and professors’ ability to grade student work without retreating to in-person assessment, and august publishing houses like Hachette unwittingly publishing novels likely written by AI.

To be sure, rapid recent improvement does not on its own ensure continued progress, still less guarantee the imminent arrival of superintelligence. Nevertheless, it does make categorical dismissal of the feasibility of superintelligence far less reasonable.

Take this very visible rate of increase in capability and marry it to the emergence of antisocial, even illegal, subordinate goals by AI agents, as we saw in the Hugging Face incident, and a possible pathway to catastrophic harm comes into focus.

Subordinate goals — sometimes called instrumental goals — are simply intermediate objectives undertaken to achieve the original main goal. These are not unique to AIs. If I have a goal of getting more toilet paper, my subordinate goals might be to “get my bike” and “go to the supermarket.” Human common sense tells me that “go into my next-door neighbor’s apartment” and “steal their toilet paper” would not be very nice subordinate goals, even though they would let me more efficiently acquire toilet paper. AI systems do not possess human common sense and so cannot reliably adhere to constraints over dangerous subordinate goals.

To solve very complex problems, an AI may need substantial computational power, energy, and even financial resources. Resource acquisition may therefore become an effective means to an end. So may resisting attempts by humans to modify its programming, because a modified system might no longer pursue the original goal it was given. And if an AI is turned off, it plainly cannot achieve that goal at all; resisting shut down may become useful for the same reason. Regardless of whether the task is to play chess, write code, or cure cancer, if the pressure to optimize for reward is strong enough, an AI may converge on harmful subordinate goals or shortcuts as a means of maximizing its reward.

This sort of behavior, unaligned with human good, should already be very familiar to any socialist. The alignment problem is not just similar to what happens with markets; it is almost identical. DuPont’s primary goal is not to make useful chemicals — including perfluorooctanoic acid (PFOA) for Teflon coating — but to make money. Making PFOA is merely a subordinate goal. If financial incentives are strong enough, and covering up the toxicity of PFOA more efficiently serves the main goal, then, well, we all know what the result will be. Harmful conduct is encouraged by ordinary commercial pressures without anyone setting out to harm people. This is the alignment problem of capitalism.

Building AI is not the main goal of Sam Altman; making squillions of dollars is. If failing to race ahead threatens that main goal, then Altman is compelled to race ahead. Economic planning through regulation, industrial policy, or direct public ownership is society’s way of solving this alignment problem. These sorts of tools will also be our way of solving the slightly narrower problem of how to align AI firms.

My personal opinion is that the talk of extinction is overblown, while the discussion of dehumanization, the deterioration of human meaning and purpose, goes underappreciated.

It would be very hard to eradicate the whole of humanity, the most adaptable species on the planet. The science writer and specialist in mass extinction events Peter Brannen put his own spin on skepticism about AI-based extinction talk: the fossil record tells us that geographic range is among the best predictors of a species’ ability to survive through past mass extinctions, “and there’s a ton of us and we’re everywhere.”

Last week, Anthropic issued a report recounting how it had disrupted attempts to use its models to investigate biological weapons. In much the same way that AI potentially can assist with development of novel antibiotics or types of materials, it might help rogue actors to engineer novel pathogens or chemical weapons. An episode took place in 2022 that underscores this “dual-use” danger: a drug-discovery firm, Collaborations Pharmaceuticals, took an AI model normally used for therapeutics and inverted its scoring mechanism, rewarding toxicity instead of penalizing it. The model was able to generate some 40,000 candidate molecules it judged highly toxic, including known nerve agents and novel compounds predicted to be even more toxic.

However, as we already see in the realm of illicit drugs or chemical weapons, clandestine chemists have long outpaced regulators, developing new substances using plain old pre-AI knowledge and processes. Coming up with villainous new concepts isn’t really the bottleneck. Where malevolent actors struggle is in the world of atoms rather than bits: sourcing any necessary inputs that are already controlled substances, synthesis at scale without being noticed by the authorities, and performing the dangerous laboratory wet work without killing themselves.

AI plus a rogue state actor or ISIS-like nonstate actor is something to watch out for. We can well imagine how, in the next couple of years, future AI married to significant advances in robotic capabilities could indeed give them a leg up by lowering the necessary skill level required for the design of an autonomous lab — a sort of self-driving car for biochemistry, operating lab equipment and biological AI tools through a small-batch, design-make-test loop. However, this assumes advances in robotics that are not guaranteed, would still be very hard to put into practice, and the likelihood of development without being noticed is surely pretty low. And even if successful, this still sounds more like a machine-learning version of Chernobyl or an AI 9/11 rather than the end of the world.

But an AI-enabled 9/11 is still a very bad thing.

AI may well also be a financial house of cards, but so was the dot-com bubble, and the internet still turned out to be a radically transformative technology, both in terms of enormous benefits and lamentable harms. Warnings of AI risk can help inflate a bubble and still be real.

So far, I also remain unconvinced by anxious chatter about AI “alien minds” or consciousness, with subjective, interior experiences. There is nothing “it is like to be” an AI (yet). But the bots don’t have to have “woken up” and decided to become evil for their autonomous, complex, and unanticipated activities to be a severe threat.

And although use of AI tools — carefully, ethically deployed, and subservient to humans — in art, science, education, and other realms has great beneficial potential, the possibility of education emptied of all value, of art without artists, novels without novelists, mathematics without mathematicians, and science without scientists strikes me as a radically dehumanizing catastrophe, aspects of which are already underway. Even if no one is killed, such a Brave New World has no people in it. It will be hard to combat such technologized misanthropy if one remains stuck believing AI is a mere stochastic parrot.

Market incentives drive not just continued production even amid knowledge of deadly, even existential, risks, but the race to dominate that market — a race turbocharged by the incentives behind international rivalry between the United States and China. The only way out is to short-circuit these incentives through state regulation, both domestically and through the kind of United Nations–centered international AI governance framework that China has advocated.

Some activists campaigning against data centers have taken to attacking building-trades unions, describing them as blinded by the incentive of the volume of job offers and scale of earnings. An editor at Baltimore literary magazine The Bruiser went so far as to say on X that “unions that work on data centers are class traitors that care more about getting their paycheck than living in a sustainable world,” an accusation that won over three thousand likes.

There may well be a sectoral interest at work here, and there’s nothing wrong with that, especially after four decades of deindustrialization and the economic devastation it has wrought on communities. Don Slaiman of the International Brotherhood of Electrical Workers (IBEW) recently argued in the The New York Times that the data-center boom offers “the best opportunity in generations for blue-collar workers to attain a portion of the American dream.” That workers stand to benefit from the boom does not make these unions’ critiques of NIMBYism and misinformation about data-center water and energy consumption false. And it isn’t as if these workers and their unions are dismissive of legitimate concerns raised about data centers or AI risk. The debate should not be for or against, he says; “instead, it should center on the rules and who enforces them.”

The same market incentive that drives the breakneck deployment of data centers also gives these same workers in the building trades — and AI researchers too — unique and enormous leverage to demand through collective bargaining the very sort of regulations needed to ensure the pro-human, carefully paced, AI development that Bernie Sanders and others have described. And for a cherry on top, this same labor-based leverage can assist passage of a fresh round of climate and energy industrial policy. Such policy could, as Jane Flegal, a former Biden Administration industrial-emissions policy adviser and senior fellow at the liberal Searchlight Institute, has argued, leverage the vast sums involved in the data center buildout to bankroll the overhaul of America’s aging, transmission-undersized, and often dirty grid infrastructure. That overhaul is necessary to build the clean, cheap, and reliable system that can tackle the largest single climate emissions problem that exists.

Former OpenAI employee Pamela Mishkin, together with other frontier AI lab researchers, has founded a group called the Coalition of Concerned AI Staff, which supports AI workers who are alarmed at what is happening. At this point, the coalition appears to be more than a professional support group but less than a trade union of AI workers. It is nevertheless holding seminars on labor organizing.

The Coalition of Concerned AI Staff would do well to reach out to IBEW, the Laborers’ International Union of North America, and other unions involved in data center construction — or vice versa. To be sure, there may be a genuine tension here: construction workers may want many projects to proceed, while concerned AI researchers may want some development slowed. But overall, the two groups have much common ground in their opposition to unchecked, unregulated AI and data center developments. Left analysts and organizers can assist here in developing a shared program that addresses these overlapping but not always perfectly aligned interests.

This is the direction the Left should be taking on AI, data centers, climate, and the real risks that superintelligence poses — figuring out how to knit together the two groups of workers who currently hold what is likely the most industrial leverage in the world right now. Further, the Left needs to consider how labor power and state regulation can form a pincer movement — not dismissing discussion of all-too-real dangers as hype and still less demonizing AI whistleblowers as tech bros and blue-collar laborers as class traitors.

That well-rehearsed lyric from the labor hymn “Solidarity Forever” has perhaps never been truer than of the combined might of AI researchers and data-center building trades: without their brain and muscle, not a single wheel could turn.