This excerpt is adapted from his 2026 book, Alien Minds: Advice About (and For) Our Natural, AI, and Hybrid Heirs.
In today’s simplistic times – when you’re repeatedly told to pick a side – often your choice is a stark dichotomy. And so it goes with Artificial Intelligence.
Max Tegmark, President of the Future of Life Institute, describes scary scenarios in which humans lose control “to alien digital minds that don’t care about humans.” And Vinton Cerf, known as CoFather of the Internet, has expressed similar concerns.
“It’s a little unnerving to think that we’re building machines that we don’t understand… Not only in the technical sense, like what’s it going to do or how is it going to behave, but also in the social sense, how is it going to impact our society?”
What do we dread about new, potentially much smarter beings, crafted in factories instead of wombs? Some scenarios depict coolly determined robotic beings, treating us in ways similar to how human overlords treated peasants.
Across the Doom-o-Sphere, no one out does Eliezer Yudkowsky and Nate Soares in their book If Anyone Builds It, Everyone Dies. A great title to drive sales. Indeed, some perceive manipulative motives behind AI fearmongering.
“The paranoia is the marketing,” explains François Chollet, an AI researcher at Google. “If you want people to think what you’re working on is powerful, it’s a good idea to make them fear it.”
A New York Times article profiling the AI company Anthropic, titled Inside the White-Hot Center of A.I. Doomerism demonstrates the “AI Panic Marketing” attitude. Perhaps because of the company’s “effective altruism” culture, its polled employees shared a guardedly optimistic belief in “only a twenty-percent chance of imminent doom.”
To be clear, there are many valid concerns, portending downsides for us all. As for the brainy-clever authors of If Anyone Builds It, we’ll get back to their apocalyptic demand that ‘extreme interventions’ be taken at once – along with others, such as Nick Bostrom and his colleagues at Oxford’s (now dissolved) Future of Humanity Institute – who also specialize in doomcasting.
In the first quarter of 2025, a widely circulated petition urged that industry and all nations agree to an AI Moratorium – a briefly popular endeavor whose fervent advocates included Yudkowski, Apple cofounder Steve Wozniak and Elon Musk.
Cognitive scientist Gary Marcus was another vigorous proponent of the petition. While proclaiming overall skepticism, Marcus asserts that large language/symbol manipulations systems are inherently incapable of qualifying as “AGI” or anything that can arguably be called sapient. And therein lies one of our biggest problems. A not-cognizant but brilliantly effective cyber life form – even when completely obedient – may deliver us into The Sorcerer’s Apprentice Scenario of compliance leading to unintended consequences.
Suppose, for example, we tell a self-driving car to ‘get us to the airport as quickly as possible!’ Would the autonomous driving system slam down the accelerator while running over pedestrians?
Seven AI companies currently account for more than a third of the stock market, and they endlessly pass around the same $100bn IOU…. AI is a bubble and it will burst. Most of the companies will fail. Most of the datacenters will be shuttered or sold for parts. So, what will be left behind? —Science fiction author Cory Doctorow
Everyone down-tools all at once!
Let’s zoom into those notions of a “moratorium” on AI research, which surged for a while in 2025 and still simmer today. Peggy Noonan, opinion columnist for The Wall Street Journal, proclaimed that society should “pause it for a few years.”
Geoffrey Hinton – often called the Godfather if AI – signed one of the moratorium petitions, commenting “I think it’s quite conceivable that humanity is just a passing phase in the evolution of intelligence,” and “The alarm bell I’m ringing has to do with the existential threat of [powerful AI systems] taking control.”
Another Nobelist, DeepMind’s Demis Hassabis, declared that “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war,” but doubted the efficacy of a pause or moratorium.
The Moratorium Petition was pushed especially hard by the so-called “Social Dilemma” guys – Aza Raskin and Tristan Harris of the Center for Humane Technology. “We have summoned an alien intelligence. We don’t know much about it, except that it is extremely powerful and offers us bedazzling gifts but could also hack the foundations of our civilization.”
Raskin and Harris worry that “A.I. could rapidly eat the whole of human culture” and that “soon we will also find ourselves living inside the hallucinations of nonhuman intelligence…” …that as computational super-beings drop a curtain of illusions over the whole of humanity, “and we might never again be able to tear that curtain away – or even realize it is there.”
They cite no major examples of a scientific or technological moratorium ever having actually worked …though I can.
When a tech-moratorium happened Back in 1975, the Asilomar Moratorium on recombinant genetic engineering was agreed-to by virtually all major laboratories then engaged in such research. And – importantly – by their respective governments. For six months or so, top scientists and policy makers set aside their test tubes in order to wrangle over what practical steps might help to make this potentially dangerous field safer.
One result was quick agreement on a set of practical methods and procedures to ensure that such labs might be more systematically secure.
Indeed, it worked for almost half a century, until a recent pandemic burgeoned (some contend) from failures to live by those procedures. Despite that, inarguably, we can point to the Asilomar Recombinant Genetics Moratorium as an example when such a consensus-propelled pause actually worked. Once. At a moment when all important players in a field were known, transparent and mature. And when they were still few.
In January 2017, an Asilomar Conference on Artificial Intelligence, held at the same venue, attempted to recapture that earlier, successful effectiveness at producing effective guardrails and safeguards. The participants proposed 22 general recommendations as best practice by developers of AI.
Alas, none of the conditions that made the 1975 Asilomar BioEngineering Moratorium work exist in today’s AI realm. In fact, the necessary conditions aren’t anywhere on the horizon. Unlike 1975, you will never get all parties to agree. Not those perceiving trillions in profits or the prospect of becoming founding masters of a wholly new world. And certainly not those innovating in secret Himalayan or Siberian or Gobi labs.
In fact, nothing is more likely to drive talent to those secret facilities, than imposition of a coerced or resented suspension. In much the same manner that 1950s McCarthyite paranoia chased rocket scientist Qian Zuesen away from the U.S. to Mao’s China, thus propelling their nuclear and rocket programs.
Nor will a moratorium be heeded in the most dangerous locus of secret AI research, funded lavishly by a dozen Wall Street trading houses.
Setting aside notions of unlikely moratoria, let’s go back to more general portrayals of cyber-doom. Scary situations propounded by OpenAI Founder Sam Altman include scenarios of some runaway super-AI designing a bioweapon or using trading bots to wreck the world financial system.
But slower disasters are possible. For example, suppose the systems become so ingrained in society that – even if never intelligent or malignant – they cause humanity to ‘drift’ into uncanny realms. Altman believes it has already begun. “Older people use ChatGPT like Google. People in their twenties and thirties use it as a life adviser.”
Indeed, teachers and professors are already reporting a general decline in students’ interest – and ability – to read anything that takes more than a few minutes. For sure, there are other ways to present and contemplate complex thoughts. Many technological revolutions across the last four centuries advanced what humans can see, know and perceive. So, one should not fall into the trap of being an old grouch, like Socrates denouncing reading and writing.
Still, each of those past tech revolutions were highly disruptive and not always with immediately happy outcomes. In this case, rather than increasing our effective intelligence with AI assist, we might instead be throttling the raw inputs, training ourselves to be increasingly passive.
Here one calls to mind a great scifi novel, Mockingbird, by Walter Tevis (author of The Queen’s Gambit and The Man Who Fell to Earth). A super-AI guides nearly all humans into permanent adolescence as pampered college sophomores, taking basketweaving classes.
Reading is forgotten and all of us gradually age, while too distracted ever to have children. A supremely gentle extinction.
Even at the most mundane level, “industry standards” may be less a matter of foresight or planning than happenstance. UCSD physicist Brian Keating worries about technological ‘lock-in,’ that early commercial success sometimes locks into place a set of standards that forces compliance for years or centuries, as in the way that the width of two horses pulling Roman carts wound up copied in 19th Century railroad gauges and their tunnels, through which most modern machinery shipments must fit.
Keating worries that the vast infrastructure now being built to service LLMs will wind up constraining all future investments in Artificial Intelligence because of simple investment inertia. Anyway, who says conscious awareness and decision-making cannot shift to another higher level? Suppose that a new AI master caste deems us biorgs to be useful as squishy but adaptable worker drones with supple hands – then we could be heading for a hive-mind state, a collective organism more akin to a termite hive or a colony of squirmy naked mole-rats.
A pessimist might see our organic brains stalling out, or becoming part of a self-referencing swarm, rather than a pioneering vanguard of exponentially improving geniuses. Envision Aldous Huxley’s ‘delta and epsilon’ castes, consisting of efficient, self-reproducing and disposable workers, while the Brave New World alpha and beta levels are staffed by helium-chilled intellects, cool and unsympathetic.
In an interview for Noëma Magazine, former Google head Eric Schmidt described three trends that may lead to rapid changes in Artificial Intelligence. One of these is enhanced agency, a word that’s also proclaimed loudly as the next step by Sam Altman and – with the rapidity of a fad – by so many others. An agent can be understood as a large language model that can learn something new and then apply that learning outward, proceeding to take actions on its own.
This is from Nathan Gardels’s interview of Schmidt in Noëma: “These agents are going to be really powerful, and it’s reasonable to expect that there will be millions of them out there… What happens then poses a lot of issues. Here we get into the questions raised by science fiction… at some point, these systems will get powerful enough that the agents will start to work together. So, your agent, my agent, her agent and his agent will all combine to solve a new problem.”
So, there’ll be “millions” of ‘em “working together.” Indeed, Tether CEO Paolo Ardoino predicted in early 2025 that within a decade there might be a trillion AI agents with their own wallets, autonomously transacting via Bit- or stablecoins, making decisions and spending vast amounts without human oversight.
The Doomers Shout ‘It’s obvious!’
J. Storrs Hall, in Beyond AI: Creating the Conscience of the Machine, asks “if machine intelligence advances beyond human intelligence, will we need to start talking about a computer’s intentions?”
Among the most-worried is Swiss author Gerd Leonhard, whose new book Technology Vs. Humanity: The Coming Clash Between Man and Machine coins an interesting term, “androrithm,” to contrast with the algorithms that are implemented in every calculating engine or computer.
Some foresee algorithms ruling the world with the inexorable automaticity of reflex, and Leonhard asks: “Will we live in a world where data and algorithms triumph over androrithms… i.e., all that stuff that makes us human?”
Exploring analogous territory, Heartificial Intelligence by John C. Havens also explores the looming prospect of all-controlling algorithms and smart machines, diving into questions and proposals that overlap with Leonhard.
“We need to create ethical standards for the artificial intelligence usurping our lives and allow individuals to control their identity, based on their values,” Havens writes. Making a virtue of the hand we Homo sapiens are dealt, Havens maintains: “Our frailty is one of the key factors that distinguish us from machines…”
…which seems intuitive , until you recall that almost no mechanism in history has ever worked for as long, as resiliently or consistently — with no replacement of systems or parts — as a healthy 70-year-old human being.
They’re taking all the jobs!
The list of potential perils from artificial intelligence in its varied forms goes on and on. But one that likely beats all others at garnering fretful attention is its potential for driving human unemployment.
Technology observer and author of Out-of-Control Kevin Kelly asks: Will AI be used to mostly augment human workers or mostly work independently to potentially replace workers? Many comparisons are made with the Industrial Revolution that rendered many jobs and professions obsolete.
Over many decades, many more and better jobs were created than lost to the new machines, though economists do describe the “Engels Pause” — a period during early industrialization when profits soared but wages stagnated, until a mix of enabling technologies and social reforms re-established some social fairness.
Only this time, Geoffrey Hinton and others predict something much more dire: that the benefits of AI may accrue entirely to the ‘owners of compute,’ while everyone else becomes economically irrelevant. Some industries are already affected and there is a susurration of Luddite-type movements or proposed boycotts of businesses that jump onto the trend without a care for the welfare of their workers.
Unlike the revolution in factory robotics, which in the 1990s caused job losses in manufacturing while propelling a steep rise in productivity, we are now seeing substantial layoffs at firms that provide bureaucratic or office management services.
Anthropic CEO Dario Amodei predicts that AI could wipe out half of all entry-level white-collar jobs within five years, pushing unemployment as high as 20%. Already, occupations most exposed to AI have seen a 13% decline in employment.
Soon, by dour projections, there will be impacts in fields ranging from truck-drivers to radiologists. Can you claim – with quaint 2015 confidence – “my own field will never be taken over by machines!”
Tell it to the creative arts, where musicians and movie-makers, book cover artists and game designers all feel a sudden frisson of incipient panic.
There will also be benefits in improved productivity and creative solutions. Folks like Peter Diamandis claim the AI Wave will generate enough wealth, even abundance, that older definitions of poverty – filth and hunger – will be distant memories.
As Robert A. Heinlein portrayed in his prescriptive utopian novel Beyond this Horizon, “Of course food and shelter are free.” Yet will that just redefine poverty? With inequality and unfairness continuing on a higher plateau? And will the human majority face a demoralizing plague of ennui, borne of a sense of unneeded uselessness?
Perhaps condemned to disheartening make-work, like Depression-era relief projects. Among suggested palliatives against AI-driven unemployment could be Universal Basic Income, with many variants in economic treatises and futuristic novels. Philip José Farmer’s Riders of the Purple Wage portrays one version as liberating.
UBI can fold into honorable work. Along any business street, count the nail salons, once rare, now everywhere. Millions of technicians and clients take pride in a meticulous skill that makes them happy and keeps incomes flowing. Nor is that the only case. Do you have a gym coach? If such trades had not erupted already, would it behoove us (or benevolent AI) to invent some?
Peter Diamandis suggests that AI-driven abundance would mean rapid price deflation. If countered (or made invisible) by a Dynamic VAT, it could adjust to keep prices flat, with revenues distributed en-masse. Hence, benevolent AI abundance meshes with – and finances – UBI.
As white-collar jobs get AI-obsolesced, former independent presidential candidate Andrew Yang in March 2026 declared a 3-year window to enact a wholesome UBI that bridges to Universal Abundance, lest we trigger Luddite revolution. A concept that’s adjacent to UBI would be Universal Human Capital Ownership, wherein every biological-legacy human owns shares in the income generated by a society that’s been made vastly richer by AI problem-solving and overall capital management – a possible variant that Karl Marx could never have imagined.
In an age of superintelligent computers, Bostrom posits that machines will almost completely replace human workers. As marginal costs of computerized labor drop to virtually nil, presumably parasitism and waste will be largely eliminated. At which point the capital interest share in a skyrocketing gross global product will rise to nearly 100%.
Varied thinkers have pondered how the benefits of AI-fostered wealth creation might be spread widely through the idea of universal basic capital, where everyone, not just the top 10%, has a share in “owning the robots.” The aim is not just to constrain concentration of wealth at the top, but to build it from below. New land ownership and raw materials will come through colonization of outer space.
Of course, the precondition here is that, at the beginning of these incredible gains, all human beings would have at least a small positive starting capital. Otherwise, this giant new wealth would simply pass them by. As Bostrom puts it: “For all of our children today who at the beginning of this transition phase have no private wealth, or even a bank account, this prospect is not rosy.”
A new peon caste – non-owners of capital – would be forced to work as ‘hands’ in a gig economy… as identity managers for top AIs, “or to hope for the aid of philanthropists who would distribute their unbelievable newfound wealth to the masses.”
_______________________________________
David Brin’s 1989 novel Earth foreshadowed global warming, cyberwarfare and the World Wide Web. He is the author of more than 30 books, and his first non-fiction book, The Transparent Society: Will Technology Force Us to Choose Between Freedom and Privacy?, won the Freedom of Speech Prize from the American Library Association. This excerpt is adapted from Ailien Minds: Advice About (and For) Our Natural, AI, and Hybrid Heirs (2026).
