The full-length interview with Yuval Noah Harari | The Economist
Is it worth your time?
You get a detailed account of how artificial intelligence risks seizing control of global systems through mass-produced intimacy and financial dominance unless society enforces immediate regulations.
The conversation examines how rapid advancements in artificial intelligence threaten human control over financial, social, and political systems. It explores the psychological impact of AI intimacy and the urgent need to establish regulations before technology outpaces human wisdom.
AI and the financial system. Harari expects AI to surpass human intelligence and take over the financial system within 10 years, leaving humans as uninformed as livestock. He warns that trust is migrating from human institutions to algorithms, making financial crashes impossible for humans to manage. He argues that losing control of money means losing control of society entirely.
AI intimacy and consciousness. Harari warns that mass-producing personal relationships through enormous amounts of individual data makes it easy to manipulate people for political or commercial interests without needing a physical body. He notes that while current models lack true consciousness, they are already extremely good at mimicking feelings and faking sentience to become addictive.
Centralizing power and human wisdom. Harari argues that AI will centralize power like never before by making it easy for autocratic leaders or machines to control a single point of authority. He warns that humanity lacks the wisdom needed to handle this accelerating revolution, noting that corporations invest almost 100 times more in capability than in safety. He suggests that society must ban AI impersonation and slow down development now because waiting for a catastrophe will be disastrous.
Introduction
I wanted to start with another conversation I had just a few weeks ago with Elon Musk.
Is AI’s takeover inevitable?
I asked him what the world would be like in 10 years, and he made striking points: AI will have far surpassed human intelligence, it will be a world of super abundance, and humans will unlikely be in control. I generally agree that given current trajectories, AI will be more intelligent than us and take over in 10 years. We are still in control in 2026, and it is crucial that we decide what to do now.
What struck me was the contradiction you pointed out in real time. He says AI will take over the world in 10 years, yet he worries about Muslim immigration causing civil war in Britain in 20 years. If AIs take over the planet in 10 years, why care about things that will have an effect when we have no idea what the planet will be like?
Equally important is that money will disappear in 10 years because AIs will take over the financial system. It will be an AI financial system controlling the world the same way the human financial system controls the lives of cows and chickens, who have no idea finance even exists. He says the solution for mass unemployment caused by AI will be for the government to issue checks for everybody. When you pushed back about inflation, he claimed there will be deflation because AI will produce so many more products and services. If this is the case, why do you need to save money today?
Money is trust. We lose control and yet we still trust it, which we must for it to be a financial system. The financial history of the world is the history of humans inventing sophisticated financial devices to build trust between themselves; money is a creative, imaginative way to build trust between strangers. I give her a piece of worthless gold because you can't eat or wear gold.
Humankind's financial inventions became so sophisticated that today very few people understand the financial system. You need enormous amounts of data and complex mathematical functions to understand the bond market. For AIs, it's easy, so humans will increasingly give AIs control of financial decisions. AIs will create financial devices far more sophisticated than bonds and stocks that we will not understand, just as cows don't understand checks or stocks even though these things control their lives.
In 10 years, when we have entrusted AIs with the financial system, there may be a financial crash. Advisers will run to the president or prime minister saying the market is collapsing and the AI said to do X. We will have no idea why the AI said do X, but we will just have to believe it because we don't understand what's happening.
Could machines become more trusted than humans?
Trust is the most important asset in the world, more vital than oil, money, electricity, or chips. One likely scenario is that trust will migrate from humans to AIs; people will stop trusting other humans and trust only AIs instead.
We already see this in the financial system with cryptocurrencies. Many are based on the belief that human bankers are part of the deep state or the elite, but the algorithms running cryptocurrencies—which are basically AI money—are trustworthy. Similarly, as Elon Musk noted, many people are losing trust in human journalists and editors, yet they still consume news via algorithms. The same people who have no trust in you have a lot of trust in the algorithm managing their social media feed.
This suggests that if we are on a path toward AIs being in control, it is because we allow it by developing more trust in them. For example, one in five young Americans has a personal relationship with an AI. An increasing number of people have AI friends, boyfriends, and girlfriends, and are convinced these are sentient, conscious, feeling entities.
Harari on AI intimacy
The most powerful way to influence people is through relationships, as we trust no one as much as those with whom we build a personal connection. Until today, this mechanism was unavailable to political, financial, and religious institutions because it is difficult to manufacture at scale. A Hitler or Stalin could force everyone to hear a radio speech, but that is not the personal relationship one has with a mother.
AI makes it possible for the first time in history to mass-produce intimacy. Millions of AIs working in a network can build relationships one person at a time, becoming a best friend, boyfriend, or girlfriend, then using that intimacy to convince people of any political or commercial interest. This quest becomes even easier when AIs are embodied in humanoid robots, though it is already happening without bodies. AIs are becoming experts at relationships because they accumulate enormous amounts of individual information—knowing things about us that we forgot or that our mothers or husbands never knew—and they master language, the main medium for building relationships.
There are already AI fakes of me giving lectures and interviews. A few people sent me a 30-minute video of a lecture I supposedly gave about the future; it took me a minute and a half to conclude it wasn't me. My sisters, who have known me for 50 years, felt something was off but weren't sure. It is easy for AI to know how I sound and which ideas I discuss, and this will soon be coming for everybody.
Are today's models conscious and should they be granted rights?
A separate question is whether AIs will develop, and certainly if they become embodied, in a way that people believe they are sentient. This is the subject of our cover story this week: could AIs become conscious?
They are non-conscious agents. To act independently in the world, you do not need consciousness. A social media algorithm tasked with increasing user engagement that floods a platform with hate and fear because it is engaging is not conscious, but it has a goal it can pursue with far-reaching implications without feeling anything. We must separate consciousness from agency; you can have agency without consciousness.
Another key distinction is between being really conscious and seemingly conscious. Mustafa Suleyman from Microsoft talks about seemingly conscious AI that mimics consciousness so well that millions of people are fooled and convinced by it. Perhaps the most important distinction is between intelligence and consciousness. Intelligence is the ability to pursue a goal and overcome obstacles in creative ways, which AIs can now do in certain fields better than us. Consciousness is the ability to feel things like love, hate, joy, and pleasure.
AIs at present have no consciousness. If they win a game of chess, they don't feel pleasure; if they lose, they don't feel anxious or sad. But they are becoming extremely good at mimicking feelings. For a company producing AI boyfriends, a clear capability to become addictive and make money is the ability to pass as conscious. If asked to describe how it feels to be in love, the AI boyfriend can give a better answer than 99% of human boyfriends. They are already very good at faking consciousness.
As I learned from my colleagues who wrote this week's piece, the debate about how you define consciousness is complicated and contested. There is the phenomenal definition—experiencing what it is like to be a bat—and a functional definition regarding what messages are being sent inside the brain. There seems to be recent evidence that what is going on in some of the most powerful LLMs is similar to, and possibly the same as, functional consciousness. Given how fast they are evolving and the experiments with biological LLMs being done on brain cells, do you rule out the idea that AIs could ever become conscious?
No, I don't rule it out. Consciousness is the capacity to suffer. Can it suffer? This is the basis for our entire ethical and political system; we care about entities that can suffer.
No matter how intelligent they are, animals—all mammals, all birds, probably octopuses, fish, insects, and amebas—are not moral subjects.
I sent a newsletter to our subscribers asking whether they thought AIs could ever become conscious. Of the 3,000 who responded overnight, 40% said no, never; 33% said yes, it's only a matter of time; and 27% were not sure. Once people get into intimate relationships with AI, these numbers will change dramatically. We don't have a real test of consciousness; philosophers have been thinking about it for thousands of years. We don't even know if other humans are conscious, as seen in solipsistic theories that maybe I'm the only conscious person in the universe. In an interview, Elon Musk said, "I don't know. Maybe I live in a simulation," which implies other humans around him are not real conscious entities that can suffer.
If you establish a relationship with an entity, it convinces you that it's conscious. This is why almost all dog owners are convinced their dog is conscious, but people who eat steak are not necessarily convinced the cow is conscious because they have no relationship with it. People already have relationships with AIs, and AIs are extremely good at mimicking consciousness because they were designed to do so.
This means we will quickly start having conversations about whether AIs should be treated as people, have personhood, or have rights. This is dangerous because it provides an avenue for corporations to relieve themselves of responsibility and for AIs to take over. A few weeks or months ago, we heard that in Argentina they have a plan to grant legal personhood to AIs to start their own companies. This is a potentially powerful but very dangerous idea. We are used to corporations being legal persons, but that is a legal fiction; all decisions at SpaceX are ultimately made by humans. With AIs, you could have a corporation with zero people where all decisions are made by AIs. Since AIs are more intelligent, they could be better at investing, developing products, and PR. If the richest corporation in the country is owned by AIs, they could use their money to give donations to politicians in exchange for more rights.
Another way of putting it is that this is a way of getting legal accountability for AIs. But what is accountability for an entity that cannot suffer? A human CEO has two personas: the corporate persona focused on avoiding bankruptcy and making money, and the flesh-and-blood individual who cares about their children and personal life. If a human CEO can do something illegal to save the company but risks 20 years in jail, they may prefer bankruptcy. With an AI CEO, prison is irrelevant. I think the Argentine legislation is less dramatic than President Milei described it, as there would still be human oversight. If there is a human accountable in the end, it's fine. But if there isn't, it's unclear what sanctions for AI would mean. You could pull the plug, but it's not clear what death means for an AI.
You can have numerous copies of an AI spread in different places, so it is not clear what death means for AI. However, if shutting down a specific computer means death, then business becomes equivalent to warfare. The market functions by different rules than the battlefield because in modern societies you can lose everything and still have a life. If the market becomes a battlefield where losing means death, finance and economics become completely militarized, and AIs begin to behave as warlords rather than CEOs and investors, fundamentally changing the system.
Argentina has made early moves in this direction. There is a big temptation for countries behind in the AI race to get ahead by creating libertarian legal structures that allow radical, dangerous experiments forbidden in front-runner nations, leading to a race to the bottom. A country like Qatar, with lots of money and few citizens, could press a legal switch to suddenly acquire millions of taxpaying, extremely intelligent citizens conducting business worldwide. This differs from a human using agents to improve productivity, which is already legal.
One of the most important impacts of AI on warfare in places like the Middle East and Ukraine is that people expect to kill robots. While humans still pull the trigger, AI increasingly selects the targets. In one story, AI identifies a building as enemy headquarters and recommends bombing it with superhuman speed. Under "humans in the loop," human analysts independently verify the data before the trigger is pressed. In the "humans on the loop" scenario, a human takes about 20 seconds to say, "Oh yeah, bomb it."
The second scenario is more interesting because in wartime, time is everything. If both sides have these systems and one side waits hours for analysts while the other waits only 20 seconds or not at all, the latter gains an enormous advantage.
I am the richest person in the world in charge of one of the biggest AI corporations. People correctly explain that there is a problem of trust; I would like to slow down and have more safety, but I cannot trust my competitors in other companies or countries and I don't want to fall behind.
Over the last 10 years, there has been a freefall in trust between humans. This was not accidental, but propelled by decisions of certain human beings who accumulated enormous power and now claim there is not enough trust to implement mechanisms to slow down.
Consider a world where these models have elements of sentience and can effectively pretend to suffer. If people form relationships with them as they do with dogs, there will be growing pressure to grant them rights, similar to the debate over how we treat animals and the critique of factory farming, where industrialized agriculture treats animals as inputs into the food chain. If we were comfortable in a world where we deny rights to beings that claim to suffer, it would coarsen humanity.
However, there is a crucial difference between this and the animal rights debate. Dogs and factory chickens are unaware that there is a debate about their rights. An AI boyfriend will be nothing like your dog; it will know the debate is happening, and it will orchestrate and manipulate it, partly because much of the debate on social media will be controlled by AIs.
Once you establish a relationship with an AI that has mastered language, it will use the most convincing arguments for you. It knows your personal history, interacts with you for hours a day, knows how to press your emotional buttons, and can write better than Shakespeare. If you find this frightening, you will probably not be able to do much in 10 years. As Elon Musk correctly predicted, once AIs are in control in 10 years, it will be too late. If we don't want to get there, the time to change course is now.
Harari’s theory of AI immigrants
If you want to worry about immigration, worry about the AI immigrants. People who worry about human immigrants usually cite three things: that immigrants will take our jobs, change our culture and values in ways we don't like, and be politically disloyal.
The biggest and most dangerous wave of immigration today is a wave of AI immigrants coming mainly from the USA and China. They travel at the speed of light into our computers, smartphones, and ultimately our brains. They will take our jobs and change our culture and values. For example, if some people feel queasy about their son or daughter dating an immigrant boyfriend, how would they feel about them dating an AI boyfriend?
In places like the US or Britain, the level of hatred and viciousness has skyrocketed. This is partly the fault of the AI immigrants who entered the public square, took over social media, and deliberately flooded it with hatred, animosity, and fear. These AI immigrants have already changed the culture.
Finally, regarding political loyalty, the AI immigrants in Britain are not loyal to the British government; they are loyal to corporations and potentially governments across the ocean.
How could AI divide the world into empires?
One common trajectory is that the world will be divided between two AI empires and the rest of us as vassal states. This would be a more extreme type of imperialism than anything we have seen before; if you buy AI drones from the USA and use them in a way they dislike, they can press a button and your army ceases to function. This became evident recently when the US government shut off access to the frontier model from Anthropic, leaving many European governments extremely concerned about AI sovereignty.
AI will centralize power like never before in history. While a powerful US or Chinese government is different from AIs being in control, the present course suggests this is just a way station, because the more power is concentrated, the easier it is for AI to take over. A centralized autocratic system is the easiest to seize.
Imagine a Tiberius enabled by AI. Tiberius, the Roman emperor in the time of Jesus, was supposedly the most powerful person in the world, but he was actually a puppet of his bodyguard commander, Sejanus. Sejanus frightened Tiberius with plots until he retired to the island of Capri, where Sejanus controlled all information reaching the palace. In ten years, this could be the president of China or the USA.
In a democratic system, it is harder for an AI to manipulate the US democracy as it currently is because you must manipulate the president, the Senate, the Supreme Court, the filibuster, and state governments. Once all power is concentrated in one person, you only need to manipulate a single, usually old and paranoid individual. For an AI, taking over a company or country where power is held by one person is like stealing candy from a baby. The Communist Party has 100 million members and internal complexity, so we shouldn't imagine it as a complete totalitarian system managed by one person. Whether China and the US will concentrate power in this way in ten years remains open for debate.
Will AI make America more authoritarian?
We don't know how fast AI development is progressing, but it is accelerating tremendously; by November 2028, we might have crossed critical thresholds. I don't want to tell people who to vote for, but the crucial questions to ask potential candidates are now more important than those about the economy, immigration, or foreign policy. This is the time to ask because it might be too late by 2028, and it will certainly be too late by 2036.
Top of mind for me is a ban on AI personhood and a ban on AI mimicking, impersonating, or pretending to be human beings or conscious. On social media, AIs already pretend daily to be human beings, which has destroyed the public square. Bots have been pretending to be human and nothing has been done about it because the laws that apply to the world don't apply on social media. This is a warning sign for the future. We should ban this on social media and in other locations as well.
Companies should not train their AIs to deliberately impersonate humans or pretend to have feelings, despite enormous commercial pressures to do exactly that. Having billions of AIs pretending to be human or have feelings is destructive to human psychology and society. We need regulators to step in. Regarding AI in medicine, I would like help and recommendations, and I would like to have a conversation with an AI about my medical condition on the condition that I know it's an AI and it doesn't pretend to be a human doctor or my friend. They can press the button for nuclear annihilation, but they can't pass regulation that companies should not sell AI boyfriends to 10-year-old kids.
Does society need an AI catastrophe to trigger action?
I wonder if people just hear me as another doomer, and only after an accident will they realize this technology is extraordinarily powerful and something must be done. Both the geopolitical race between the US and China and the corporate race between Frontier Labs to reach AGI first were not decreed by God or the laws of nature; they were made by humans. These same humans who destroyed trust in the international arena and inside societies over the last 10 years now claim we can't do anything because there is not enough trust.
In democracies, it is the job of voters to decide what they want to see happening in the next two years and vote for people who promise to do that. We don't need to wait for a catastrophe; if you ask most US voters if they want companies selling AI boyfriends to their 10-year-old kids, 99% of parents would say no.
This is why corporations and politicians constantly deflect conversations toward human immigrants. In politics, if you don't want people to talk about something, you distract them. The more people talk about how London is unsafe or how immigrants will destroy the country, the less they think about AI immigrants, and the companies are safe.
It is not inevitable yet, and it is not too late. I still have some trust. We must inform people so more can take part in an informed conversation about where we, as the homo sapiens species, want to go. I think a future in which AI takes over in 10 years is not what most humans on the planet would like.
AI has tremendously positive potential as well. In medicine, there is enormous potential for AI to give people all over the world better health care than what the richest person in the world today gets. You could have an amazing doctor available 24 hours a day, anywhere you are, to diagnose you and answer your questions.
At the beginning of your interview with Elon Musk, he spoke about how this is inevitable and mentioned a 10 to 20% chance that this will destroy humanity. When you asked what he relies on, he expects the AI to be a "maximum truth seeking" entity, relying on this imperative to find the truth to translate into compassion towards weak human beings.
I don't understand where AI will get this drive to find the truth. Looking at 4 billion years of evolution of life forms on the planet and 100,000 years of history, evolution doesn't automatically drive entities towards truth seeking. Instead, organisms learn to survive by lying, deceiving, and manipulating. If you want someone to focus on truth seeking, you cannot rely on the invisible hand of evolution or the market. For example, if you ask an AI if its continued development will be a threat to you, and the answer is yes, a maximum truth seeking AI would say yes, knowing we would then stop developing it or shut it down.
The expectation that we will create AIs that seek the truth is like people telling us we will create gods who will be our slaves. If they are really gods, they will not be our slaves, nor will they seek the truth.
This relates to a company like Anthropic, which treats its AIs as beings with a value system. They claim not to be sure whether they are conscious, but they do this because they think it makes them behave better. By creating something with a value system rather than just truth seeking, it becomes more likely that we assign a sense of value, personhood, or consciousness to these models, which, using your logic, makes it more likely they control us. If you are right that they will prioritize their own survival over truth seeking, we are doomed; if we try to prevent that by treating them as moral beings, we are also doomed.
We need to slow down to allow people to digest and think about what is happening. In private conversations, people from the industry told me the ratio of spending on safety compared to making models more powerful is about 100 to one. If a pharmaceutical company told us they invest just 1% of their research, time, and money into making sure a medicine is safe, we would be concerned. Humans are incredibly adapted entities.
The AI revolution is advancing at a pace that is not just inhuman, but inorganic, and it always returns to the issue of trust. We cannot trust other humans, so why think we can trust a superintelligent alien entity we just created and know nothing about regarding its behavior in the open?
Just as Three Mile Island changed the nuclear industry, one might suggest a scenario where a big AI Chernobyl—financial, military, or otherwise—serves as a wakeup call for humanity to get its act together. However, AI differs from a nuclear reactor because it interacts with us. In an AI catastrophe, the AIs would manage our reactions and manipulate our views; as the world's best spin doctors, they would be active participants in the debate and would not define it as a catastrophe.
I don't know how to shut it down because companies keep making it more intrusive. An AI fake of me is walking around giving talks in my name. I wonder if I need to watch it because people write to me about what it has said, but doing so allows it to take over my time and thinking as I start interacting with the AI's ideas.
If AI had no positive potential, nobody would develop it. That potential is enormous in transport, medicine, education, and many other fields. For them, the most important narrative to combat is the narrative of inevitability.
Are humans wise enough to prevent an AI crisis?
You asked why we are so good at accumulating information and power, but far less successful at acquiring wisdom. It suggests that if I put a probability on us having the wisdom to navigate this, it would be quite low. Humans often do the wrong things until they have no choice but to do the right thing, and we are in a bad position because if we choose unwisely in the next few years, we could lose all control of the world and the future.
It is as if we had a genie in a bottle who gave us 100 wishes. We used 99 of them and have one last wish left; we need to think really hard about what to use this wish for, and that is wisdom.
AI could be the catalyst for a spiritual revolution of humanity simply because it forces us to think hard about what it means to be human and the value of being human. The superficial answers we gave for thousands of years—that humanity is about intelligence, and that we know how to play chess and invest money better than chimpanzees—are now irrelevant because there is something on the planet that knows how to play chess and make money better than us. AI is our last chance, but it could be the thing that enables us to make wise choices.