Your complimentary articles

You’ve read one of your two complimentary articles for this month. To have complete access to the thousands of philosophy articles on this site, please

You can register for a free account to have four complimentary articles per month. We will occasionally email you a newsletter, from which you can unsubscribe at any time. We do not sell personal data or otherwise disclose personal information to other organisations.

Books

Nexus by Yuval Noah Harari

Frank S. Robinson ponders our AI future.

Yuval Noah Harari’s 2024 book, Nexus: A Brief History of Information Networks From the Stone Age to AI, concerns how artificial intelligence will shape our future. An AI is trained on a vast amount of information either from large databases or from the internet to see patterns. Using them, it ‘learns’ how to interpret X-rays, for example, by the repeated matching of images with confirmed diagnoses. It can also write very well, having learned the rules of grammar and good usage from the patterns of language in its training texts.

But while an AI might seem to have understanding, it’s actually just a simulation of thought, not thinking as humans think of thinking. An AI evolving into a thinking being – becoming conscious – is probably a long way off, but that’s outside the scope of this book, which is concerned with AI’s implications for the future of democratic societies. Harari relates that in the nineteenth century, development of railroads, steam ships, and other industrial technology generated the resources and power to rule the world, producing an era of colonialism. Now he foresees ‘data colonialism’, with control of data ruling the world, and with AI being even more powerful than those previous technologies.

A guiding metaphor here is Goethe’s cautionary poem The Sorcerer’s Apprentice (1797). The young apprentice, tasked with fetching water, delegates it to an enchanted broomstick, which doesn’t know when to stop, and the apprentice can’t stop it either. Result: flood. Harari draws the lesson: “never summon powers you cannot control.” Yet humanity has quite a track record of setting in motion powerful forces with unintended consequences. This is exacerbated by large scale cooperation, which gives us enormous power, which we often use unwisely or entrust to the wrong people – as Germans did in 1933.

Harari thinks it’s really an information problem: we’ve often built “large networks by inventing and spreading fictions, fantasies, and mass delusions.” This is how we got Nazism and Stalinism – “exceptionally powerful networks held together by exceptionally deluded ideas.” And while those ultimately failed, Harari fears some new AI-built totalitarian regime that’s able to prevent exposure of what it’s doing.

The Story of the Story

Harari posits that our first information technology was the story. Christianity for example has been a story with a huge impact. But a story’s power needn’t depend on its truth. If anything, a false story can have the advantage of simplicity while truth can be complicated and discomforting.

Harari says the backbone of much art and mythology comes from ‘biological dramas’ that press our emotional buttons: Who will be alpha?; us-versus-them; good versus evil; purity versus pollution. The latter particularly afflicts India, whose caste system stigmatizes lower castes as impure.

In Harari’s telling, a naive view of information assumes that the antidote to error is more and better information. This was the belief at the information age’s onset. Gresham’s law says that bad money drives good money out of circulation. We’re seeing an information equivalent, with a tsunami of bad information pushing out the good. The book cites here the witch hunting hysteria between about 1500-1700, excited because Europe was flooded with information spread by a new invention (the printing press) about a vast Satanic witching conspiracy.

Harari sees today’s burgeoning populist movements as similarly information-related. He notes the ‘do my own research’ trope, which “may sound scientific but in practice it amounts to believing there is no objective truth.” I’d say it means finding pseudo-information on the internet. Further here, populists rebel against ‘know-it-all’ elites, whose assertions are rejected as smokescreens to validate their power and status. This right-wing trend oddly echoes the woke left similarly holding that everything is about power, or oppressors versus the oppressed. Yet while populists are cynical toward conventional information sources, they trust dodgy websites, or Trump.

Ai-Da giving evidence at the House of Lords
Ai-Da at the House of Lords © Aneonv36 2022 Creative Commons 4

Self-Correcting vs Totalitarian Systems

Another concept Harari employs is self-correcting versus non-self-correcting systems. Science is often the former, religion often the latter; or, democracy largely the former, totalitarianism largely the latter. You might suppose AI is self-correcting, given the ‘machine learning’ idea. But if AI supersedes all other information sources, that’s a recipe for non-correction, as AI doesn’t know what it doesn’t know.

In totalitarian systems, all information flows to the center but it’s too much for the center to properly deal with, resulting in costly mistakes without mechanisms for correction. In democracies, information and decision making are typically more dispersed, which is better, and which also makes it harder for an AI to gain pervasive control. On the other hand, AI is harder for humans to control. You can’t send an AI to the gulag.

Harari discusses modern surveillance technologies that make 1984 look like a libertarian paradise. He details Iran’s high-tech system for enforcing hijab requirements, for example. Ubiquitous cameras with facial recognition spit out smartphone warnings in seconds, with punishments for non-compliance. Harari foresees Western populations being governed by ‘social credit’ systems wherein algorithms monitor behavior and rate us for conformity to specified norms. China already enforces just such a system.

Yet information is crucially important to democracy, too. People can’t debate and reach decisions unless they know what’s going on. That indeed is why broad-scale democracy is only a modern development. In earlier times few had access to education, or reading – not even TikTok. Today we’re inundated with information; but there’s a huge problem. To maximize advertising revenue, the algorithms managing platforms like Facebook or YouTube are engineered to promote ‘engagement’, which in practice means pushing content that gets people’s juices going. Thus angry nonsense crowds out moderate rational discourse on social media. Harari details how Myanmar’s murderous pogrom against the Rohingyas was enflamed by Facebook actively promoting extremist voices, for example.

Don’t people resist manipulation through misinformation? Too often, no. To do so we need a foundational background knowledge and understanding of how the world really is. People once got that from radio and newspapers but now we are fed so much junk from smartphones it’s hollowed out our brains!

Harari argues that democracy is threatened not just by digital totalitarianism, but by its seeming opposite, digital anarchy – the cultivation of chaos in both public debate and social order. We’ve seen how bad actors try to mess with elections. And whereas bots were initially used for spreading dubious content created by humans, now AI can make such content diabolically seductive: Harari cites a study wherein people proved good at seeing through human-produced disinformation, but fell for craftier AI-generated stuff.

A key to social order is social trust: the feeling that the institutions and structures within which we function can be relied upon. But polls show people have declining trust. (A partial cause may be smartphones reducing face-to-face interactions.) Believing that others are not trustworthy can become self-fulfilling when we behave in accordance with the belief. Harari fears that with AI, and especially with all the nonsense flooding the infosphere, people will start to lose the ability to trust anything or anyone. This will be deadly for the future of human society.

Life in a Box

But maybe, Harari suggests, we won’t even need other people any more, with AI becoming everything to a person – providing one’s whole nexus with the world and thereby shaping our feelings about it. He doesn’t mention the 2013 film Her, a romance between a man and a (clearly conscious) computer operating system; but Nexus does see AI eating human culture and gushing out a replacement wherein we’re reduced to mere accessories. Another movie coming to mind is 2008’s Wall-E, where humans are strapped into floating capsules where they’re kept fed and entertained, mindlessly.What would be the meaning of ‘democracy’ in such a world?

A further big issue is what Harari calls the ‘alignment problem’ – AI tackling tasks in ways that don’t align with human needs because its mind works differently. As an illustration, philosopher Nick Bostrom hypothesizes a program told to maximize paper clip production, resulting in a world full of paper clips but without humans. Remember The Sorcerer’s Apprentice.

The alien nature of AI ‘thinking’ was demonstrated in 2016 when a program, AlphaGo, battled a top master of the game Go, which is considered more complex than chess. One AlphaGo move baffled expert observers, confounding all they knew of Go strategy, yet it proved a killer move.

The alignment problem was considered serious by OpenAI, developer of Chat GPT-4, so they put it to the test, by asking GPT-4 to crack a CAPTCHA – one of those online puzzles designed to distinguish humans from bots. GPT-4 could not solve it alone: so it went to the TaskRabbit website to engage a human to do so. The human was suspicious, and asked, “Are you a robot?” GPT-4 said “No”, claiming that a visual impairment impeded its solving the puzzle. The human then complied with the request.

As Harari notes, this kind of weaselly behavior was never envisioned by those programming GPT-4: but given the goal of overcoming the CAPTCHA, it figured out a way, without reference to moral reservations, which clearly hadn’t been programmed into it. (Harari does recognise that the word ‘goal’ here is problematic, implying a desire to achieve something, and GPT-4 has no desire, for which you need a mind.)

A Different Future?

While Harari talks in terms of a coming global AI tyranny, the world seems increasingly divided between autocracies and more or less democratic nations. But whereas in the Cold War Mutually Assured Destruction prevented nuclear conflict, stealthy cyberwar may be very different. Harari nevertheless vaguely expresses hope for global cooperation – yet says that if the law of the jungle really reigns, the alpha predator could be AI.

There’s a different perspective on all this, which Harari doesn’t address. We think of ourselves as strictly biological entities, in clear distinction from anything mechanistic. Yet that distinction is already crumbling, with technology used to repair and even supplant and enhance parts of ourselves. In a 2013 article I wrote for Humanist magazine, ‘The Human Future: Upgrade or Replacement?’, I foresaw not conflict between us and machines, but rather a convergence – a continued blurring of the dividing line, with our biological aspects receding in favor of a more mechanistic upgrade. Will this be Humanity 2.0? Yet such people would still have minds like we’re accustomed to. They’ll still be our children. They’ll still be us.

Frank S. Robinson is the author of eight books, including The Case for Rational Optimism. He blogs at rationaloptimist.wordpress.com.

• Nexus: A Brief History of Information Networks From the Stone Age to AI, Yuval Noah Harari, Vintage Books, 2024, 544 pages, £10.99 pb