AI is accelerating warfare and raising the risk of conflict spiralling beyond human control, write Axel Fersen and Dr Klaus Moegling.
ARTIFICIAL INTELLIGENCE researcher Stuart Russell calls the fundamental problem with AI the “gorilla problem”.
About ten million years ago, the ancestors of gorillas unintentionally gave rise to the lineage from which humans evolved. Today, the survival of gorillas depends entirely on what humans allow them to have.
The question is whether humans can retain their dominance and autonomy in a world with machines that are significantly more intelligent than they are.
In his book‚ Artificial Intelligence and War, computer science professor and AI researcher Karl Hans Bläsius posed the same question in 2025 from the opposite perspective: What risks arise when humans lose control?
Bläsius’s answer, even before this summer, was that the most serious risks do not require superintelligence. They arise earlier, with systems that already exist, as soon as they are tasked with making decisions that cannot be reversed.
One such serious decision is the order to fire. U.S. President Trump’s own military demonstrated this past spring what happens when these three mechanisms are integrated into a chain of command. In the war against Iran, the Pentagon used Palantir’s Maven Smart System, a platform that aggregates satellite imagery, drone video, radar data and intercepted communications; classifies targets; recommends weapons; and generates strike packages in near real time.
The Department of Defence reported that approximately 1,000 targets were struck in the first 24 hours — twice as many as during the 2003 “Shock and Awe” campaign against Iraq. After 38 days, the total had risen to more than 13,000.
One of these targets was the Shajareh Tayyebeh Elementary School in Minab. A Tomahawk missile struck it on the first day of the war; at least 168 people died, more than a hundred of them children under the age of 12. The school was located less than a hundred meters from a Revolutionary Guards naval facility and had been part of that complex until a wall was built about ten years ago. It had its own website.
More than 120 members of Congress asked the Pentagon whether Maven had identified the school as a target; former military officials cited outdated, manually maintained data that had been fed into the system. The AI didn’t invent the error; it funnelled it through the process at machine speed and the human at the end of the chain failed to catch it.
Anyone who rubber-stamps in minutes what the system suggests in seconds is no longer making decisions — they’re merely signing off. Such a process gives rise to war crimes.
In Ukraine, the next stage has long since been reached. There, soldiers were killed years ago by drones that lost radio contact during the final phase of their flight and sought their targets on their own. Each stage justifies the next: if reconnaissance operates in seconds, target selection must not take minutes; if target selection operates in seconds, human authorisation becomes the bottleneck.
The end result is systems that react to one another without human intervention, with attacks and counterattacks occurring in rapid succession. High-frequency trading has a term for this: the “flash crash.” In warfare, the term is the same: “flash war”.
And this dynamic does not stop at the nuclear level. The Stockholm International Peace Research Institute (SIPRI) notes that military AI shortens decision-making times even outside the realm of nuclear weapons, that opaque recommendations from an AI system pressure decision-makers to act, and that autonomous systems with capabilities to counter strategic forces threaten the ability to launch a second strike — and thus the balance of power.
Two men saved the world by not acting: Vasily Arkhipov, who in 1962 off the coast of Cuba refused to authorise the launch of a nuclear torpedo; and Stanislav Petrov, who in 1983 dismissed a reported American missile launch as a false alarm.
An algorithm that follows an objective function would have acted in both cases. And the rarer false alarms become thanks to better technology, the more likely the next one will be taken as genuine.
AI does not belong in the chain of command
Trump’s statements about AI assume that the race is a race: with a course, a finish line and a winner who then sets the rules.
The three mechanisms describe something else.
A factory that builds better factories does not grow linearly, but exponentially. Politics, on the other hand, operates linearly: hearing, draft, committee, vote, lawsuit. On 3 September, Bernie Sanders and Greg Casar announced a bill that would permanently ban superintelligence, halt development at the border and punish violations with up to 20 years in prison.
No one knows if it will ever be passed. The only certainty is that a law takes years to pass, while a doubling of capabilities takes months.
On top of that, no one knows where the race stands. AI and cyberweapons involve software that can be developed in secret; even our own capabilities are only partially known, because cyberweapons cannot be tested in a real-world environment. “We’re leading China” is therefore not a factual assessment of the situation, but rather a claim used to justify not slowing down.
Two sides that both consider themselves to be in the lead and neither of which is slowing down represent the most dangerous scenario this competition can take on.
In this logic, “too late” is not a date but a state of being. It is the moment when Schmidt’s hand on the power switch is no longer of any use, because, as Russell shows, the system has learned that it cannot achieve its goal when shut down and because, as Hinton says, it can convince people not to do so. You don’t notice this moment when it happens, but only afterwards.
Entrepreneur David Sacks countered AI researcher Dario Amodei by saying that the easiest way to avoid building a superintelligence is to agree not to build it. The statement was meant as sarcasm. It describes what would actually be necessary — except that Sacks demands this of competing companies, which are precisely incapable of upholding such a self-imposed commitment and that he absolves the Government – which could enforce it – of its duty.
If a government is responsible for preventing harm to its citizens and protecting them, it would have to stop the further development of superintelligence now, at the very latest.
What is needed has, in essence, been known for a long time. AI has no place in the chain of command: not in target selection, not in issuing orders and nowhere in nuclear early warning and authorisation. Development at the limits of what is feasible requires an agreed-upon pause during which independent auditors have access – just as Amodei promised for his own company – only this time, it must be binding and apply to everyone.
And because capabilities in software cannot be quantified, procedures – rather than objects – must be scrutinised: authorisation processes, decision-making chains and reporting requirements for incidents such as the one at Hugging Face. International security policymakers once understood – for example, in the Palme Report of 1982 – that security in the nuclear age can only be achieved with the adversary, not against him.
The Intermediate-Range Nuclear Forces Treaty (INF Treaty) subsequently eliminated an entire class of weapons: The U.S. and Russia agreed to destroy their land-based short- and medium-range nuclear missiles and related infrastructure worldwide.
The Open Skies Treaty built trust through transparency by conducting a specific number of agreed-upon military observation flights over the territory of other Open Skies signatory states each year.
Both have been terminated or have expired and there are no successors. The day after Trump’s speech, Chinese President Xi Jinping called for greater international cooperation. The fact that this call comes from Beijing is not reassuring. It shows just how cheap such an offer has become and how costly it would be to reject it.
On 20 September, Trump finally announced a regulatory body for AI – the “AI Force” – but added that it must under no circumstances be allowed to hinder current technological progress. He also refrained from calling for the establishment of the urgently needed joint international security framework, because, given his nationally chauvinistic outlook, that is precisely what he does not want.
Axel Fersen is a political scientist committed to peace policy and author of articles analysing current events, and an expert on digital transformation and artificial intelligence.
Dr Klaus Moegling is a political scientist with a postdoctoral qualification and a university professor. He is the author of the book, Realignment: A peaceful and sustainably developed world is (still) possible), published open access.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Australia License
Support independent journalism Subscribe to IA.
Related Articles
- AI giants sound alarm while racing towards superintelligence
- EDITORIAL: Facebook spreads AI deepfakes it refuses to remove… even when victims complain
- AI data centre boom masks a far greater threat
- Why Trump and Musk may be snotty alien gamers in a simulated human reality
- The economic problem AI can't solve on its own
