AI weapons systems are already here. Algorithms must not decide who lives and dies | Kenneth Roth

PUBLISHER ARTICLE PREVIEW

From the original article

Israel’s conduct in Gaza shows the danger of using AI in warfare. Humans must remain at the center of wartime decisions

As tech leaders warn about the threats of artificial intelligence, most conjure up images of AI agents running amok, such as hacking systems used to run our critical infrastructure, banks or even militaries. Yet one serious danger is mentioned less frequently – the threat posed by AI – empowered lethal weapons. Fully autonomous versions of these weapons are colloquially known as killer robots.

Israel’s conduct in Gaza shows that, in an important respect, such AI weapon systems are already here. An algorithm has been deciding who lives and who dies.

For more than a decade, governments have been meeting in Geneva to stop the development of autonomous AI weapons. In early September, 128 governments met to approve a report that could serve as the foundation for a binding treaty.

But a collection of lawyers sent by Donald Trump and Vladimir Putin teamed up to water it down. Notably, they weakened a provision that requires humans to review military targets developed by AI before a strike.

Fully autonomous weapons remain the focus; the use of AI to determine targets for conventional weapons is not. At risk is the longstanding quest to maintain a “human in the loop” – that is, meaningful or effective “human control” of AI-empowered weaponry.

For many people, talk of killer robots conjures up images of Arnold Schwarzenegger’s The Terminator. I have often thought about the problem in the context of the Tahrir Square uprising at the height of the Arab Spring that led to the ouster of the Egyptian president Hosni Mubarak.

Mubarak’s fate was sealed when the Egyptian army made clear it would refuse to fire on Egyptian protesters. But killer robots would have had no such qualms. If Mubarak had been able to deploy them, he might have stayed in power.

The negotiators in Geneva are not even addressing this problem. The Trump-Putin lawyers made sure that the agreed report referred only to war, not repression.

Today, amid the drone warfare in eastern Ukraine, Russian and Ukrainian troops are experimenting with fully autonomous weapons to overcome jamming of the signals used to control drones. We can expect these nascent deployments to expand.

Yet the “killer robot” metaphor is deceptive. If we look at