Forget ‘superintelligence’: error-prone AI nearly sparked world war three this month | Timnit Gebru and Emily M Bender

PUBLISHER ARTICLE PREVIEW

From the original article

The risks of AI aren’t what we think they are, as a recent security incident between China and the United States reveals

Amid a barrage of news stories warning about superintelligent machines rendering humanity extinct, a CNN story describing the opposite scenario – one in which the US military’s reliance on brittle chatbots almost brought the US into war with China – went mostly unnoticed by the public.

The biggest international AI news of the past three weeks was Anthropic engineer Jacob Coxon’s resignation. According to him, OpenAI and Anthropic are “racing straight towards self-improving superintelligence and gambling with our lives”. Coxon’s description of a “terminator” scenario, a machine becoming much smarter than humanity and deciding to wipe us out, captured the public, journalists’ and politicians’ attention, stirring waves of fear-inducing, credulous reporting, and policy actions.

California governor Gavin Newsom issued a mandate to create an “AI kill-switch”; New York governor Kathy Hochul is similarly exploring the possibility of having “AI kill-switches”; and Senator Bernie Sanders and Congressman Greg Casar introduced legislation to “ban artificial superintelligence”.

While so many of us were focused on fictional superintelligent machines that we were told could hypothetically cause human extinction, however, there was a more serious story being reported.

A series of events almost got us to armageddon, not caused by “rogue, superintelligent AI”, but by government reliance on error-prone systems that are marketed by their manufacturers as being close to superintelligent.

A CNN report published on 18 September, which hasn’t been verified by other major outlets, describes how the US “almost started a war” with China, based on an intelligence report that claimed a Chinese ship was transporting components of nuclear weapons. Military personnel made plans to intercept the Chinese vessel, complete with aircraft and soldiers ready to board the ship.

A military