Anthropic’s IPO pitch includes a warning about human extinction

The Claude maker warns its own models could resist shutdowns and cause catastrophic harm. Anthropic has formally warned investors that its technology may pose “existential risks to humanity” in a long-awaited initial public offering prospectus circulated with a small group of partners in recent days. The nearly $1 trillion AI start-up led by Dario Amodei devoted almost a third of its lengthy S-1 filing to detailing “risk factors,” including the potential of increasingly advanced AI models to manipulate, blackmail, and exhibit other unpredictable behaviors.

Anthropic outlined more prosaic risks, including the extreme concentration of its customer base, with close to a quarter of revenue last year coming from just two clients, according to people familiar with the filing. The extraordinary warnings in its risk disclosures add to the challenge faced by public market investors in valuing the leading US AI group when it launches what is expected to be the most highly valued IPO of all time.

The Claude maker also provided investors with a clearer picture of the challenging economics…