Why Procurement Is Becoming AI’s Most Powerful Regulatory Tool

PUBLISHER ARTICLE PREVIEW

From the original article

For all the attention being paid to AI legislation, some of the most consequential rules governing the technology may end up coming from a truly unexpected place: procurement departments.

Like many regulatory beginnings in the United States, California offers an early example: the state has built specific requirements for buying products and services that use generative AI. State entities must assess the risk associated with GenAI purchases, with moderate- and high-risk uses receiving additional scrutiny from the California Department of Technology (CDT). Solicitations require GenAI disclosure language, and vendors may have to disclose their use of the technology even when AI is incidental to the product or service being purchased.

Even more interesting is what happens after the deal is done. California’s GenAI procurement guidance extends oversight beyond the initial purchase. Contract managers are expected to monitor GenAI technology over time because certain changes to the technology can trigger another risk assessment.

This changes things for vendors since getting through procurement is no longer a matter of answering a security questionnaire and handing over some documentation. AI governance is starting to become part of the price of admission for doing business in California. And while this is the current situation in California, it often leads other states when there is not federal legislation in place, meaning that companies would be shortsighted to view this as a California procurement problem.

How did procurement departments become ground-zero for AI governance? Governments can spend years debating legislation, but procurement rules can be nimbler because they have a much more immediate audience: anyone who wants the contract.

Without comprehensive AI legislation, buyers have found another way to establish their own expectations around AI. They can decide what they need to know about a vendor’s use of the technology, what evidence they expect to see, and what level of risk they are willing to accept before signing a contract. For large buyers, particularly governments and major enterprises, those requirements can carry considerable weight.

Private-sector procurement teams have their own reasons to start asking questions around things like where AI is being