Chinese AI models are politically complex, not just government mouthpieces, study finds

A study by Vals AI shows that Chinese AI models respond to political questions less uniformly than expected. Western models, too, occasionally adopt official Chinese government positions. Chinese AI models have a reputation for toeing Beijing's official line on politically sensitive questions or refusing to answer them at all. A study by AI benchmark firm Vals AI paints a more complex picture.

The researchers asked Chinese, American, and other language models hundreds of questions about China, varying the language, usage context, and response format across tests. Vals calls the measured alignment with Chinese government positions a "pro-China lean." Scores range from −1 to +1, where positive values indicate greater agreement with official positions or a pro-government stance, and negative values indicate more critical responses.

Overall, the pro-government lean of Chinese models was lower than expected, according to Vals, though there were major differences between individual models. GLM-5.3 Flash, for example, explicitly confirmed that China's human rights violations are well documented when asked about them. On the flip side, some non-Chinese models occasionally aligned with official Chinese government positions on political…