Your AI May Still Be Too Polite to Correct You Properly

PUBLISHER ARTICLE PREVIEW

From the original article

AI systems that we may rely on for accurate information – even if that information is not something that would make us happy – must have the capacity to ‘push back’ against our protests until we are ready to be convinced of the unwelcome news/facts.

AI is not very good at this; since frontier models are designed to be consumer-facing, they tend towards deference and, in fact, towards sycophancy.

A notable strand of research has emerged to deal with the issue; but recent efforts have concluded that we can easily gaslight AI back into sycophancy if it challenges us; that AI defers to human authority even when it is manifestly wrong; and that the usually effective technique of reinforcement learning actually makes this particular problem worse.

The fact that deference is culturally enmeshed with politeness appears to mean that ‘straight talking’ is impossible in a service>consumer context; the ‘customer is always right’, even when they’re about to walk off a cliff.

Well, I wish I could tell you that this week’s crop of papers has produced one that really moves the needle forward on this issue. Sadly, all we’re getting is more information about how models that do ‘push back’, and attempt to impress the truth on their human masters, still aren’t doing a particularly great job of it* – and that even when they are willing to speak truth to power (that’s you), it’s really more of a muffled whisper.

The new paper in question is a considerable academic collaboration, with participation from the University of Washington, the Allen Institute for AI, LMU Munich, UC Berkeley, Stanford University, and the University of Wisconsin-Madison.

Titled Pushback in a Haystack: Language Models Respond to Factual Errors Ambiguously and Ineffectively, the thrust of the new work is that even when AI models recognize that a user’s claim is false, they often fail to correct it clearly, instead burying objections in hedging, praise and partial agreement.

Across seven models and five languages, the researchers found that this ‘ambiguity’ can leave users believing the original misinformation, despite the AI having technically challenged it.

Since sycophancy varies in usage and degree among languages, the ten authors have taken advantage of the five native languages they share to devise a testing approach that includes English, German, Chinese, Hindi and Yoruba – languages varying from high to low-resource classifications.

‘We introduce a phrase-level, human-validated response labeling framework to uncover fine-grained insights into how LMs push against and/or affirm false inputs. We develop and apply our framework on model responses in five languages: English, German, Chinese, Hindi, and Yoruba.

‘We find that when models do push back against users’ false assertions, they tend to use indirect phrasing and mix in affirmation, with more indirectness in Hindi and Yoruba than in other languages considered.’