You Can’t Sample -Test a Brand That Talks to Everyone Differently

PUBLISHER ARTICLE PREVIEW

From the original article

A few days ago, I got into a pretty heated discussion with my friend Faniprize. She’s the person I tell fresh work stories to when I sense a bigger question in them but can’t name it yet.

This time it was AI agents. Not whether businesses need them, that’s been discussed enough. I was more interested in what happens after a company implements an agent and it starts doing real work, because that’s where the less obvious problems begin.

I told her about a recent project with a recruitment agency. An AI agent was handling part of the communication with potential clients. From a marketing point of view it looked good: it knew the product and the audience, the tone was right, and the conversations felt personal rather than templated.

Individual conversations seemed fine. Then we put several next to each other.

In very similar situations, the agent had made slightly different promises. It gave one person one timeline for getting back in touch and another person a different one. In a few conversations it was basically saying “we can be flexible here” about individual terms, although the company had never given it the freedom to agree to those terms.

There was no dramatic hallucination, which made the problem harder to notice.

Marketing teams naturally spend a lot of time on tone of voice and positioning. A recruitment company shouldn’t sound like a crypto startup, and a bank shouldn’t communicate like a fashion brand. But underneath the way a brand speaks, there’s another layer: what is the agent actually allowed to promise on the company’s behalf?

A beautifully written message doesn’t help much if the client later discovers the terms it discussed don’t exist. At that point it’s no longer a communication issue. It’s a reputation issue.

We’ve taught agents to sound like the company. Now we need to check whether they decide like it.

The normal check is to read a handful of conversations. Is the product information correct? Did the agent follow instructions? Did it say anything strange?