Friends don’t let friends feed their data to AI agents

PUBLISHER ARTICLE PREVIEW

From the original article

You may have consented to sharing with Muse, but all your friends sure didn’t.

There’s a growing trend in the consumer tech world: gadgets designed to spy on everything and everyone around you, regardless of whether or not anyone else actually consented to it. Now, with the advent of “AI personal assistants” like Meta’s Muse, you’re being cheerfully asked not just to give up your own data, but also the information of your friends, family, the person you went on a Hinge date with five years ago — and virtually anyone else you’ve ever come in contact with.

Since Muse was released last month, the agent has already been downloaded millions of times and hit No. 1 on Apple’s App Store. With the help of its creepy virtual avatar — a Temu Labubu-esque creature with a disarming grin — the app requests and encourages users to grant access to bank accounts, passwords, contacts, chat logs, and anything and everything that might help it to do the user’s bidding.

“These systems are advertised as autonomous helpers, genies, or servants who do what you need before you ask, seamlessly,” Meredith Whittaker, the president of the Signal Foundation, makers of the popular encrypted messaging app, told The Verge. These autonomous actions are applied “not just to the person who installed Muse on their device, but also to everyone that person communicates with, as extrapolated from their most intimate information — emails, messages, contacts, photos, and more.”

With AI agents, tech companies are letting consumers automate their everyday tasks by extending their omnivorous style of mass surveillance to massive amounts of sensitive data. It’s the same Faustian tradeoff Silicon Valley has been offering us for decades: your private data in exchange for a little bit of convenience. And for many people, the idea of having a virtual assistant that does menial tasks like filling out invoices and sending emails is compelling — especially if they don’t realize how much data they’re actually giving away in return.

“For any agent to be useful and actually help you achieve your goals, it needs to have context about you and those you interact with,” Meta spokesperson Daniel Roberts told The Verge regarding other people’s data shared with Muse. “Muse gathers that based on public information and from what you’ve chosen to share, which is how it remembers the person who just sent you an invoice is in fact the plumber who you previously hired to complete some work in your bathroom or which flowers your spouse said they liked best.” Roberts pointed to Muse’s opt-in process and various options and settings that will “help people make the privacy choices that are right for them.”

“The way that the user interface is set up is designed to encourage you to share as much as possible, often more than you realize, until it’s too late.”

But not everyone has been able to navigate these options and settings. Already, users are describing incidents that are both alarming and entirely predictable. Some have reported the virtual assistant making all sorts of trouble, like grabbing millions of text messages they believed were off limits. Meta has disputed that the assistant has the ability to access text messages without permission, again attributing the incident to misconfigured app permissions and describing Muse’s ability to read text messages as “entirely opt-in.”