Google wants video AI to understand what people need as they work

A research team led by Google has introduced StreamVibe, a video agent designed to decide for itself when to speak up. The tests show above all how hard that remains for today's AI models. Researchers from Google, Stanford, Oxford, and several U.S. universities are working on an assistant that watches you cook or assemble furniture and speaks up only when needed.

With StreamVibe, they've built a video agent designed to understand even vague requests such as "Please help me if I run into any difficulties." According to the paper, existing video assistants are built for clear-cut commands like "Let me know when the water starts boiling," so they rarely pick up on unspoken intentions. They also have a hard time tracking what's already been done, which causes them to repeat advice or respond too late, and when users change their requests midway through, the assistants often stick to the original instructions.

To measure the problem, the team compiled a benchmark of about 4,100 examples from more than 183 hours of everyday life and software recordings. Gemini 2.5 Flash suggested when…