AI Now Takes the Wheel on Your Phone

Gemini's new task automation feature is currently in beta, enabling AI to operate applications on select smartphones. This innovative technology allows Gemini to handle tasks like ordering food or booking rideshares, though it is limited to a small number of services and is still a work in progress.

While Gemini is slower than human users for immediate tasks, its ability to function in the background and even while the phone is unattended is a significant advantage. Users can observe Gemini's progress through on-screen text, witnessing its real-time decision-making as it navigates app interfaces.

Despite occasional errors and a sometimes slow execution, Gemini's task automation requires explicit user confirmation before finalizing any purchase or booking. This safety measure ensures users retain control, and the AI has demonstrated surprising accuracy in completing tasks with minimal necessary adjustments.

Gemini's integration with calendar and email allows it to proactively schedule tasks, such as booking an Uber for a flight based on calendar entries. This demonstrates a sophisticated level of contextual understanding, surpassing traditional digital assistants.

However, the current app interfaces are not designed for AI interaction, appearing brittle and impractical for automated tasks. This suggests that future advancements may rely on developers adopting more AI-friendly protocols like Model Context Protocol (MCP) or app functions, with Gemini's current iteration serving as a potential catalyst for such changes.