Google I/O 2024 witnessed DeepMind’s demonstration of real-time AI interaction using computer vision with Project Astra.
Project Astra by Google demonstrated an AI chatbot utilizing a smartphone camera to provide real-time descriptions of the surroundings.
During Google I/O 2024’s keynote session, the company presented its extensive range of artificial intelligence (AI) models and tools that have been in development for some time. Many of the features introduced will soon be available for public previews in the coming months. However, the most intriguing technology showcased at the event won’t be available immediately. Developed by Google DeepMind, this innovative AI assistant, dubbed Project Astra, was unveiled, demonstrating real-time, computer vision-based AI interaction.
Project Astra represents a significant advancement in AI capabilities beyond existing chatbots. Google employs its largest and most powerful AI models to train its production-ready models. Demis Hassabis, co-founder and CEO of Google DeepMind, highlighted Project Astra as an example of ongoing AI model development. He introduced it by stating, “Today, we’re excited to share some new advancements in AI assistants, which we’re calling Project Astra. We’ve long aimed to create a universal AI agent that can truly assist in everyday tasks.”
Hassabis outlined a series of criteria that the company had established for these AI agents. They must be capable of comprehending and reacting to the intricate and ever-changing real-world surroundings, as well as retaining visual information to build context and initiate actions. Additionally, they should be adaptable and personalized, enabling them to acquire new abilities and engage in conversations seamlessly.
With this depiction, the CEO of DeepMind presented a demonstration video illustrating a user holding a smartphone with its camera app active. Engaging in conversation with an AI, the user received instant responses to various vision-based inquiries. The AI effectively utilized visual information for context and possessed generative capabilities to address related questions seamlessly. For example, when the user displayed crayons and requested an alliterative description, the chatbot promptly replied, “Creative crayons colour cheerfully. They certainly craft colourful creations.”
But the demonstration didn’t end there. Later in the video, the user pointed towards a window revealing buildings and roads. Upon questioning about the neighborhood, the AI swiftly provided the accurate response, showcasing the AI model’s proficiency in computer vision processing and the extensive visual dataset required for its training. However, the most intriguing display occurred when the AI was asked about the user’s glasses. Despite briefly appearing on the screen and quickly disappearing, the AI accurately remembered their position and guided the user to locate them.
Project Astra remains unavailable for public or private preview at this time. Google continues its development efforts, exploring potential use cases for the AI feature and determining the best approach to make it accessible to users. While this demonstration could have been considered a remarkable AI achievement, some of its spotlight was dimmed by OpenAI’s recent Spring Update event. At the event, OpenAI introduced GPT-4o, which exhibits comparable capabilities and emotive voices, enhancing the AI’s human-like qualities.
