Google Photos introduces an AI search feature, 'Ask Photos' | TechCrunch

Google Photos introduces an AI search feature, 'Ask Photos' | TechCrunch

Google Photos is getting an AI infusion with the launch of an experimental feature, Ask Photos, powered by Google's Gemini AI model. The new addition, which will launch later this summer, will allow users to search their Google Photos collection using natural language queries based on AI's understanding of their photo content and other metadata. Take advantage of.

Before users can search for specific people, places or things in their photos, thanks to natural language processing, the AI ​​upgrade will make finding the right content more intuitive and reduce the manual search process, Google said. announced on Tuesday. The annual Google I/O 2024 developer conference.

For example, instead of searching for a specific object in your photos, such as “Eiffel Tower,” you can now ask the AI ​​to do more complex tasks, such as “What national park have I visited?” Find the best picture among them.” AI uses a variety of signals to determine what makes a given set of images “best,” including brightness, blur, lack of background distortion, and more. This can then be combined with understanding the geographic location of a set of images or dates to retrieve only images taken in US National Parks.

Image credit: Google

The feature builds on the recent launch of Photos Stacks in Google Photos, which groups nearly duplicate photos together and uses AI to highlight the best photos in the group. As with PhotoStacks, the goal is to help people find the photos they want as their digital collections grow. According to Google, over 6 billion photos are uploaded to Google Photos every day so you can get an idea of ​​the scale.

In addition, the “Ask a Photo” feature will allow users to ask questions to get other types of helpful responses. In addition to asking for the best photos from a vacation or other group, users can ask questions that require an almost human-like understanding of what's in their photos.

For example, a parent can ask Google Photos what themes they used for their child's last four birthday parties, and it will return images and videos about mermaid, princess, and unicorn themes. With can give a simple answer which ones were used before and when.

Image credit: Google

This type of query is made possible because Google Photos not only understands the keywords you enter, but also natural language concepts, such as “themed birthday party”. It can also leverage the multimodal capabilities of AI to understand whether an image contains text that may be relevant to a query.

In another example presented to the press by CEO Sundar Pichai ahead of today's Google I/O developer conference, a user asked the AI ​​to show their child's swimming progress. AI packed highlights of photos and videos of a swimming baby over time.

Another new feature uses search to find answers from text in images. That way, you can take a picture of something you want to remember later — like your license plate or passport number — and then ask the AI ​​to retrieve the information when you need it.

If the AI ​​ever gets something wrong and you correct it — perhaps flagging a photo that isn't from a birthday party or a photo you don't highlight from your vacation — it remembers that reaction. It will improve with time. This also means that the AI ​​becomes more personal to you the longer you interact with it.

When you find photos you're ready to share, AI can help generate a caption that summarizes the content of the photos. Currently, this is a basic summary, however, does not offer the option to choose from different styles. (But considering it's using Gemini under the hood, a cleverly written hint might work to revert a certain style if you try it.)

Google says it will have guardrails for not responding in some cases (presumably no one asks the AI ​​for the “best nudes”?) while training the model to potentially exclude offensive content. was But the feature is starting as an experiment, so it may need additional controls over time as Google responds to how people use it.

The Ask Photos feature will initially be supported in English in the US, before rolling out to more markets. It will be a text-only feature for now, like asking AI chatbot questions. Over time, though, it may integrate more deeply with Gemini running on Android-like devices.

The company says that Google Photos does not use users' personal data for advertising. Google says humans will also not review AI interactions and personal data in Ask Photos, except “in rare cases to remedy misuse or harm.” Google Photos also does not use people's personal data to train other generative AI products, such as Gemini.

Read more about Google I/O 2024 on TechCrunch
About the Author

Leave a Reply