Google is introducing a new AI-driven feature in its Lookout app – Assisted Vision. Visually impaired people can thus discuss with the AI in order to ask it all the details of an image by asking several questions.
The Google Lookout app’s Image Q&A feature allows you to chat with an AI and ask them to describe an image
Describing an image is no longer a very complex challenge for a mature artificial intelligence. However, Google has managed to take the concept one step further to make it more practical and impressive. The American giant has just revealed the “Image Q&A” feature in its accessibility application Lookout – Assisted Vision and we were able to demonstrate it briefly a few days earlier during Google I/O 2023.
The concept is easy to summarize: you ask any question to understand what’s in the picture and the artificial intelligence will give you a clear answer every time. The purpose of this feature — and of Lookout in general — is to help visually impaired people identify what’s in a photo or drawing. Hence the term “Q&A”, which means questions and answers. The demo, it had something really breathtaking.
Basically, you upload an image to the app and ask the AI to describe it. The latter will run and explain to you what we see. This first description is effective and might suffice if you just want a quick summary. However, if you want to go into more detail, simply ask a question in writing or out loud.
AI answers your questions about the description of the image
During the demo at I/O, Lookout’s Image Q&A option was faced with an image and here’s the conversation that was then had with the human user.
AI: In this picture I see a dog running on the beach. I can also see that he has a bullet in his mouth. In the background I see water, mountains and the sky.
Human : What breed does the dog belong to?
AI: The dog is a crossbreed.
Human : Does the dog look playful?
AI: Yes, the dog is playful.
Human : Describe the ocean.
AI: The sea is calm and a mountain can be seen in the background.
Human : Are the waves big?
AI: No, the waves are small.
Human : Are there clouds in the sky?
AI: Yes, there is a cloud in the sky.
Human : Is the beach crowded?
AI: No, the beach is empty.
The conversation isn’t necessarily the most natural, but it’s fluid and clear enough to present an interesting solution for visually impaired people, who often have to settle for very short descriptions on the Internet.
There they have a tool with great potential at their disposal that gives them a precise idea of an image.
We filmed the said demonstration. Let’s just clarify that this recording is not of high quality, it was mainly intended as a support for writing this article.
YouTube link Subscribe to Frandroid
Finally, remember that Lookout is an application that has been available on the Play Store for a while. This accessible tool already had several interesting functions for the visually impaired: reading business and loyalty cards or labels, recognizing the denomination of a banknote and helping to scan a document.
To follow us, we invite you to download our Android and iOS application. You can read our articles and files and watch our latest YouTube videos.