With the new Project Starline prototype, Google may have figured out how to revolutionize video calling through a clever combination of AI and 3D modeling. We were able to try it and it’s amazing.
Project Starline // Source: Google
The man in front of me hands me an apple. She seems to be within range, about 30 centimeters away. I stretch out my hand… but I can’t grab the fruit. I know exactly why, and yet my brain has trouble understanding everything. I’m standing in front of Google’s latest Project Starline prototype and it’s just crazy.
In fact, on the sidelines of Google I/O 2023, the American giant presented a new version of Project Starline. A still experimental technology that promises Revolutionize our video calls.
A most surprising video call
When I was invited to test this experience, I’m in a small room in one of the many Google buildings scattered around Mountain View. I notice a chair, a table and especially something like that large 65 inch TV (and a very large soundbar) in front of which I am asked to take my seat.
The Google spokesperson closes the door behind me and settles into another room downstairs using the same device. Using intermediary screens, he starts a video call with me. Nothing very extraordinary… a priori.
Project Starline // Source: Google
my interlocutor, Jason Lawrence, co-founder of the projectHe really seems to be sitting on the other side of the table. Something deep inside me can’t process the fact that Jason isn’t in the same room as me.
I lean left, right, and see the sides of his face clearly, as I might if he were actually a meter away from me. He gives me the famous apple and I feel like if he threw it it would hit me in the face. He offers me a check, clenched fist to clenched fist, and here I’m almost surprised not to touch anything.
“A Magical Window”
The 3D effect is breathtaking, nothing more nothing less. Especially since the edges of the TV seem to have disappeared and it’s the various cameras that frame it that allow me to guess its contours. The immersion is very good, and some artifacts in the image – Jason’s silhouette sometimes waves – remind me I’m just seeing a 3D projection of him.
Unfortunately, taking photos and video recordings is prohibited. A person at Google informs us unofficially that our cameras and smartphones cannot correctly transmit the 3D experience of Project Starline. So you have to be content with the official pictures and rely on me.
YouTube link Subscribe to Frandroid
The question inevitably arises: How does Project Starline work? Google describes a technology that behaves “like a magic window”. You might think magic is a simple hologram. In reality, however, it is about artificial intelligence, photorealistic 3D modeling and highly efficient real-time processing.
The goal: to improve the quality of exchanges by making them more natural, even when two people are in two different cities.
3D modeling in the Starline project // Source: Google
The first release of Project Starline actually dates back to 2021. Back then, however, the system required much larger hardware, the size of a “restaurant shack”. What is meant are the American-style dining boxes, consisting of two large benches and a table.
The brand new prototype I was able to try is much more compact and promises to be much easier to install, for example in a meeting room or even at home in the more distant future.
A natural conversation
The cameras around the screen capture my movements and the direction of my gaze. An artificial intelligence developed by Google retrieves this data in real time. She can thus reconstruct a photorealistic 3D image of me to show Jason while giving me a perception of my interlocutor that is the same as I would have had if he were actually physically in front of me. All depth effects are transferred perfectly.
Project Starline // Source: Google
That’s what excites me the most. Technically I haven’t seen a real video of Jason. I saw a live 3D reconstruction. In addition, this notion of real time is essential. If there was latency, the immersion would be completely disrupted.
Google therefore prides itself on being able to compress and interpret the data on the fly, leaving the discussion very natural. The artifacts mentioned earlier in this article are undoubtedly due to small bumps in the process.
An experimental product
Finally, note that if I lean over too far, a notification on the TV prompts me to refocus.
No price or release date has been announced yet. For now, Project Starline is still a slightly wacky experimental product that’s set to make a big splash for years to come. Meanwhile, I still haven’t managed to catch the apple.
NB. Our journalist Omar attends the Google I/O in Mountain View as part of a press trip organized by Google.
Would you like to join a community of enthusiasts? Our Discord welcomes you, it’s a place of mutual help and passion for technology.