Google has started an open source project called Gameface. The goal: to be able to operate a computer cursor without hands, but with facial movements. The main interest is to help people with disabilities among their members so that they can use a computer with as few restrictions as possible.
The Gameface Project // Source: Google
It’s okay to talk about artificial intelligence or the latest Android feature, but if it’s just that, it’s not necessarily moving society forward, especially when it comes to accessibility. Google unveiled the Gameface project at its I/O 2023 conference, billed as “a new open-source hands-free gaming mouse.”
A mouse that isn’t actually a mouse
In fact, Gameface is not a gaming mouse, but software. The latter actually follows the movements of the user’s head and only needs a webcam to do so. The advantage is that it doesn’t require expensive hardware like a disabled mouse or a set of motion tracking sensors.
YouTube link Subscribe to Frandroid
The movements and gestures performed are recognized by machine learning models connected via a tool called MediaPipe, allowing the cursor to move or clicks to be performed. The software places 486 points on the user’s face and creates a mesh. For Lance Carr, a World of Warcraft player with muscular dystrophy who helped develop Gameface, “It’s so precise I can even italicize it.”
The Gameface Project // Source: Google
To adapt to all users, software parameters allow to customize the triggering of a cursor action according to the intensity of their gesture. As the name suggests, Gameface is specially designed for gamers. Video game practice may require some responsiveness and precision, which is a real need these days.
The Gameface Project // Source: Google
The Gameface tool is still under development, but the source code is available on Github.
Would you like to join a community of enthusiasts? Our Discord welcomes you, it’s a place of mutual help and passion for technology.