Aircraft Explorer is an interactive and accessible experience that allows users, particularly those with disabilities, to navigate through a menu of airplanes using facial gestures (left/right). Each facial gesture triggers the display of a 3D model of the airplane along with a corresponding description. This enables disabled individuals to explore information without the need for a keyboard.
The experience combines several technologies, including the AI-powered “Project Gameface” by Google and the Intuiface Accessibility Sandbox.
Project Gameface is an open-source hands-free gaming mouse that empowers users to control a computer’s cursor through head movements and facial gestures. Users can raise their eyebrows to click and drag, or open their mouth to move the cursor. These movements are captured by a webcam and translated into computer commands using a machine learning model.
The Intuiface Accessibility Sandbox is an experimental framework designed to enhance the accessibility of Intuiface experiences. It enables the use of voice input and/or keyboard events to navigate UI menus and content, providing visual and audio feedback.
HOW IT WORKS
Follow these step-by-step instructions to test this prototype on your computer:
- Download and install Project Gameface on your PC. Make sure you have a webcam connected.
- In the keyboard binding settings, assign a facial gesture to a specific keyboard key. For example, we assigned the “Left Arrow” keystroke to “Mouth Left” and the “Right Arrow” keystroke to “Mouth Right”.
- Download and run the Aircraft Explorer Intuiface experience (33MB)
- Activate the face control feature in “Project Gameface” and use your facial gestures to navigate through the experience.
This prototype was developed in collaboration with The Museum of Flight in Seattle, USA.