Domingo, UX for the blind

Leveraging virtual reality to assist the visually impaired with the digital world

How Domingo was born

The two most important days in your life are the day you are born and the day you find out why.
(Mark Twain)

Artistic depiction of how sight is clearly the main medium to access digital devices

Domingo is about opening our eyes recognizing the limits of current assistive technology for the visually impaired.

A sight-only approach

Digital devices evolved by taking real world entities and translating them into a visual metaphore. A real, concrete desk is now images on a screen, only accessible through sight.

A limited experience

Assistive software like screen readers offer great value to an impaired user. Still, the experience itself is far from being natural, fast and fluid. Ux still lacks a valuable hearing platform that can replace a graphical user interface.

Moving into a digital space

Blind users can't really "navigate" a system and only rely on a sequential stream of words. It is just a description of something clearly not designed for them.

Moving from 0 to 1

Just like it happened with the first pcs, we can come up with a new interaction method that does not only rely on visual information. We can use all of the possibilities that modern technologies can offer.

Digital environments

Digital devices are not only tools, but actual environments meant to be explored and lived. We want this to happen to everybody, without physical barriers.

Blidness is not a limit

Interfaces must be built following new approaches. Blindness is not an absolute limit, and only becomes so when not taken into account while designing a system.

Current solutions

Artistic depiction of how we rely on our senses to reach our goals

Analyzing the 5 human senses, we can recognize how we usually interact with our devices:

Sight based solutions

Immediately recognizing the position of elements on the screen through sight, a mouse pointer is used to immediately instruct the device to perform actions.
On smaller devices like smartphones, fingers and touch gestures are used instead of a mouse pointer to directly manipulate graphical elements.

Touch based solutions

Touch, altough essential in today mobile-first world, can be limiting. A virtual keyboard can be less effective than a physical one for a blind user.
There are some hardware devices to offer additional feedback, like Braille displays. They are less common and quite expensive.

Hearing based solutions

The screen reader is probably the main tool of interaction for the visually impaired. It provides the user a description of graphical elements on the screen, reading them sequentially.
Assistants (like Siri or Google one) are used to obtain informations and dictate commands.

User interaction with digital devices by the visually impaired is still limited

Without sigth there's no spacial information. The user does not have access to an immediate "digital space" to explore in which information is present and organized.
Users need a constant feedback to be reminded of their current status
Using a pc is considerably slower

The main interaction is now designed with a sight-centered approach. How could things change if we could better translate graphical elements with different mediums?

Domingo is a technology reinventing how blind users interact with digital devices, leveraging positional sounds and gestures

Domingo's answer

Domingo is a software that tracks user's head movements, and by adding sounds, enhances the common user experience of a digital device.

The entire development is based on a blind-first approach, to fully provide people in need an accessible environment, decoupling sight and interaction.

Domingo leverages virtual reality and, by translating to sounds some of the graphical elements, ensures a natural user experience.

With Domingo, a blind user can finally navigate and move into a digital environment.

Usage and development

The concept that fuels Domingo could be extended to multiple scenarios.
Sighted people could use it too, exploring new possibilities of interaction. It could be:

used as assistive software, as originally conceived

used within the VR world to explore new interaction possibilities

paired to many different devices like GPS or POS, where normal interaction could be difficult

State of the art

Domingo is now a prototype, only compatible with Microsoft Windows.

Even in early stage, Domingo is working and can be testded in it's main features, with good compatibility with existing assistive software.
At the moment Domingo is capable of abstracting windows on a screen, positioning them in the virtual space surrounding the user. This space can be navigated via dedicated trackers and binaural audio feedbacks. Every window can be "pointed" via head pose, so that the operating system can switch between active applications.

Current compatible trackers are:

Android smartphone with VR visor (like Google Cardboard)

Microsoft Kinect

Generally, every device capable of tracking it's pose and send it over network

What is the purpose of this prototype?

The goal of this prototype is to validate and test the idea behind the project, to define future guide lines to this technology.

Modular architecture

The program has been developed with a modular approach, so that it's easy to extend. Supporting new trackers and editing the settings is easy and mostly focused on trackers themselves. The arrangment of windows in the virtual space is not fixed and can be adjusted to user's need.

Growing up

We are currently searching for ways to grow this idea further. We need entities like incubators or accelerators that believe in our idea and could help us reach our goals. Contact us to know more!