The COVID-19 pandemic highlighted the need for contactless communication. Physical interactions became limited, so people increasingly turned to digital and touchless solutions for everyday activities. That occasion accelerated the adoption of gesture & voice recognition technologies.
The innovations have laid the foundation for new types of interfaces: gesture-based and voice-controlled. Some business owners have already explored and integrated them into their businesses, while others have yet to encounter them in practice.
New technologies often spark skepticism and mixed opinions. People have spent years using keyboards, mice, touchpads, and touchscreens. It’s hard for them to break away from familiar habits, and many wonder why they should even bother.
To dispel any doubts, our UI/UX design department has decided to explore new ways of interacting with devices, how they influence interface design, and the benefits they bring to businesses and our day-to-day lives.
Contents
- The Role of Voice & Gesture Based Interfaces in Changing User Experience
- How & Where It Works: Gesture Recognition Technology
- How & Where It Works: Voice Recognition Technology
- Current Limitations of Voice and Gesture Based Interfaces
- Design Considerations for Touchless Interfaces
- Conclusion
The Role of Voice & Gesture Based Interfaces in Changing User Experience
History makes it clear that all innovations in human-computer interaction have been aimed at reducing intermediary objects. The gradual reduction of the number of devices needed for control eventually led to their complete elimination, resulting in touchless technologies.
And this makes sense not only in terms of convenience and speed, but also from a hygiene perspective. When the world faced the pandemic, people started paying more attention to personal hygiene. We have become more mindful of the surfaces we touch and, sometimes even unconsciously, try to reduce their number. That’s why the emergence of touchless technologies is a logical development.
With the rise of touchscreens, gestures and voice commands have become an integral part of interacting with devices. According to Mordor Intelligence, both gesture recognition and voice assistant markets will see big growth through 2030 – gesture tech at about 17% per year, and voice assistants growing even faster at around 27% yearly.
This growth is primarily driven by the expanding use of the Internet of Things (IoT) and increasing digitization across retail, automotive, banking, and healthcare industries.
What is the reason for such popularity of touchless technologies? They make our daily life at home, at work and on the road more convenient and efficient. In general, the benefits of interfaces that don’t require physical control can be summarized into two main points:
- Allow doing multiple tasks at once. These new ways of controlling your tech, just by speaking or waving your hand, are making life so much easier. You can do things like turn on your lights or order groceries just by asking, or skip to the next song with a quick hand gesture. It’s super handy when you’re in the middle of cooking (like when your hands are covered in flour) or when you’re driving and need directions without taking your hands off the wheel.
- Comfortable for people with disabilities. Voice and gesture based user interfaces are really changing lives for a lot of people who might have trouble using regular devices. People who have arthritis can just tell their smart home system to adjust the thermostat or turn on lights instead of struggling with switches. People with visual impairments can do their job by talking to a computer instead of having to navigate a keyboard and screen. Touchless technologies are making everyday stuff more doable for the physically challenged, no matter if they’re at home doing household chores, working at the office, or even just trying to use an ATM at the bank.
In 2025, companies are encouraged to add voice and gesture control to their products because that’s what customers want. These innovations will undoubtedly affect UX/UI. In our article The Importance of User Experience (UX) you can read why it should not be underestimated.
Now let us show you how voice & gesture recognition technologies work and where you could apply them.
How & Where It Works: Gesture Recognition Technology
Gesture control is a technology that allows users to interact with virtual objects without physical contact. Camera-based systems and infrared sensors detect human gestures and movements of fingers, hands, the head, or the entire body, then convert them into commands using mathematical algorithms.
Gesture based user interfaces have been widely adopted in industries such as automotive, healthcare, consumer electronics, gaming, aerospace, defense, and other end-user sectors.
This technology initially gained popularity in the gaming industry, especially with the release of Nintendo’s Wii console in 2007 and Microsoft’s Kinect motion control three years later. These systems introduced a range of gesture-controlled games, including Star Wars, Sonic Free Riders, dance simulators, pet care simulators, sports game collections, and more.
Today, gesture-controlled games can also be found in the App Store. One such game, Don’t Touch, is controlled using gestures detected by the mobile device’s camera. In this game, the goal is to avoid obstacles to achieve the highest score. As you progress, the speed increases, making it more challenging to avoid them.
But gestures are used not only in games. For example, GestureTek, one of the first and leading companies in the field, developed video gesture control virtual reality solutions used in museums, science centers, amusement parks, trade shows, retail stores, real estate presentation centers, corporate showrooms, boardrooms, digital signage networks, airports, stadiums, and even in the healthcare sector.
One example of a gesture-controlled app available on iOS devices is Touchless Browser that lets users operate it remotely via hand gestures detected by the camera. It’s ideal for situations like eating, cooking, or using your device from the back seat of a car. The app allows actions like scrolling, clicking links, navigating, and web searching via audio input. It works best at a distance of 30 cm to 200 cm from the screen and on devices with A12 Bionic chips or higher.
At Rubyroid Labs, we specialize in designing intuitive, user-friendly apps with innovative features like gesture control. Let us help bring your app ideas to life with cutting-edge design and technology.
How & Where It Works: Voice Recognition Technology
Voice interfaces rely on speech recognition technology and advancements in natural language processing (NLP). A microphone catches sound waves and a speech model processes them. It takes out the noise and breaks the sound into little pieces (features), which are 25 millisecond long. These pieces are then compared with the words that the model knows, and the system finally understands what was said and carries out the task requested.
This function is used in voice assistants, voice-controlled devices, and transcription services. It’s becoming more popular in the IoT, automotive and finance industries due to its ability to provide hands-free control while driving or doing household chores.
What makes a voice interface unique is that you can’t see it. It’s essentially one of three types of user interaction scenarios:
- fact-based Q&A
- a structured dialogue designed by developers
- free-form conversation
Voice assistants smoothly switch between these modes during conversations. Typically, a voice-based experience has a visual counterpart, and the deeper the menu layers in a graphical interface, the more useful a voice assistant becomes.
People see voice interfaces as a way to simplify tasks, as it’s often easier to say a command, than to tap through multiple screens. That’s why they’re widely used in situations where finding the right function visually can be challenging.
For example, when managing a smart home, it is easier to give verbal commands to Alexa, which connects to all household devices, rather than taking out your phone, opening the app, and configuring it manually. It’s possible to handle utility bills, mobile top-ups, or sending money using the same device.
The voice payments feature is supported by the popular Amazon Echo and Google Home smart speakers, as well as Siri and Google Assistant virtual assistants. Mostly, banks use Siri to perform voice remittance, account enquiries, and payments. Among the ones that had already implemented voice recognition, Bank of America was in 2018.
Voice assistant management apps are gaining popularity as they make it easier to control virtual assistants like Siri, Alexa, Google Assistant, Bixby, and others. Apps like Voice Command Hub, Voice Assistants Commands, and Voice Commands for Siri categorize commands into sections such as device settings, music, navigation, and smart home control, allowing users to find what they need quickly. You can even save your favorite commands for easy access.
In cars, voice interfaces help adjust climate control, set music volume, decline calls, ask for directions without taking hands off the wheel. GlobalData’s report Innovation in automotive: in-car voice assistants states that there have been over 720,000 patents filed and granted in the automotive industry in the last 3 years.
Hyundai Motor is one of the leading companies in the field of in-car voice assistants, filing numerous patents in this area. The company has introduced its own in-car voice assistant, known as the Hyundai Intelligent Personal Assistant. Kia and Ford Motor are also among the key players in this field, filing patents related to in-car technology.
Current Limitations of Voice and Gesture Based Interfaces
Like any other technology, gesture and voice interfaces do have some limitations. In order to use them effectively, it is necessary to address certain challenges.
False Activations
How can a recognition system distinguish intentional movements from accidental ones? This is a complex challenge that needs a solution. What if a driver makes an unintended hand gesture while operating a moving vehicle, and the system misinterprets it? This could pose a serious safety risk. A vehicle that incorrectly recognizes commands might be even more distracting than a traditional dashboard with physical buttons.
Cultural Differences
The issue of how gestures are interpreted differently across cultures, ages, professions, and other factors is just as significant as the technical challenges. There are hardly even ten universally recognized gestures. Moreover, if you ask people, even within the same family, what gesture they associate with a specific action (like lowering the music volume), they’ll likely demonstrate completely different movements. This means users would have to memorize predefined gestures and their assigned meanings. And when the system has a wide range of functions, that becomes anything but simple.
Naturalness
To make a voice interface feel natural for a person, we need to understand why our conversations with other people are considered natural. We experience ourselves differently when talking to different people. With some, it’s easy and comfortable, while with others, we can’t quite find common ground.
What influences our impression? Certainly, it depends on the traits of the person we’re talking to, the topic, and the situation. But likely, those aren’t the only factors. Communication is an incredibly complex process. People still face challenges when communicating with each other, and to establish communication with technology, we need to teach it something we haven’t fully mastered ourselves.
Appropriateness
Speaking actions out loud won’t be appropriate in every situation. For example, no one would likely want to dictate their bank card number or account password when surrounded by people. It would be a strange scenario if on public transport everyone was giving commands to their voice assistants.
Data Security
If voice identification is implemented, hackers could use our voice for various types of breaches. Voice interfaces are necessary when using a device the usual way isn’t possible.
Unclear Functionality
The functionality of a voice interface is unclear until we test it ourselves. Since there’s no visual guidance, the user can’t immediately understand what tasks the voice interface can handle. Users need prompts, or they might not even realize that it can do something that way.
Today’s voice and gesture interfaces still have limitations, but technology is constantly evolving to overcome these challenges. No innovation is perfect from the start, but that shouldn’t stop us from trying.
At Rubyroid Labs, we’re passionate about pushing the boundaries of seamless interaction. That’s why we’ve compiled key insights and recommendations to help you integrate voice and gesture based user interfaces effectively into your product.
Design Considerations for Touchless Interfaces
We’ve been in design for over 12 years, so our Software Design department shared some key insights on what to keep in mind before creating a voice interface.
Firstly, it’s essential to have a clear understanding of the following:
- Why would the user prefer voice control over a traditional method?
- What’s the business benefit?
If you have solid answers to these questions, the next step is to plan and create a user interaction scenario. It’s best to start by creating a plan in FigJam – mapping out all the possible dialogue options. Then, the scenario should be tested with real users.
A good scenario ensures that:
- The user can speak naturally, and the interface understands them.
- The voice interface provides relevant, fact-based, clear and informative responses, with minimal need for follow-up questions.
If your scenario requires a large number of back-and-forth clarifications, it’s better to stick with a graphical interface. A voice interface is particularly useful when its graphical counterpart would require too many deliberate clicks.
When designing a gesture-based user interface, you need to consider the following factors:
Spacing between elements. When using the interface through gestures, such as hand movements, the level of accuracy tends to decrease. The buttons and interactive components should be positioned further apart than in conventional interfaces, to prevent users from inadvertently clicking or selecting the wrong option.
Scale. This goes hand in hand with spacing. Observations show that gesture-based interfaces are more comfortable to use when text and other elements are larger. Because of this, gesture control isn’t ideal for small screens.
Choosing supported gestures. Movements should logically correspond to the action they trigger. It’s best to use gestures that users are already familiar with. Even better, start with a short tutorial to introduce them to functionality.
Feedback. Users need confirmation that their action was successful. This can come in the form of a sound, a color change, text, or animation effects.
Micro-animations. When hovering over a button, it should change its state, just like in graphical interfaces.
No matter what type of interface you choose for your app, always start by considering your audience’s needs and your business goals. To get the best results, it’s a good idea to talk to experts.
Conclusion
The evolution of human-device interaction has always aimed to make technology more intuitive and accessible. Voice and gesture-based interfaces are the next step in this progression.
However, despite these advancements, traditional graphical interfaces are far from obsolete. Right now, we mostly see a mix of voice, gesture and graphical interfaces. This harmonious blend of approaches offers greater flexibility and caters to diverse user preferences. By choosing this option, businesses not only elevate user convenience but also secure a competitive advantage.
If you’re looking to integrate voice- or gesture-based interfaces into your app, our team at Rubyroid Labs can help. With our expertise in UX/UI design and development, we’ll create an intuitive and visually appealing interface that sets your product apart.
