What if the devices we touch every day could speak back to us: not through a screen or a speaker, but through sensation itself?
That was the central question driving our June 10 webinar, “Beyond the click: perception-based haptics for the next generation of HMI,” where three leading experts in haptics and human-machine interaction gathered to explore how touch feedback is evolving from a novelty into a genuine communication channel.
Why haptics matter now
Every human-machine interaction follows the same arc: a user forms an intention, acts through touch, voice, or gesture, the device processes that input, and then responds. Touch is uniquely positioned in this loop – it is the only sense that is simultaneously input and output. A finger can press a button and feel the resistance at the same time. No other sense works that way.
Yet for decades, commercial devices have treated haptics as a single bit of information — a click, a buzz, a confirmation. Haptics is now at an inflection point. The market is moving toward richer feedback with multi-actuator systems, wearables, and platform-level APIs. It is no longer just a research topic; it is becoming a product and platform discipline.
"The question is not how to add a vibration. The question is: what should the user feel, and why?"
Touch is a high-bandwidth communication channel
Hong Tan‘s presentation delivered the most striking evidence that our current use of haptics is dramatically underutilizing what the human body can do.
Consider Tadoma, a communication method used by deaf-blind individuals. By placing a hand on a speaker’s face — thumb across the lips, fingers along the cheek — a skilled practitioner can perceive natural speech in real time, including accents and nuance. Where a click carries one bit of information, Tadoma operates at roughly 12 bits per second. The gap is enormous, and it points to something fundamental: the skin is a rich sensory surface, not a simple on/off switch.
Hong’s research has taken that principle into a practical system called TAPS – the TActile Phonemic Sleeve. This forearm-worn device maps the 39 phonemes of English – 25 consonants and 14 vowels – to distinct vibrotactile patterns delivered through an array of vibrators on the arm. Users feel a pattern, decode it as a phoneme, and sound out words.
The learning results are striking. Across more than 100 participants with no prior exposure to the system, six learned to reliably recognize 500-word vocabularies at a rate of one word per minute or faster in under 10 hours of total training. The remaining participants learned between 100 and 300 words. For a completely novel sensory channel, that is a remarkable outcome.
TAPS was not developed exclusively for people with sensory impairments. It was designed for anyone in a situation where eyes and ears are unavailable or overloaded from noisy industrial environments to operating rooms. The broader principle: touch can carry far more information than any current product uses it for, and perception-based design is the key to unlocking that capacity.
Bringing haptics to the windows platform
Nesh Gandhe shifted the conversation from research to implementation, showing how Microsoft is bringing structured haptic feedback to the Windows operating system at scale.
The research makes the case. The question is how to turn it into something millions of people experience every day.
Most computing interactions today complete visually. You drag a file, snap a window, align an image in PowerPoint and then scan the screen to confirm it worked. The ambition behind Windows haptics is to change that: to give users subtle, contextual physical confirmation that their action landed, without requiring them to look.
User research at Microsoft surfaced three consistent effects when this kind of feedback is done well:
- Clarity
users feel more confident that their actions registered. - Delight
the physical connection to the device is experienced as genuinely pleasurable, even if hard to articulate. - Invisibility through habituation: within a day or two, users stop consciously noticing the haptics but on a device without it, something feels broken or missing.
Designing complete haptic systems
The design philosophy is built around direct cause-and-effect: haptic feedback fires only in response to a deliberate user action, never for passive notifications. Priority scenarios include precision alignment (snapping elements, hitting screen boundaries), action zones (drag-and-drop, detents on sliders), and creative tools like PowerPoint and video editing software, where physical confirmation of alignment reduces the need to squint at guidelines.
There is also a harder engineering challenge underneath all of this: how do you ensure that the same intended haptic experience feels consistent across dozens of different hardware manufacturers? The answer being developed in partnership with the Haptics Industry Forum is a standardized measurement rig that mimics the mechanical load of a human finger pressing a touchpad, then defines perceptually relevant metrics that any supplier can verify against. Design guidelines for developers are expected to be published in July 2026.
The system view: from click to communication
Stepping back from any single device or platform, a well-designed haptic system is never just a waveform. It is the outcome of a carefully tuned stack: perceptual intent at the top, software abstraction layers, firmware and embedded control systems, actuators and sensors at the hardware level, and measurement systems throughout to ensure consistency across units and across users.
The landscape spans a wide spectrum from simple single-actuator devices delivering a click, to spatial, multi-actuator wearables capable of carrying full tactile languages. Innovobot’s haptic glove-and-cuff system for surgical robotics is an example that sits toward the complex end: a custom tactile vocabulary that communicates tissue resistance, pressure thresholds, and proximity information directly to a surgeon’s hands. Delivered through touch rather than additional visual or audio alerts, this type of feedback has been shown to improve procedural accuracy and reduce cognitive load in the operating room.
Looking further ahead, teleoperation and physical AI represent the next frontier. As robots begin interacting with the physical world and humans take supervisory or collaborative roles, the need to feel what the robot feels becomes critical. The lab-scale research of the past few decades is moving steadily into commercial and enterprise applications and haptic design will need to move with it.
Three takeaways
The session closed with three points Manuel distilled from the discussion:
- Haptic feedback is underused. Most devices treat touch as a single-bit signal. The science and the products show it can do far more.
- Perception-based design is the path forward. Understanding what humans can feel, and designing signals around those capabilities, is what separates meaningful haptics from noise.
- Platform-level implementation makes it scalable. Whether at the OS level or across embedded and wearable device categories, bringing haptic intent into shared, standardized frameworks is what allows the technology to reach people at scale.
Want to learn more about haptic design, HMI strategy, or perception-based interface development? Contact us at connect@innovobot.com
Speakers:
Hong Z. Tan, Lead Haptics Scientist, Google (Android & Pixel), brought deep academic and industry expertise rooted in haptics psychophysics. A Fellow of the IEEE and former faculty member at Purdue University, Hong spent four years at Microsoft Research Asia before joining Google, where she leads haptics development for Pixel devices and Android APIs. Among other contributions, she is known for her work on the TActile Phonemic Sleeve (TAPS), a device capable of encoding up to 500 English words through vibrotactile patterns on the forearm.
Manuel Cruz, Operating Partner, Chief Scientist & Head of HMI Lab, Innovobot Labs, opened and moderated the session. A pioneer in haptics with more than 300 granted and pending US patents, Manuel has helped shape haptic feedback technology across mobile, medical, automotive, gaming, XR, and mobility applications, and served as president of the Haptics Industry Forum.
Gnyanesh (Nesh) Gandhe, Senior Human Factors Engineer, Microsoft focuses on multi-modal interaction design across the Windows platform, Surface, Xbox, and Mixed Reality. His mission: to move beyond visually dominant interfaces and bring haptics into everyday computing as a reliable and meaningful layer of feedback.