Haptic holography or mid-air haptics promises to bring virtual reality to life, but a new study reveals a surprising physical obstacle that will need to be overcome. A research team at UC Santa Barbara has discovered a new phenomenon that underlies emerging holographic haptic displays and could lead to the creation of more compelling virtual reality experiences. The team’s findings were also honored by a publication in the “Science Advances” journal.
Holographic haptic displays use phased arrays of ultrasound emitters to focus ultrasound in the air, allowing users to touch, feel and manipulate three-dimensional virtual objects in mid-air using their bare hands, without the need for a physical device or interface. While these displays hold great promise for use in various application areas, including augmented reality, virtual reality and telepresence, the tactile sensations they currently provide are diffuse and faint, feeling like a “breeze” or “puff of air”.
“Our new research explains why such holograms feel much more diffuse or indistinct than would be expected,” says Yon Visell an associate professor at UCSB's College of Engineering, whose research focus is on interactive technologies with an emphasis on haptics, robotics and electronics.
The study led by Yon Visell and Gregory Reardon, a doctoral student researcher, used laser vibration measurement, simulations and perception experiments to comprehensively study ultrasound-excited waves that arise in the skin during haptic holography. They discovered that holographic displays excite widespread vibration patterns – so called shear shock waves – in the skin.
In haptic holography, Visell explains, shock waves are created when ultrasound waves are focused and scanned in mid-air, causing vibrations in the skin. These vibrations can interfere with each other in a way that amplifies their strength at some locations, a phenomenon known as constructive interference. The formation of shock waves creates a trailing wake pattern that extends beyond the intended focal point, reducing the spatial precision and clarity of the tactile sensations. As an analogy, according to the researchers, if the focused sound beam is a fast-moving boat on the water, the shock wave pattern is a wake trailing the boat. Current holographic haptic displays excite shock wave patterns that are so spread out in the skin that the sensations feel very diffuse.
The researchers used a PSV Polytec Scanning Vibrometer to perform vibration imaging of human hand surfaces as users felt holographic haptic feedback. The measurements revealed the prominent manifestation of ultrasound-evoked shear shock wave patterns in the skin.
“Our study reveals how holographic haptic displays – a promising new technology for virtual reality and telepresence – require new knowledge in acoustics innovations in design,” Visell says. “By understanding the underlying physics of ultrasound-generated shear shock waves in the skin, we hope to improve the design of haptic holographic displays and make them more realistic and immersive for users.
Such haptic displays could enable us to augment our physical surroundings with a limitless variety of virtual objects, interactive animated characters, or graspable tools that can be not only seen, but also touched and felt with the hands."
After predicting the shear shock wave in numerical simulations, Visell and his team verified the results by visualizing the real wave patterns elicited by focused ultrasound on skin with a scanning laser vibrometer. Subsequent perception experiments correlated the wave patterns with user perception to assess the influence on user experience.
Fig. 1 shows the experimental setup. The scanning vibrometer captured surface oscillations elicited as a tissue phantom with mechanical properties emulating those of human skin was stimulated via focused ultrasound delivered from a phased array of ultrasound transducers shown in green (Fig. 1A). Finally in vivo vibrometry measurements were captured from human hands at more than 300 measurement locations (Fig. 1B).
Consistent with wave mechanics and consistent with findings from the numerical simulations, large-amplitude shear shock wave patterns with speed-dependent wake angles were elicited when the ultrasound source scanned across the surface at transonic to supersonic speeds (Fig. 2 right). These shear shock patterns lagged behind the focus location and formed rapidly, over the course of milliseconds. The surface area of their wake region was orders of magnitude larger than the effective acoustic focal area. Thus, the effective focusing resolution was dominated by shock wave formation rather than by ultrasound focusing. Wave energy trailed the focus location within a wake 10 cm or more in length and also extended several centimeters in directions perpendicular to the scanning path. Lower scanning speeds (v = 2 m/s, Fig. 2 left) yielded wave patterns that were concentrated near the focal location due, in part, to viscous damping. However, these waves were smaller in amplitude and lower in frequency and thus would be felt only faintly by the skin.
The insets in Fig. 2 represent the frequency content in the excited wave patterns. As the Mach number increased, a broader range of higher frequencies was excited.
The measurement results agree very well with the numerical simulation as the following movies illustrate for scan speeds of v = 4 m/s and 7 m/s:
Next, Visell and his team turned to in vivo measurements of human hands (Fig. 3). In those experiments ultrasound scanning paths extended along an axis of the volar hand surface, from the wrist to the tip of the index finger. The linear paths were laterally modulated transverse to the motion direction (yielding zigzag path shapes) because such modulation has been found to elicit stronger sensations. The lowest and highest scanning speeds (vl = 1 and 11 m/s) were, respectively, within the subsonic and supersonic regimes.
Consistent with theory and findings from the numerical simulations and phantom tissue experiments, low scanning speeds (vl = 1 m/s) excited shear wave patterns that extended outward from the instantaneous focal location. At higher scanning speeds (vl > 4 m/s), the patterns formed wakes that trailed the motion by 10 cm or more. The length of these wave patterns exceeded the approximate focal width (0.6 cm) by more than an order of magnitude.
These phenomena are reflected in human haptic perception, shown by comparing the results of the in vivo vibrometry test data (Fig. 4A, blue) with behavioral data that were obtained via an experiment on tactile motion perception (Fig. 4A, red). During each trial, participants felt a focused ultrasound stimulus that scanned from the wrist to the tip of the index finger or vice versa; they reported the direction of scanning motion. Perceptual accuracy was greater for lower scanning speeds (P < 0.0001) and declined to chance levels at the three highest scanning speeds (vl = 4, 7, and 11 m/s), which produced the most elongated wakes. An integrative analysis comparing data from the perception and vibrometry experiments revealed that perceptual accuracy declined monotonically with increasing wake length (Fig. 4B).
The team’s discovery of the previously unknown shock wave phenomena that underlie haptic holography provides an important step forward in creating haptic holographic displays that may enable users to more realistically and immersively interact in the future metaverse.
Images courtesy of the authors unless otherwise specified. Cover image: jamesteohart/shutterstock.com
