The entertainment industry is abuzz thanks to the launch of next-gen immersive technologies such as the Apple Vision Pro and recently announced Meta Quest Pro 2. Talk is now naturally gravitating towards how extended reality (XR) can be enhanced with the help of haptic technologies that allow us to experience physical stimuli alongside virtual experiences.
Haptic technologies open new possibilities for realism in virtual and immersive environments and can amplify our connected experiences beyond the current limitations of screens and devices. Greater connectivity and emerging immersive technologies can both support and enhance these experiences, whether it’s through in-built haptic feedback within VR headsets and their controllers, or via accompanying devices that allow for new tactile sensations to be felt by the user, complementing their virtual experience.
A new level of immersive entertainment
Haptic technology’s use within XR is currently primarily centred around vibrotactile feedback. This is predominantly through controllers which vibrate as the user interacts with the virtual environment. Gaming is perhaps the most obvious and well-known application of haptics, from the rumbling vibrations of a controller as your character runs, to cutting-edge bodywear that reacts to virtual stimulation, such as being fired at in first-person shooters. VR was quick to embrace this aspect of haptic hardware to enhance immersion.
Beyond gaming, experiments are underway to determine how haptics can be used to enhance immersion in both live and virtual environments, such as sporting and esports events. The upcoming 2024 Olympics and UEFA 2024 Euro Championships have coincided with a boom in popularity of live sports and esports among streaming audiences worldwide, and the prospect of haptic-enhanced video can add a new depth of sensation that will help audiences feel more connected than ever before.
Major breakthroughs are being made by organisations such as the Japan Broadcasting Corporation (NHK), which is creating specialist equipment to enhance the viewing of live sports, from handheld devices that react to an athlete’s motions, to chairs that respond to motions such as throws, tackles and physical contact, like the Razer Esther gaming seat cushion and Razer HyperSense headphone.
Limitations and concerns to keep in mind
In order to properly ensure that XR experiences are enhanced with the addition of new haptic technologies, it is essential to recognise the importance of the Quality of Personal Experience (QoPE). QoPE must not be determined by the haptic devices themselves but should be enhanced by their addition and adapted to each user. A robust wireless network performance that supports XR experiences is key. Mobile VR environments require high-speed, low-latency communications, and if the internet is to transmit tactile sensations in real time through streamed media, latency cannot exceed one millisecond if the human neurological system is to not dissociate the sensations with visual stimulus. Standards in wireless networking are crucial here – for 5G, 5GSA, 6G and beyond – to ensure the smooth and reliable transmission of sensory data.
The visual constraints of the current generation of haptic hardware also need to be considered. Many devices are currently heavy, bulky or restrict user movement. These sensations detract from the experience haptic tech is trying to achieve, by making it less natural and lifelike – and therefore less immersive – and hinders interoperability between different devices.
Support of codecs and standards is essential
Current haptic technology has limited most feedback systems to vibrotactile sensations. However, promising advancements in material science and wearable technologies indicate that responsive haptic devices such as full-body suits and advanced gloves capable of allowing users to experience sensations such as heat and texture are on the horizon.
For these advancements to truly take root within the VR space, adherence to standards and protocols will be integral. Innovations in immersive video codecs and advancements in haptics research are empowering a new dimension of interactive and sensory-enhanced communications, services, and experiences. The MPEG immersive video (MIV) standard allows for the compression of immersive video, in which a real or virtual 3D environment is captured by a number of real or virtual cameras or synthesised. These critical immersive standards enable enhanced interoperability and quality of experience across various devices and different immersive experiences, while achieving sustainable data rates to ensure they can be seamlessly streamed by networks.
MPEG standards such as Scene Description, Visual Volumetric Video-based codecs (V3V) and newer haptics-dedicated codecs enrich immersive experiences with a new and broader depth of tactile sensations. Through the support of these standards, a suite of haptic technologies can theoretically be deployed and supported, allowing users to experience a new, deeper layer of immersivity, whether they’re exploring virtual worlds, gaming in VR landscapes, experiencing live sports events, or enjoying their favourite TV shows and movies.
Enhanced virtual worlds on the horizon
Haptic technologies have the potential to revolutionise how we experience virtual worlds, entertain ourselves and even work. It may be some time before we see affordable and deployable suites of haptic tech that users can enjoy at home or in business environments, but strides are being made constantly and recent announcements show that it is coming. With 5G and 6G key enablers of XR experiences through immersive media codecs, the next generation of immersion is on the horizon, with the crucial technologies and standards being developed today.
