In Episode 1, we explore the application of multi-sensory virtual reality in healthcare, and specifically future applications to surgical training.
Given the development of COVID 19, the world may seek to accelerate future ways to provide healthcare diagnostics and treatments – remotely and virtually. Haptic perception (the sense of touch) is a concept that is in its infancy and currently has not developed full feedback that can apply to all parts of the body for surgical training.
We chat with Dr. Pierre Boulanger about the application of proxy haptics or proxy touch that will enable full body force feedback in virtual reality training for surgical teams.
We’ve all seen Yoda do his force feedback training with Luke Skywalker.
But back to reality and this episode – Pierre explains how it works and Madhi, Pierre’s student does the demonstration using proxy haptic technology.
Dr. Boulanger’s Blog
In our laboratory, we have developed a solution to the realism of haptic perception in VR by using the concept called “proxy haptics.” In proxy haptics, real physical props are placed around the real environment to match their virtual counterparts. If the physical props are co-registered with the virtual world, a compelling sense of tactile sensation can be achieved, for example, how a mannequin can be used to represent a virtual patient. If the mannequin is co-registered in 3D relative to the virtual user locations, one can be tricked to believe that they are touching a real patient.
On Multi-Sensory Virtual Reality
In recent years and with the advancement of technologies, the applications of Virtual Reality (VR) have been used in many fields. Technologies such as high-resolution digital displays, GPUs and CPUs, are now able to render complex virtual worlds in real-time. Modern VR systems can create high-quality VR environments that can be used in numerous applications such as entertainment, education, and medical training.
VR has been used in surgical training to either replace more expensive training techniques (such as cadavers) or serve alongside additional training techniques to increase a surgeon’s skills. As VR technologies evolve, one can trick the brain into thinking that a user is seeing and touching real-world objects. To trick the brain at a believable level, one needs to be able to generate feedback to multiple human senses (vision, haptic, sound) that are realistic and consistent with behaviors in the real world. Because of these requirements, many systems have been proposed to deal with the multi-sensory outputs VR needs to produce in order to be useful for real-world applications such as surgical training. Visual perception can be dealt with very well using modern Head-Mounted Displays (HMDs) and advanced rendering software.
Haptic perception however is still at its infancy and has not reached the same level of realism that HMDs can provide. Many haptic systems use force-feedback devices to create a sense of touch, most of them require to hold a wand attached to a small robot or wear a glove that provides forces or friction to a user’s hand movement. The problem with standard haptic devices is that they do not provide haptic feedback to all parts of the body hence reducing the sense of immersion and making their use impractical in complex simulations.
About Dr. Pierre Boulanger PhD
Pierre is currently the Director of the Advanced Man Machine Interface Laboratory (AMMI) as well as the Scientific Director of the SERVIER Virtual Cardiac Centre in the Mazankowski Heart Institute. In 2013, Dr. Boulanger was awarded the CISCO chair in healthcare solutions, a 10 years investment by CISCO systems in the development of new IT technologies for healthcare in Canada.
Dr. Boulanger received his in Engineering Physics and his Masters in Physics Laval University, and his Ph.D. in Electrical Engineering from the University of Montreal. Dr. Boulanger cumulates more than 35 years of experience in 3D computer vision, rapid product development, and the applications of virtual reality systems to medicine and industrial manufacturing. Dr. Boulanger worked for 18 years at the National Research Council of Canada as a senior research officer where his primary research interest was in 3D computer vision, rapid product development, and virtualized reality systems. He now has a double appointment as a professor at the University of Alberta Department of Computing Science and at the Department of Radiology and Diagnostic Imaging.
