In the previous Episode 1,”From Bricolage VR to the Robotics Lab”, Dr. Martin Ferguson-Pell spoke of two streams of VR. In Episode 2, we focused on the second stream – of exploring VR technology applications with inter-disciplinary teams at the Alberta Robotics Lab, in collaboration with faculties of medicine and rehabilitation medicine. In Episode 2, we discussed the key component of pragmatics through speech linguistics helps to create a more interpersonal VR experience.
As Dr. Martin Ferguson-Pell stated, “The second stream will create remarkable immersive, extended reality experiences that integrate sophisticated graphic design, animation and artificial intelligence to produce highly interactive experiences where the user is able to move spatially in the simulation. In the future, these systems will add authentic audio, haptics and other sensory experiences so that the user believes they are present. ”
In this Episode 3, Dr. Martin Ferguson-Pell shares insights on how the sophisticated motion haptics system in their Robotics Lab enables them to use a dancer’s motions to create a more realistic animation for a more realistic interpersonal VR experience. So what is haptics? “Haptic technology or kinesthetic communication recreates the sense of touch by applying forces, vibrations, or motions to the user“. Source Wikipedia
FutureCite:
From the motion haptics video, can you provide further insight into how your research teams used haptics to create a more real life VR experience?
Dr. Martin Ferguson-Pell:
In our Occupational Therapy Objective Clinical Structure Exam (OSCE) example we were able to combine a traditional VR experience with natural language processing and AI to enable a student to practice a clinical exam, before doing the real thing. We know that our students experience very high levels of anxiety before taking their OSCE exam. They have to interview standardized patients (actors) to demonstrate both the clinical diagnostic skills and also their interpersonal, emotional intelligence skills. We are concerned that what we are actually examining is a student’s ability to manage the anxiety, rather than their clinical know how. Our OSCE VR experience enables students to practice with an AI-enable avatar. The Q&A associated with an OSCE tests their clinical knowledge and if they wish they can review their interactions critically before taking the OSCE exam proper. The response so far has been very positive and students in other health disciplines who have to take similar exams are beating a path to our door, asking us to create similar simulations for them. The wider scope to use these techniques for other examinations, interviews and anxiety raising situations we believe is quite large, and success in their development will hinge on how immersive and realistic we can make the experiences, and particularly the interactions with the avatar.
FutureCite:
Martin, so in short what we have covered are applications in VR for healthcare where by adding core elements of pragmatics (one of which is speech), and another haptics (one of which is motion), we can enrich the interpersonal VR experiences of users.
About Dr. Martin Ferguson-Pell
Dr. Ferguson-Pell is a biomedical engineer and credentialed clinical scientist. He has dedicated his academic career to the study of secondary complications of physical disability and studying underlying causes to reduce their incidence. He designed and developed a new Rehabilitation Robotics Laboratory in the Edmonton Clinic Health Academy and uses virtual reality systems to simulate the biomechanical and physiological conditions faced by long-term wheelchair users and elite wheelchair athletes. This work includes the development of a novel approach to analyzing EMG from upper extremities and shoulders using wavelet + principal component analysis integrated into an FPGA system. The system can identify and measure, in real time, muscle fatigue and changes in muscle recruitment patterns.
About the University of Alberta Robotics Lab
The Rehabilitation Robotics is an inter-disciplinary research group focused on improving quality of life through robotics and technology. Their research focuses on wheelchair biomechanics, new technologies to assess spinal structure and function, assistive robotics and virtual reality in rehabilitation medicine.
