Mixed Reality & Virtual Reality
In 2016 I was fortunate enough to be at one of the conference parties where someone happened to have two Microsoft HoloLens headsets and was demonstrating them to the small crowd of curious nerds that had gathered around him. Well, I was one of those nerds and before long had the pleasure of donning one of the sets and so was introduced to the wonders of true mixed reality.
Much like a small welding mask, in both look and feel, the HoloLens is essentially a set of transparent screens that sit in one’s field of view by means of the headstraps that keep the device in place. Whilst not especially comfortable and certainly not something anyone is going to ever be in a rush to wear out in public on account of looking, frankly, ridiculous, the experience that it delivered was compelling. With the use of a simple gesture, specifically an upward ‘throwing’ movement, a menu popped into view suspended perfectly in mid-air and crystal clear as if it were right there in the real world in plain sight of everyone around me. Of course it wasn’t and the only person able to see this hologram was me. Selecting from the menu was as simple as reaching out and ‘touching’ the desired option and within seconds a holographic representation of the Earth was spinning languidly before me. I could ‘pick up’, ‘move’ and otherwise manipulate the item in front of me as though it were a physical object, and if I did move it, for example off to the right, out of my field of view, that is precisely where it remained and where I found it again when I turned back round. The human body application was similarly cool, as I was able to explore the various layers of anatomy through interaction with a highly rendered hologram. Whilst comical for onlookers not wearing a HoloLens, as I appeared to apparently be pawing away at thin air like someone suffering a particularly lucid acid hallucination, the thrill of what I was actually seeing and engaging with myself allowed me to ignore my daft appearance.
What are the medical education applications for such mixed reality technology? Whilst holographic visual representations of anatomy are, at first, a magical phenomemon to experience and a pretty cool party piece, it is the fact that mixed reality sees realistic holograms merged, or so it appears to the user, onto the real world, in contrast to virtual reality, which replaces the real world experience with an entirely digital one, that lends itself to unique educational applications. Anatomy instruction by being able to accurately overlay and track in real-time deeper layers onto a real-world physical specimen, enabling students to understand the wider context in which various anatomical structures sit is a far more compelling and useful application of MR than a simple floating graphic. Similarly, surgical training involving holographic overlays onto a real-world, physical object or combined with haptic technology to elicit tactile feedback, offers the potential to deliver programmable, repeatable, easily accessible practical training with minimal expense and zero waste on account of there being no need to have physical biological specimens.
Imagine: a fully-functional and resourced dissection and surgical training lab right there in your clinic or home and all at the press of a digital button. Imagine how confident you would become at that new, nerve-wracking surgical procedure if you had the ability to practice again and again and again, physically making the required cuts and placing the necessary implant, being able to make the inevitable mistakes that come with learning anything new but at zero risk to your patient. Being able to step up to the surgical plate for real and carry out that same procedure that you have rehearsed and developed refined muscle memory for, feeling the confidence that a board-certified specialist with years of experience has, and all without having had to put a single animal at risk – that’s powerful. That’s true action-based education at it’s most compelling and it is a future that both VR and MR promises.
I predict that the wide adoption of graphically rich, immersive and realistic digital CPD programmes, through both VR and MR, will result in a renewed engagement of professionals with CPD training and ultimately lead to more confident, skilled, professionally satisfied and happier clinicians. I, for one, know that were I able to complete practical CPD by simply donning a headset and loading up a Vive or HoloLens experience from the comfort and convenience of my clinic or home, all whilst still being able to interact in real-time with colleagues both physically present and remote, my CPD record would be bursting at the seams. That has to be a great thing for the profession, our clients and society in general.