Surgeons in the UK are using Microsoft’s mixed-reality headset to “see inside” patients before they operate on them.
A team at Imperial College London are wearing HoloLens devices in operating theatres so they can spot key blood vessels, bones and muscles, thus making procedures quicker and safer.

HoloLens allows the surgeons to take CT scans that have previously been completed and overlay 3D digital models of them on to a patient’s limb during reconstructive surgery.

The technique has been used to help surgeons successfully move blood vessels from one part of the body to another to help open wounds heal. Patients have included a 41-year-old man who injured his leg in a car crash, an 85-year-old woman who fractured her fibula and a person who developed an infection that required surgery.

Dr Philip Pratt, a research fellow in the Department of Surgery and Cancer at Imperial College London, says HoloLens is allowing surgeons to understand a patient’s unique anatomy very quickly and accurately.

“To perform the best operation, you have to plan it meticulously beforehand. This technology allows us to experience the data that we have collected from patients before their operation in the most realistic and natural way. You look at the leg and essentially see inside of it; you see the bones and the course of the blood vessels,” he says.

James Kinross, a consultant colorectal surgeon at St Mary’s Hospital who has used HoloLens during operations, agrees that seeing inside a patient could be crucial to the success of a procedure.
“You don’t want to make an incision and find out that you should be two centimetres over here, because that might compromise the operation. This is all about best outcome for the patient.

Patients who have suffered an accident may have open wounds that require reconstructive surgery. Skin and blood vessels are taken from a healthy part of the body and used to cover the wound, enabling it to close and heal properly.

A vital step in the process is connecting the blood vessels of the “new” tissue with those at the site of the wound, so oxygenated blood can reach the area.

CT scans are conducted on the patient to map the limb before the operation begins. These are then uploaded to HoloLens, which places the images on top of the patient. Multiple surgeons wearing HoloLens headsets can also see what their colleagues are specifically looking at, allowing greater collaboration.

Surgeons have traditionally used a handheld ultrasound scanner to find vessels under the skin by detecting the movement of blood. However, this is very time-consuming and still requires some guesswork as to where the vessels are and their path through body tissue.

“Mixed reality offers a new way to find these blood vessels accurately and quickly by overlaying scan images onto the patient during the operation,” Dr Pratt adds.

While the use of HoloLens in operating theatres is at an early stage, researchers are confident it could be applied to other areas of surgery requiring tissue flaps, such as breast reconstruction following mastectomy.

They are keen to conduct further trials with larger sets of patients and at different hospitals.

Rather than place users in a fully computer-generated world, as virtual reality does, HoloLens allows users to place 3D digital models in the room alongside them. As the Windows-10-based product does not have wires or external cameras, or require a phone or PC connection, users can walk around the objects they create and interact with them using gestures, gaze and voice.

It has been used by NASA to recreate Mars in its offices, allowing scientists to conduct virtual operations on the Red Planet; police forces are using HoloLens to “record” crime scenes and reconstruct them at police stations; and architects are utilising the device to create better public buildings.