HoloRegi

Neuronavigation with HoloLens 2

Neuronavigation is technologies used by neurosurgeons to navigate inside the patient’s skull by registering medical images and the physical world. At the neurosurgery OR in University Hospital Zurich, the geometry of the physical world is scanned by infrared system from Zeiss and the registration is done using software provided by BrainLab and Medtronic. They are highly delicate devices and are unaffordable for some hospitals. They are also sometimes inconvenient to work with because they take a long time to implement and the quality of registration can vary a lot.

A typical neuronavigation system using infrared in the OR. A mechanical part with four infrared balls is clamped to the head of the patient and detected. The software registers the 3D mesh reconstructed from MRI with the scanned data from infrared.

In current solution with Hololens-based neuronavigation, a pointer is used to indicate several points on both the hologram (mesh reconstructed from MRI and shown on the glasses) and the patient in the same order to register the hologram. There are several issues with this approach:

  1. It is time-consuming: The surgeon needs to manually indicate all the facial landmarks. And if one makes a mistake in the process, he/she has to start pointing all over again.
  2. It has large errors: According to report, the average fiducial registration error of this approach is about 8.5mm, which can be large for clinical accuracy.
  3. It is unstable: the quality of registration heavily depends on the accuracy of picked point pairs.
Neuronavigation developed by Augmedit. On the left, the generated MRI mesh. Middle, planning before neurosurgery. Right, guidance during neurosurgery.

With the novel technology of mixed reality, we implement a new neuronavigation system based on HoloLens 2, where the hologram of medical images is superimposed on the real patient using an automatic registration algorithm. The average registration error is only 2.5mm, enabling accurate neuronavigation and planning for surgeons. By solving the mentioned problems, we substantially improve the current solution and build a fast, accurate, and stable workflow.

Left: The test result on real patient in the OR. According to the feedback from surgeon, we have achieved clinical accuracy. Right: Test on phantom head. Because there is no noise around (such as the mouth tubing or hair), the registration is (and should be) nearly perfect.

If you want to discuss other potential surgical applications with mixed reality, feel free to email me!