A Hmd Visualization Based Mixed Reality System for Applications in Neurosurgery

In this proposal, we demonstrate a head-mounted display (HMD) devices based mixed reality (MR) system for applications in neurosurgery. We expect that the system can be applied to different stage in medical treatment procedure, such as pre-operative 3D medical data inspection and analysis, medical diagnosis explanation, pre-operative surgical planning, surgical simulation and image-guided navigation during the surgery (IGNS). Since virtual and augmented reality HMD devices and visualization technologies have their own advantages and disadvantages in different way, we believe that it is necessary to design different visualization protocols for different stage in the treatment procedure. In this study, we plan to design three different visualization schemes by using different MR visualization techniques, and we expect the system to become a role of aid in the procedure. This project is expected to be accomplish in three years. In the first year, immersive VR visualization scheme for 3D medical object visualization will be completed. In the second year, two types of AR visualization scheme will be accomplished, including optical see-though AR immersive video see-through based AR. In the last year of the project, we will integrate different techniques and apply it to total procedure to evaluate the result.

Project ID:PB10607-1400
External Project ID:MOST106-2221-E182-025