Interactive medical augmented reality system for remote surgical assistance

Medical surgery is considered as a difficult technique and needs to be performed by well-trained and experienced medical personnel to complete the procedure. During a complicated surgery, an experienced surgeon at distant can provide sophisticated instruction or guidance to help the operating surgeon when it is necessary. In this study, we have designed an interactive augmented reality system for remote surgical assistance. The surgeon at the remote site can indicate or contour a region of interest (ROI) by manually drawing on streaming images which were captured by the camera at the local site. The coordinate of the ROI in the 3-D physical space can be estimated by using the CT images of the patient. Therefore, the ROI and the anatomical information of the CT images can be rendered on the captured images, overlaid on the patient and projected onto a head-mounted display of the local operating surgeon, providing an across-space augmented reality visualization.