Utilization of Augmented Reality in Assisting Surgical Needle Insertion Guidance

Surgical needle insertion is a standard and routine medical procedure in medical practice, such as abdominal radio frequency ablation (RFA) and needle biopsy. To guide the needle accurately inserted into the targeted deepseated position in the tissue following a designated trajectory, the clinicians majorly rely on real-time image feedback for needle trajectory visualization and prediction via ultrasonography and/ or optical camera feedback. However, during needle puncture guiding procedures, the needle trajectory may not always be accurately reported, leading to inconvenience or even post hazards to the procedure, with one major leading obstacles is the needle can only be tracked with a limited camera field-of-view. Here, we proposed a hybrid pose estimation augmented reality (AR) algorithm that allows simultaneous conduct of entire needle set tracking to broaden the detectable operation range of the needle insertion procedure. The algorithm not only can track the 3D pose of the needle handle (via marker tracking), but can also track the pose of the needle itself (via key points detection), therefore enlarging the trackable zone during the procedure. The detectable view is measured to have up to 3.13-fold field-of-view improvement, with an overall accuracy of 2.9\pm 0.01 mm being achieved. The proposed algorithm provides an advancement in using CCDcamera as a tool to implement AR-based surgical guidance and should benefit the current surgical procedure such as the RFA or biopsy.