Towards computer aided guidance system for robot assisted laparoscopic radical prostatectomy
Date
2021
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
University of Delaware
Abstract
Robotic surgery platforms such as the da Vinci surgical system from Intuitive Surgicals along with pre-operative multi-parametric MR imaging have become increasingly common for surgical treatment and diagnosis of patients with prostate cancer. For surgeons using robotic surgical platforms, maintaining spatial awareness of the anatomical structures in the surgical area is key for good outcomes. They use the pre-operative imagery to plan for the surgery and build a mental map of the patient specific anatomical structures. An image guidance system is aimed at helping surgeons navigate and make critical decisions during surgery by fusing pre-operative and intra-operative, patient specific, image data. ☐ This thesis presents the work done towards developing a prototype Mixed Reality system which can interface with the da Vinci surgical platform. This system creates aligned 3D anatomical models from pre-operative MRI and intra-operative stereo laparoscopic images, and allows a surgeon to visualize and interact with them in a 3D virtual space. Creating 3D models of the anatomical structures for visualization involves defining a shape model/representation and then fitting it to the point cloud extracted from segmented pre-operative images. The thesis discusses the shape models and their fitting techniques that were developed during this work. The shape models discussed here include: (i) Compact global geometric model, which was used to model shapes of fungal spores and create templates for spore detection in microscopic images of maize leaf surface, (ii) Piece-wise implicit shape representation based on radial basis function neural networks, and (iii) A hybrid shape model as the combination of the two former classes of representation which has both implicit and parametric forms. A deep learning approach is also presented to model the statistical shape of the prostate which can be used to regularize the shape model fit to point cloud extracted from segmented pre-operative images. Images from stereo laparoscope of the da Vinci surgical robot are used to 3D reconstruct the patient anatomy of interest by employing calibrated shape from stereo images approach. To align the pre-operative and intra-operative 3D anatomy, a robust landmark free shape registration method which solves for the similarity transform to align shapes was developed. To visualize and interact with the aligned models, a mixed reality application was developed using a virtual reality framework which allows surgeons to view, correct for misalignments and manipulate models from both modalities in 3D. This allows them to visualize occluded and obscured organ boundaries as well as other important anatomy that are not visible through the laparoscope alone, thus facilitating better spatial awareness during surgery. As part of a feasibility study, the prototype system was deployed during 9 robot assisted laparoscopic prostatectomy procedures at National Institute of Health. The feedback from the experts indicate that such a system would be useful and encourages further development of the system for real-time deployment during robot assisted radical prostatectomy procedures.
Description
Keywords
Image guided surgery, Mixed reality, Robotic surgery, Shape from stereo, Shape registration, Shape representation