Novel Vision for Neurosurgical Education: Anatomical Depth Estimation and 3D Reconstruction of Microsurgical Anatomy Using Monoscopic photogrammetry and Machine Learning

Introduction: Advances in artificial intelligence have enabled development of novel image processing and 3D rendering techniques. We implemented neural networks to transform 2D anatomical images into immersive, navigable, and volumetric models rich in anatomical detail useful for neurosurgery training and simulation.

Methods: Images were acquired from cadaveric cranial dissections and from an open-access online microsurgical anatomical image database (Rhoton Collection). A pre-trained neural network capable of depth estimation using a single 2D picture as input was used to create a depth maps (pixelated images containing distance and spatial information). Correlation analysis was performed between depth estimations and neuronavigation measurements for the dissections. Using 3D reconstruction libraries, volumetric models were created using the original photograph as texture of the rendering. Virtual reality (VR) experience was assessed using a VR headset, and augmented reality (AR) was assessed using a quick response (QR) code–based application and a tablet camera.

Results: Correlation was found between processed image depth (pixel depth) estimations and neuronavigation-defined coordinates at different levels of magnification [Spearman's Rho -1.0 (P < 0.001), -0.98 (P < 0.001 and -0.97 (P < 0.001) for low, intermediate, high magnification images of an orbitozygomatic approach dissection]. Twenty anatomical models were created from laboratory dissection and online database images. Interactive visualization and magnification allowed multiple perspectives for an enhanced VR experience. The QR code offered a convenient method for importing anatomical models into the real world for rehearsal and for comparing other anatomical preparations side by side.

Conclusion : We investigated the use of machine learning to render 3D reconstructions from depth estimation information acquired from 2D microsurgical images. Preservation of anatomical detail including texture, colors, and illumination added to an enhanced educational experience. Realistic depth and visuospatial perception of anatomical structures could be experienced by visualization of these models using virtual and augmented reality interfaces.