Utilizing Diffusion Machine Learning Models to Modify Neuroimaging: A Novel Tool for Neurosurgical Pedagogy and Operative Planning

Introduction: Diffusion models (DMs) are generative models that work by adding Gaussian noise to training data, and then recovering the data through a series of de-noising steps. These models have gained popularity over the past few years because they allow for robust artificial intelligence (AI) guided image creation and editing. Here, we show that DM inpainting can accurately add or remove pathology from a patient’s neuroimaging, allowing for the creation of hypothetical scenarios that can be used for neurosurgical pedagogy or operative planning.

Methods: We used DALL-E, a new DM-based AI system that can create realistic images and art from a description in natural language. Normal neuroimaging gathered from de-identified, publicly available sources were added into the DALL-E model including: Model 1) a lateral cervical X-ray and Model 2) an axial computed tomography head (CTH) scan. Sections of each image were intentionally deleted, and prompts were given to the DM model for image reconstruction and inpainting.

Results: A series of seven prompts were investigated using the DM model. Text prompts using Model 1 included: “anterior discectomy and fusion”, “anterior discectomy and fusion spinal screw”, “spinal cancer”, and “spinal fusion”. Text prompts using Model 2 included: “brain bleed”, “brain hemorrhage”, and “brain tumor”. In all cases, the DM-based model successfully added the desired prompts within the provided neuroimaging. Analysis of the new AI-generated imaging showed realistic image reconstructions using the DM inpainting technique.

Conclusion : Using DM-based AI algorithms, it is possible to edit patient neuroimaging to either add or remove pathology or surgical implants. As these models become more refined, they may be used as a neurosurgical pedagogy tool to simulate scenarios in which healthy patients have progressing neuropathology, or for purposes of removing pathology from a sick patient to simulate surgical treatment.