Comparing a Real-time Artificial Intelligence Instructor to Human Expert Instructors in Teaching of Simulated Surgical Technical Skills– a Randomized Controlled Trial

Introduction: Errors relating to technical skills can result in poor patient outcomes in brain tumor surgery. Trainees develop their bimanual skills mainly by apprenticeship model, however, this training may involve increased risks to patients. Artificial intelligence (AI)-powered virtual reality systems provide risk-free training and an objective performance evaluation during realistically simulated neurosurgery cases. This work compared a real-time intelligent tutoring system with in-person human instruction in teaching surgical technical skills.

Methods: Ninety-eight medical students from four medical students participated in the trial. They performed a virtually simulated brain tumor resection five times and were randomly allocated into three feedback groups: (1) no-real-time feedback, (2) real-time AI instruction, (3) in-person human instruction. The first repetition of the task was performed without receiving feedback (baseline). Participants in Group-1 received visual post-hoc feedback. Participants in Group-2 were given auditory instruction in real-time by the AI system involving six instructions. After each task, the students were shown their performance error-video clips generated by the AI system alongside the expert-level demonstrations relating to each error. Participants in Group-3 were instructed by an expert instructor present in the room during the tasks. After each task, the instructor summarized the areas of improvement and demonstrated correction techniques. All participant data was scored by the AI system. Participant performance score was compared from the first to the last task within groups and between groups to compare learning.

Results: Participants in AI and expert-instruction groups significantly improved their performance scores by the third and second repetition, respectively (p < 0.01, p=0.01), compared to the baseline performance. Participants in AI group achieved significantly higher scores than expert-instruction group in the fifth repetition (p < 0.01).

Conclusion : Intelligent systems may provide efficient, objective pedagogical tools in the teaching of surgical bimanual skills in tumour surgery. These systems may aid in the development of competency-based standardized surgical curricula.