Design of interactive augmented reality functions for robotic surgery and evaluation in dry‐lab lymphadenectomy

Background Augmented reality (AR) has been widely researched for use in healthcare. Prior AR for robot‐assisted minimally invasive surgery has mainly focussed on superimposing preoperative three‐dimensional (3D) images onto patient anatomy. This article presents alternative interactive AR tools for robotic surgery. Methods We designed, built and evaluated four voice‐controlled functions: viewing a live video of the operating room, viewing two‐dimensional preoperative images, measuring 3D distances and warning about out‐of‐view instruments. This low‐cost system was developed on a da Vinci Si, and it can be integrated into surgical robots equipped with a stereo camera and a stereo viewer. Results Eight experienced surgeons performed dry‐lab lymphadenectomies and reported that the functions improved the procedure. They particularly appreciated the possibility of accessing the patient's medical records on demand, measuring distances intraoperatively and interacting with the functions using voice commands. Conclusions The positive evaluations garnered by these alternative AR functions and interaction methods provide support for further exploration.