MiDAS: A Multimodal Data Acquisition System and Dataset for Robot-Assisted Minimally Invasive Surgery
Published in The International Journal of Medical Robotics and Computer Assisted Surgery, 2026
This paper introduces MiDAS, an open-source and platform-agnostic multimodal data acquisition system for robot-assisted minimally invasive surgery. The system synchronizes external hand tracking, RGB-D sensing, pedal signals, and surgical video without requiring proprietary robot interfaces, making it easier to collect rich datasets across both research and clinical robotic platforms.
The paper validates these non-invasive sensing modalities against internal robot telemetry and shows that external motion signals can support downstream gesture recognition with performance comparable to robot kinematics in some settings. It also releases new multimodal datasets, including realistic suturing data from da Vinci Xi training scenarios, to support reproducible research in surgical activity understanding.