From teleoperation to autonomous robot-assisted microsurgery: A survey

Microsurgery requires manipulating delicate tissue or fragile structures such as small blood vessels, nerves, and tubes through a microscope. The operation accuracy of humans' hand is about 0.1 mm under optimal conditions, which makes microsurgical operation challenging. Physiological tremors or high-frequency involuntary hand movements with amplitude over 100 mm may influence safety during microsurgical operations.

Moreover, the poor sensory feedback in the microscale poses extract challenges to the maneuvers in confined environments. This indicates that the ability to measure the microscale interaction forces between the microsurgical tools and the objects is significantly important. Other challenges come from the mapping strategies for teleoperation, the limited region of interest provided by the microscope, the obstructed view when operating in complex structures, and potential collisions between instruments and delicate tissue regions.

To address the challenges mentioned above, the last decade has seen emerging technologies that assist robotic surgery in terms of imaging, sensing and robotics. Many great efforts have been made to develop robot-assisted microsurgical platforms. To ensure that no extra force is imposed on the targeted operating area, embedded micro-sensing systems for microsurgical tools are needed.

To protect the fragile micro components by uncontrollable exerted force, virtual fixtures are employed to avoid damage when approaching forbidden regions. Tremor removal techniques are utilized on the robotic platforms to ensure the reliability of the robotic system for microsurgery. Micro-imaging techniques at the cellular level expand the capabilities of microsurgical operations by providing accurate and effective guidance.

A paper, published in the journal Machine Intelligence Research, shows several key milestones for robot-assisted microsurgery (RAMS). One of the first microsurgical manipulators for eye surgery was developed at Northwestern University, following which the steady-hand concept was proposed by Taylor et al., targeted at submillimeter manipulation. This concept was further developed into the cooperatively controlled mode, in which the robot can assist the operator in manipulating tissue within defined force limits.

Followed by this tendency, the cooperative control paradigm is widely used in the robotic platforms for ocular surgery, where the surgical tool is held simultaneously both by a robotic arm and the surgeon's hand during the surgical operation. Robotic system "A 73" was developed for paranasal sinus surgery, which was tested on macerated cadaveric heads for validation.

In addition to the grounded robotic systems mentioned above, microsurgical handheld devices, also known as ungrounded robotic devices, are developed for microsurgery. Handheld devices are better alternatives in some scenarios since they are compact, lightweight and easy-to-use, compared to their grounded counterpart.

For example, Micron, a well-known handheld robotic device for microsurgery, can sense dynamic movements using accelerometers, and filter the tremor to provide stable operation. The end-effector of the robot is controlled by a parallel robot consisting of three piezoelectric stacks.

As for clinical translation, the world's first robot-assisted eye surgery was performed by surgeons at Oxford's John Radcliffe Hospital for a membrane peeling operation using the teleoperated preceyes surgical system. It was featured in motion-scaling and tremor-suppressing functions. The robotic ENT microsurgery system (REMS) is a commercial robotic platform, which was built specifically for otolaryngology-head and neck surgery (OHNS). Following that, the Bern University robot has shown promising results for ENT surgery in clinical translation.

In the microscale, the interaction between robots and humans has become much more challenging. In order to reduce the fatigue for surgical operation, provide greater dexterity during manipulation, avoid hand tremors and conduct more precise surgical tasks, human-robot shared control and robot learning techniques are worth developing to support RAMS.

With a higher level of autonomy, surgeons can focus on the crucial and complex parts of surgical procedures while the repetitive and tedious work can be done by robots. Moreover, a robot with higher intelligence may lead to a better operation quality, since the hand tremors can be removed while the precision for operation can be enhanced.

Microsurgical robots and instruments are developing towards safer and smarter for wider clinical uptake. A framework for six levels of autonomy for medical robotics has been proposed, which includes no autonomy (Level 0), robot assistance (Level 1), task autonomy (Level 2), conditional autonomy (Level 3), high autonomy (Level 4), full autonomy (Level 5).

Since there are remaining issues and open challenges for safety, ethics, regulation of medical robots with high autonomy, researchers mainly focus on microsurgical robots with autonomy level from 0 to 3. To this end, researchers will introduce the development of RAMS from teleoperation (no autonomy) to conditional autonomy with the support of advanced AI techniques including learning from demonstration (LfD) and reinforcement learning (RL).

Recently, various data-driven techniques such as imitation learning, RL, transfer learning and deep learning, etc., have been successfully employed for robotic skill learning and generalizing. The comparison and discussion of different machining learning methods for robot manipulation skill learning are extensively surveyed in the review paper.

By incorporating robot learning techniques into RAMS, the microsurgical systems can carry out some microsurgical sub-tasks automatically, which can reduce surgeons' workload and generate more clinical outcomes.

This review article is organized as follows. Section 1 is the introduction of the motivation and the key milestones of the development of RAMS. The current robotic platforms for different types of microsurgery are briefly introduced in Section 2. Section 3 describes available imaging modalities for microsurgery and microscale sensors that are suitable to be incorporated into microsurgical tools.

Grounded robotic systems for microsurgery are described in Section 4, while ungrounded handheld smart surgical devices are reviewed in Section 5, including force control and haptic feedback devices, tremor suppression and image-guided devices. Section 6 summarizes the existing robot learning techniques that have a high potential for deployment in RAMS and introduce the future-outlook for intelligent microsurgical robotics. Finally, conclusions are drawn and the evolution tendency is presented in Section 7.

Ophthalmology, otology, rhinology and laryngology are typical types of microsurgery. In Section 2, researchers illustrate the characteristics and challenges of different types of microsurgery, which reveals the benefits provided by the RAMS and numerous opportunities in this research area. They also introduce microsurgical skill training using robotic platforms, which is indispensable for the development of RAMS.

Imaging and sensing systems are indispensable for RAMS. The image quality has a close relationship with the construction of accurate 3D surface models of patients' anatomy, the detection of the disease location. The detection of microstructures throughout the intraoperative phase is important for further decision-making and operation.

Moreover, microsurgery is normally targeted at delicate and precise tissue manipulation. The measurement of the microscale interaction forces between the manipulator and the tissue is required for microsurgery to provide force feedback to the operators. This leads to the development of microscale sensing techniques for RAMS. In Section 3, available medical imaging modalities and microscale sensors for integration to microsurgical tools are introduced.

Grounded robotic systems for microsurgery can be divided into master-slave teleoperation systems and collaborative control systems. Master-slave control, also known as leader-follower control or teleoperation, is a common form of control for robotic surgery. The motions of the operator are captured by the master devices and replicated by the slave robot. As for cooperative control, the surgeon and the robot held the surgical tool together for surgical operation.

The force exerted by surgeon on the tool can be sensed through a force sensor, while the tool can move accordingly. The motions of the robot comply proportional to the user exerting forces, which can ensure safety. In Section 4, grounded robotic systems for microsurgery are introduced. In this section there is a figure that shows some typical grounded microsurgical robots for RAMS and a microsurgical research platform and a Table that summarizes the grounded microsurgical robotic platforms introduced in this review paper.

Ungrounded handheld smart devices can be easily integrated into the surgical workflow without disrupting the traditional procedures of operation. Handheld robots assist the operating surgeon in a more natural way without motion constraints since no mechanical linkages are involved. Force control and haptic feedback devices, tremor suppression and image-guided devices have been explored to demonstrate the emerging trends and opportunities for handheld robots for microsurgery. In Section 5, researchers introduce ungrounded handheld devices for microsurgery. In this section there is a figure that shows some typical types of ungrounded handheld devices for microsurgery and a Table which summarizes the key features of the ungrounded microsurgical robotic devices introduced in this review paper.

Among the learning methods, learning from demonstration (LfD) (also named programming by demonstration PbD or imitation learning) is an effective way to transfer skills from humans to robots. The effectiveness of LfD has been proved in several fields, such as assembly, medical scanning and robot-assisted rehabilitation.

Reinforcement learning (RL) enables the robot to learn in an interactive environment by trial and error. It has been used for medical robotics in health care systems. Researchers believe that RL can be applied for intelligent microsurgical robots and contribute to microsurgery. In Section 6, researchers introduce robot learning techniques that have a high potential to support RAMS by enhancing the autonomy level of microsurgical platforms or devices.

The future of microsurgery is developing toward early and precision intervention, this requires the development of micro-instruments that can facilitate in vivo microscale surgical operation. Researchers envision that the inherent challenges for RAMS will be addressed while robot-assisted techniques can enhance the efficiency of microsurgery and bring significant clinical benefits.