Iyengar, K;
Dwyer, G;
Stoyanov, D;
(2020)
Investigating exploration for deep reinforcement learning of concentric tube robot control.
International Journal of Computer Assisted Radiology and Surgery
, 15
pp. 1157-1165.
10.1007/s11548-020-02194-z.
Preview |
Text
Iyengar2020_Article_InvestigatingExplorationForDee.pdf - Published Version Download (1MB) | Preview |
Abstract
PURPOSE: Concentric tube robots are composed of multiple concentric, pre-curved, super-elastic, telescopic tubes that are compliant and have a small diameter suitable for interventions that must be minimally invasive like fetal surgery. Combinations of rotation and extension of the tubes can alter the robot's shape but the inverse kinematics are complex to model due to the challenge of incorporating friction and other tube interactions or manufacturing imperfections. We propose a model-free reinforcement learning approach to form the inverse kinematics solution and directly obtain a control policy. METHOD: Three exploration strategies are shown for deep deterministic policy gradient with hindsight experience replay for concentric tube robots in simulation environments. The aim is to overcome the joint to Cartesian sampling bias and be scalable with the number of robotic tubes. To compare strategies, evaluation of the trained policy network to selected Cartesian goals and associated errors are analyzed. The learned control policy is demonstrated with trajectory following tasks. RESULTS: Separation of extension and rotation joints for Gaussian exploration is required to overcome Cartesian sampling bias. Parameter noise and Ornstein-Uhlenbeck were found to be optimal strategies with less than 1 mm error in all simulation environments. Various trajectories can be followed with the optimal exploration strategy learned policy at high joint extension values. Our inverse kinematics solver in evaluation has 0.44 mm extension and [Formula: see text] rotation error. CONCLUSION: We demonstrate the feasibility of effective model-free control for concentric tube robots. Directly using the control policy, arbitrary trajectories can be followed and this is an important step towards overcoming the challenge of concentric tube robot control for clinical use in minimally invasive interventions.
| Type: | Article |
|---|---|
| Title: | Investigating exploration for deep reinforcement learning of concentric tube robot control |
| Location: | Germany |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1007/s11548-020-02194-z |
| Publisher version: | http://doi.org/10.1007/s11548-020-02194-z |
| Language: | English |
| Additional information: | This work is licensed under a Creative Commons Attribution 4.0 International License. The images or other third party material in this article are included in the Creative Commons license, unless indicated otherwise in the credit line; if the material is not included under the Creative Commons license, users will need to obtain permission from the license holder to reproduce the material. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ |
| Keywords: | Concentric tube robots, Deep reinforcement learning, Robot control, Surgical robotics |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10102276 |
Archive Staff Only
 |
View Item |
