Whole-Body Safety-Critical Control Design of an Upper Limb Prosthesis for Vision-Based Manipulation and Grasping

In this paper, an upper limb prosthesis has been furnished with a novel vision-based manipulation and grasping strategy. The proposed whole-body safety-critical control design includes vision servoing, multiple tasks planning with strict priorities, which can be formulated as an hierarchical multi-task optimization (HMO) problem with safety conditions-expressed as control barrier functions (CBF). Firstly, a modified YOLOv7 algorithm with key points detection is developed to determine the grasping pattern of the object and extract its edge contour information using a depth camera. An HMO-based strategy with a notion of CBF, providing inequality constraints in the control input, is proposed to handle multiple prioritized tasks with various constraints to offer guarantees of safety with the whole-body motion in consideration. Then the HMO problem is solved by a neuro-dynamics optimization solution online. Finally, experiments are implemented by using a self-developed upper limb prosthesis. Experimental results validate the performance of the proposed whole-body control strategy. Note to Practitioners -The intuitive, convenient and autonomous control of prosthetics has always been the object of researchers' efforts. This paper proposes a whole-body safety-critical control design, including artificial perception system, autonomous control system under visual guidance, and user volition control system. The HMO strategy with safety conditions - expressed as control barrier functions in the context of real-time optimization-based method can fully consider the task requirements of different priorities and realize the orderly execution of prosthetic tasks. The vision feedback can detect surrounding environments in real time to obtain fine position information and optimal grasping pattern, which are conducive to reduce the amputees' cognitive burden and provide new ideas for the whole body control of prostheses.