Controle natural humano-robô orientado a usuário com Thin-plate Splines e LRCN

Abstract

This work proposes a novel vision-based robotic-arm teleoperation approach. By using a single depth-based camera, such an approach exempts the user from using any wearable devices. Through applying a natural user interface, such an approach also leverages the conventional fine-tuning process of the robotic position control calibration, turning the process into a direct capture of the human body. The proposed approach consists of two main parts. The first is a nonlinear customizable movement mapping based on Thin-Plate Splines (TPS), to directly transfer human body motion to robotic arm motion. Such mapping allows for matching dissimilar bodies, with different kinematics constraints and different workspace shapes. The second is a Deep Neural Network hand-state classifier based on Long-term Recurrent Convolutional Networks (LRCN), which exploits the temporal coherence of the acquired depth data. In the end, validation and evaluation of the proposed approach are performed. For the hand-state classifier, a cross-validation experiment comparing the current approach with a baseline is performed. Results reveal an increase in the classifier accuracy through exploring the temporal coherence present in sequential depth data. For the movement mapping, a user study is performed over a set of practical experiments involving variants of pick-and-place tasks in a simplified manufacturing environment. For this study, we developed a validation environment using Robot Operating System (ROS) as the main framework. Also compared to a baseline, the position mapping approach using TPS revealed better comfort and precision of user control in regions near to robot workspace boundaries, where the baseline approach showed a poor performance. Moreover, results suggested that the new approach did not present an increase in the experiment’s task difficulty.