Robot Control Using Natural Instructions Via Visual and Tactile Sensations

Stress-free interaction between humans and robots is necessary to support humans in daily life. In order to achieve this, we anticipate the development of new robots equipped with tactile and vision sensors for receiving human instructions. In this article, we focus on spontaneous movements that do not require training, such as pointing and force adjustment and that are suitable for daily care. These movements, which we call natural instructions, involve the transmission of human instructions to robots. In this experiment, we examine a robot equipped with vision and tactile sensors capable of receiving natural instructions. Our new robot accomplishes a retrieving and passing task using the natural instructions of finger pointing and tapping with the palm.