Project Description: In this project, we present an approach that allows an autonomous robotic blimp to detect and follow a human. This accomplishment is the first Human Robot Interaction (HRI) demonstration between an uninstrumented human and a robotic blimp. Robotic blimp is an ideal platform for HRI missions because it is safe to humans and can support sufficient flight time for HRI experiments. However, due to complex aerodynamic influence on the blimp, the human following task for indoor robotic blimp with a single on-board camera is a challenging problem. We integrate face detection and KLT feature tracker algorithms to achieve robust face detecting and tracking. Based on the face center detected in the real-time video stream, we estimated the 3D positions of the human with respect to the blimp. Vision-based PID controllers are designed based on estimated relative position and the motion primitives of the robotic blimp such that it can achieve stable and continuous human following behavior. Experimental results are presented to show the human following capability. The video demo conducted by Dr. Yao is available on Youtube.
In addition, we also demonstrated a natural interaction between a human and a robotic blimp via human face detection and hand gesture recognition. The blimp can detect a human’s face and hand, recognize vertical and horizontal hand gestures and react to the gestures by spinning and moving away from the human. The video demo is available.
Related publications:
- Ningshi Yao, Qiuyang Tao, Weiyu Liu, Zhen Liu, Ye Tian, Peiyu Wang, Timothy Li and Fumin Zhang, “Autonomous flying blimp interaction with human in an indoor space,” in Frontiers of Information Technology and Electronic Engineering, 20(1), 45-59, 2019. Available online [bibtex]
- Ningshi Yao, Emily Anaya, Qiuyang Tao, Sunji Cho, Hongrui Zheng and Fumin Zhang, “Monocular Vision-based Human Following on Miniature Robotic Blimp,” in Proc. 2017 IEEE International Conference on Robotics and Automation (ICRA), 3244-3249, Singapore, 2017. Available online [bibtex]