Integrating Stereo Vision with a CNN Tracker for a Person-Following Robot
Abstract: In this paper we introduce a stereo vision based CNN tracker for a person following robot. The tracker is able to track a human in real time using an online convolutional neural network. Our approach enables the robot to follow a target under challenging situations like occlusions, appearance changes, pose changes, crouching, illumination changes or people wearing the same clothes in different environments. The robot follows the target around corners even when it is momentarily unseen by estimating and replicating the local path of the target. We build an extensive dataset for person following robots under challenging situations. We evaluate the proposed system quantitatively by comparing our tracking approach with existing real-time tracking algorithms.
The dataset, poster and paper are all available for download.
If you use the dataset in your research please cite the following paper(s):
Integrating Stereo Vision with a CNN Tracker for a Person-Following Robot, By Bao Xin Chen, Raghavender Sahdev and John K. Tsotsos, In the 11th International Conference on Computer Vision Systems, Schezhen, China, July 10-13, 2017.
Person Following Robot using Selected Online Ada-Boosting, Bao Xin Chen, Raghavender Sahdev and John K. Tsotsos, In 14th Conference on Computer Vision and Robotics, Edmonton, Canada, May 17-19, 2017.
Robot following a person under different conditions (varying poses, motions, temporary disappearance, illumination conditions, appearance changes, occlusions) in different environments:
| Crowded University Corridor. The robot is following the person in university corridors under crowded environments and is able to distiguish between 2 people wearing same clothes under occlusions, pose changes, etc. | University Hallways. The robot is able to follow the correct target under appearance changes, wearing/removing jacket, putting on/removing a bagpack, occlusions, tracking under similar clothes, etc. |
| Lab, Corridor, Seminar Room. The robot follows the person inside a lab environment, corridor and a seminar room. The robot is able to follow the person in cluttered environments, illumination changes, occlusions, etc. | Corridor/Elevator/Lab/Hallway The robot follows the person even when the target (human) cannot be transiently seen in the image. The robot replicates the local path for continuing the following behaivour. |
|
Corridor Sequence. The robot follows the person in a narrow corridor environment with occlusions from another person. |
Target Transiently Not Seen. The robot follows the person even when the target (human) cannot be transiently seen in the image. The robot replicates the local path for continuing the following behaivour. |