Vision-Based Stair Detection and Terrain Classification Algorithms for Multi-Terrain Mobile Robots

Abstract

This thesis presents vision-based stair detection and environment classification algorithms for mobile robots capable of traversing staircases and different types of terrains. These algorithms are developed for a specific hardware platform, called \Robot{}, which is equipped with wheel-and-leg transformable mechanism enabling multi-terrain locomotion. The design of the hardware platform is optimized to allow for climbing over irregular terrains and continuous obstacles, such as staircases. It is equipped with Jetson TX2 as the main processing board, an Inertial Measurement Unit (IMU) and a Global Positioning System (GPS) for odometry, and a Light Detection And Ranging (LiDAR) device and an RGB-Depth (RGB-D) camera. The stair detection algorithm takes the color and depth image feed from the RGB-D camera and uses it to identify straight line patterns that could constitute a stairway. To further embed the robot with the terrain classification capability, the color images are segmented into traversable and non-traversable regions, thereby making urban environments more accessible. Taking the computational limitations into account, it is explored how these schemes can be integrated into the robot navigation stack using Robot Operating System (ROS).