Non-Rectified Binocular Stereo Matching

Stereo vision is an important non-invasive ranging technology, which is widely used in many scientific and industrial applications, such as industrial automation, unmanned-vehicle navigation, 3D tracing, security monitoring, and aerial photogrammetry. In applying a stereo vision system for 3D data reconstruction, stereo matching is required to triangulate coordinates based on epipolar geometry. The procedure is usually simplified into one dimensional problem by rectifying the image pairs. However, the rectification is a nonlinear mapping which is prone to failure in low textured regions. In this two-year project, we will focus on the development of advanced stereo matching methods for unrectified image pairs. For the first year, we will firstly advance the development of a reliable calibration procedure to enhance accuracy of stereo vision systems. We plan to apply the particle swarm optimization method on the checkerboard calibration method to study the relationship between non-parallel binocular stereo vision system and two-dimensional disparity maps. Then, based on the edge-preserving guided image filter model, we will build a two-dimensional aggregation scheme. In order to analyze the performance of the algorithm, we will also investigate evaluation method for the matching performance that is based on feature point detection and two-dimensional disparity maps. In the second year, we will study the possibility of introducing deep learning techniques to enhance the robustness of two-dimensional stereo matching. Deep machine learning, such as Autoencoder and Convolutional Neural Network (CNN), will be introduced for the matching. The performance evaluation method developed in the first year will be used for network training. Finally, experimental evaluations on benchmark and challenging real-world images will be conducted to confirm effectiveness of the developed scheme.

Project ID:PB10703-1482
External Project ID:MOST106-2221-E182-033