Calibration and Application of Optical Stereo Vision System Based on PTZ Cameras

This three-year project aims at developing a stereo vision system based on PTZ (Pan-Tilt-Zoom) cameras. For the first year, we will develop a complete geometric model and calibration procedure for the intrinsic and external parameters of the system. Current available literature assumes that all the rotational axes are perpendicular with each other. This simplified geometric model seriously limits system accuracy and deteriorates stereo matching. We will develop the calibration method based on Zhang’s approach and the particle swarm optimization (PSO) method. In the second year, the dual-PTZ-camera stereo vision will be used for 3D reconstruction of real objects. Each 3D coordinate of an attribute location is calculated from matching patterns in the disparate images. To find stereo matching, DIC (Digital Image Correlation)、ALS (Adaptive Least Squares Correlation)、SIFT (Scale-Invariant Feature Transform) and Harris are available alternatives. Where DIC and ALS, although can provide accurate results, rely on accurate initial guess. On the other hand, SIFT and Harris can effectively manipulate translation, rotation, and scaling operations, but suffer from the scarceness of matching points. With these understanding, we will combine merits of SIFT and ALS by using SIFT to find reliable initial matching locations, and then search for more locations by the integration of transformation operation in the ALS with the PSO method. For surfaces lack of significant attributes, structured light will be applied. In the third year, the system will be extended to three PTZ cameras for a complete extraction of 3D surface information, such as human face, within one simultaneous capture. In each capture, disparity is generated between images captured from the left and the central camera, and those from the right and the central camera. The research will be focused on the integration of 3D coordinates obtained from these two image pairs, where the inconsistency is unavoidable due to limited calibration accuracy. The integration will be treated as a registration problem solved by the PSO method. The costs required for the optimization procedure can be measured using the Chamfer-based Distance Transform. Besides, a real object will be generated using a rapid prototyping machine for validation of the developed techniques.

Project ID:PB9907-13164
External Project ID:NSC99-2221-E182-012