Integration of PTZ Camera and Kinect Range Sensor for Three-Dimensional Reconstruction

Project: National Science and Technology CouncilNational Science and Technology Council Academic Grants

Project Details


3D data acquisition and reconstruction is a technique for obtaining geometric digital models of objects or scenes from sensed data. With the progress of stereo vision algorithms and availability of low-cost cameras, the recent decade has seen remarkable growth in the use of binocular stereo vision system for 3D reconstruction. In our previous project, camera calibration algorithms are developed for extrinsic and intrinsic parameters of a PTZ stereo vision system. Noticeably, cameras with pan-tilt-zoom maneuverability allow for greater freedom in data collection. However, commercially available PTZ cameras, such as Sony D-70, has only a limited pixels of 640 by 480 and their repeatability depends on moving direction with significant individual difference due to backlash, rendering them unsuitable for precise applications. We thus plan to build a customized binocular stereo vision system in this project, which will be equipped with higher resolution cameras of 1920 by 1020 pixels and driven by servo motors. In addition to extrinsic and intrinsic parameters of a stereo vision system, corresponding locations on a pair of images of the same scene are required to triangulate three dimensional coordinates. When there is significant deviation in view angles or the scene is lack of image features, the search of reliable corresponding location is vulnerable to failure and is an important problem pending solutions. Originally bundled with Xbox 360 to play games without holding a controller, the Microsoft Kinect is a low-cost range-imaging sensor capable of delivering 3D point clouds and images in real-time. While spatial and depth resolution may limit its use in high-accuracy systems, Kinect has attracted interest from developers wishing to exploit the system's 3D imaging capabilities to industrial and scientific applications. In this project, we will integrate Kinect with the home-made binocular stereo vision system by making use of the depth information provided by Kinect to find corresponding locations on pairs of images of the same scene. The uncertainty model will be used for the initialization of potential image regions within patches commonly seen by the cameras. The project will be conducted in two years aiming at enhancing accuracy and reliability of the stereo vision system by integrating the home-made PTZ binocular system with Kinect. For the first year, we will build the system, investigate accuracy and resolution of Kinect, and develop a registration algorithm between the binocular subsystem and Kinect. These efforts will result in the construction of error distribution model of Kinect and put reconstructed 3D feature points in the same coordinate system. In the second year, we will research into the issues of stereo matching for 3D reconstruction. The project will focus on providing reliable algorithms to search for corresponding locations on a pair of images from the binocular system. The registration algorithm and error distribution model of Kinect developed in the first year will be used to facilitate the integration of data points collected from the maneuver of the PTZ cameras.

Project IDs

Project ID:PB10207-1806
External Project ID:NSC102-2221-E182-073
Effective start/end date01/08/1331/07/14


  • Kinect range sensor
  • Active sensor calibration
  • 3D acquisition
  • Error


Explore the research topics touched on by this project. These labels are generated based on the underlying awards/grants. Together they form a unique fingerprint.