Research on Slam Techniques for Handheld Devices with Single Camera

Hand-held intelligent devices, which are equipped with camera, accelerometer, gyroscope, compass and large display, are proliferating to consumers in their everyday lives. Mainstream products of the devices include iPod touch, iPhone and iPad of Apple, Galaxy Tab of Samsung, ViewPad of ViewSoni, and Eee Pad of ASUS, just to name a few. Some interesting programs on the smart phones, such as the application called Monocle for the iPhone, developed by Yelp, are able to superpose data on live video feed, allowing user to conveniently get information of the road, buildings, and restaurants, etc. of his surrounding area. These applications, although useful, do not fully exploit the potential of the devices, as them require only rough registration between the images and the overlaid information. Accurate registration is required when texts or graphs are to be precisely superimposed at specific locations of the scene, such as the situation when the police are estimating the direction to an object from the outside of a building, assuming the relative position between the object and the building is known beforehand. In accurate registration, estimating the motion of the device relative to a sparse map of landmarks for real-time camera tracking is required. This is part of a well defined Simultaneous Localization and Mapping (SLAM) problem in the robotics community, where localization and mapping of a scene are deeply coupled. Nevertheless, computational resources in the handheld intelligent devices are highly limited and not familiar to ordinary programmers, making tackling the problem a true challenge. Besides, there is lack of reliable motion model for the handheld devices when compared with the ordinary visual SLAM problems. We plan to use the information supplied by accelerometer and gyroscope in the extended Kalman filter to undertake this difficulty. This two-year project aims at investigating the real-time visual SLAM problem for a hand-held intelligent platform, equipped with a camera, an accelerometer, a gyroscope, and a compass, under practical constrains on available computing power, and exploring its potential applications. For the first year, we will work on the developing skills of the platform, develop a procedure for the single-camera calibration, and implement algorithms for feature extraction, and pattern matching. We plan to adopt the Harris Corner Detector algorithm for feature extraction, and Sum of Squared Difference (SSD) and Cross Correlation for pattern matching. In the second year, we will develop the visual SLAM algorithms based on the results of the first year. The algorithms will fully utilize the real-time information supplied by the available sensors. Furthermore, the depth values of feature points, which are difficult to estimate consistently, will be inferred by the displacement obtained by the SLAM algorithms and confirmed by the Bayesian inference.

Project ID:PB10007-2318
External Project ID:NSC100-2221-E182-008