Utilizing Depth Information for Emerging 3D Applications

Thesis (Ph.D.)--University of Washington, 2016-06

In recent decades, we have witnessed the explosive growth of research in computer vision, graphics, and robotics brought by the evolution of depth cameras. With depth information, computers will have a better understanding of the physical world. However, the quality of the depth images captured by current low-cost depth sensing devices is often poor and noisy. In this dissertation, we describe the efforts that we have made in utilizing the potentially noisy depth information for some emerging 3D applications. Our contributions are mainly in three areas: First, to improve the construction of 3D object models using noisy depth information, we propose a novel algorithm that can achieve a better 3D point cloud registration. Second, since the resolutions of the depth images from current low-cost depth sensing devices are often not sufficient for practical applications, we propose novel algorithms that perform better than existing state-of-the-art algorithms for depth image super resolution. Third, we utilize the depth information to help create a large-scale street-view dataset with semantic and instance level scene labeling using 3D to 2D label transfer for the purpose of scene understanding and autonomous driving research.