Capturing, Processing, and Synthesizing Surfaces with Details

Abstract

Geometry acquisition and processing have become increasingly popular in computer graphics and vision, with demand for high-quality models driven by advances in 3D printing, realistic real-time renderings of 3D avatars in video games, digital libraries for historical objects etc. In this thesis, we focus on techniques to produce and process detailed geometry including acquisition of real world objects, processing and fusing the captured data, and synthesizing new surfaces from existing ones.

First, we summarize a 2D acquisition technique called photometric stereo to capture high resolution surface details. As a validation step, we solve a misalignment problem for photometric datasets. After the dataset is validated, the surface normals can be computed using one of the photometric stereo algorithms. In order to decide which algorithm to use, we present a synthetic photometric benchmark to evaluate various algorithms for different scenarios.

To produce a detailed surface in 3D, we propose an approach to combine a rough 3D geometry with detailed normal maps obtained from different views. We begin with unaligned 2D normal maps and a rough 3D geometry, and automatically align each normal map to the 3D geometry. Next, we map the normals onto the surface, correct and seamlessly blend them together. We then optimize the geometry to produce a high-quality 3D model.

Next, we introduce a semi-automated system to convert photometric datasets into geometry-aware non-photorealistic illustrations of surface details that obey the common conventions of epigraphy (black-and-white archaeological drawings of inscriptions). This system is composed of rectification of the surface normals to correct camera perspective, segmentation of the inscriptions from the background, classification of the inscription based on carving technique, and stylization of the inscriptions in various styles.

Finally, we present an algorithm for realistically transferring surface details (specifically, displacement maps) from existing high-quality 3D models to simple shapes that may be created with easy-to-learn modeling tools. Our key insight is to use metric learning to find a combination of geometric features that successfully predicts detail-map similarities on the source mesh, and use the learned feature combination to drive the detail transfer.