Skip navigation
Please use this identifier to cite or link to this item:
Title: Learning and Deploying Local Features
Authors: Zhang, Linguang
Advisors: Rusinkiewicz, Szymon
Contributors: Computer Science Department
Keywords: Deep Learning
Local Features
Subjects: Computer science
Issue Date: 2019
Publisher: Princeton, NJ : Princeton University
Abstract: Many computer vision applications including image matching, image-based reconstruction and localization rely on extracting and matching robust local features. A typical local feature pipeline first detects repeatable keypoints in the image (i.e., keypoint detector), and then computes a short vector to uniquely describe each keypoint (i.e., feature descriptor). Both the keypoint detector and the feature descriptor are conventionally hand-crafted based on what is intuitive to the designer. For example, corners or blobs are popular choices of keypoints, and the image gradient is a useful clue for descriptors. However, it is often difficult to define these principles to accommodate various applications. In this thesis, we study data-driven approaches which can more easily tailor the local feature pipeline for target applications. We start with a mobile robotics application that leverages local features extracted from ground texture images to achieve high-precision global localization. The second part of the thesis addresses the problem that existing keypoint detectors that are optimized for natural images suffer from sub-optimal performance on texture images. We therefore learn a keypoint detector specifically for each type of texture using a deep neural network. Our detector automatically learns to identify keypoints that are distinctive in the target texture rather than relying on a set of pre-defined rules. Finally, we focus on a non-parametric approach for learning feature descriptors. Many well-performing local feature descriptors are trained using a triplet loss that includes a tunable margin, which limits its ability to generalize to other types of data and problems. We propose to replace the hard margin with a soft margin that self-tunes as learning progresses. To summarize, we first demonstrate through a novel visual-based localization system where a customized local feature pipeline is critical. Then, we tackle both the keypoint detector and the feature descriptor with generalizable data-driven approaches.
Alternate format: The Mudd Manuscript Library retains one bound copy of each dissertation. Search for these copies in the library's main catalog:
Type of Material: Academic dissertations (Ph.D.)
Language: en
Appears in Collections:Computer Science

Files in This Item:
File Description SizeFormat 
Zhang_princeton_0181D_12958.pdf181.8 MBAdobe PDFView/Download

Items in Dataspace are protected by copyright, with all rights reserved, unless otherwise indicated.