Learning in the Wild: Challenges With Out-of-Distribution Data

Abstract

A signature of human intelligence is the ability to recognize and learn from unfamiliar data with only a limited amount of exposure. Most artificial intelligence models, on the other hand, lack both the awareness of and adaptability to data outside their training distributions. When encountering out-of-distribution data in the real world, models often produce undesirable, sometimes catastrophic results without realizing their failure. In this two-part thesis, we discuss several challenges that modern intelligent systems must face in order to become more (1) aware of and (2) adaptable to out-of-distribution data, through the lens of computer vision.

In Part I, we study the awareness of out-of-distribution data by examining the problem of out-of-distribution (OOD) detection, which aims to identify inputs on which a given image classification model should not be trusted to make a prediction. While OOD detection is widely studied, current research only considers images from classes that did not appear in the model’s training data, while ignoring the vast spectrum of other possible data the model may encounter in the wild. To address the shortcomings of this formulation, we propose to evaluate OOD detection from the model's perspective to seamlessly incorporate all out-of-distribution data. Our empirical analysis reveals the simplest OOD detection baseline outperforms all prior state-of-the-art OOD detection methods under the new perspective. This suggests the time is ripe to rethink the task of OOD detection and to adopt a generalized framework which encompasses a broader range of real-world situations.

In Part II, we study the adaptability to out-of-distribution data by examining problem of Image Colorization on out-of-distribution archival photos. Our work is part of a comparative photography effort to document glacial recession and climate change over the past century, through reproducing archival photographs of alpine landscapes in the Patagonia region. Concretely, we aim to restore colors in the archival photographs to establish clear and precise visual comparisons that highlight the reduction in glacial volume. However, as archival photographs are out-of-distribution for many state-of-the-art deep-learning-based colorization methods. models often demonstrate poor performance in generating realistic colorization. To overcome this limitation, we propose a novel method based on diffusion probabilistic models to augment the colorization process with minimal amount of user guidance. Our restoration results are judged to the most plausible among representative prior approaches approaches by 44% of subjects in a 116-person study.