Learning under Data Irregularity and Uncertainty

Abstract

Artificial Intelligent and Machine Learning (AI/ML) systems have been widely adopted with the increasing availability of data in a variety of applications such as computer vision, activity recognition, autonomous driving, healthcare, and many other science and engineering applications. Several challenges arise in translating them for effective and reliable decision making. Besides common challenges in analyzing sensor behavioral data, such as missing values and outliers, growing concerns of overfitting arise in widely used ML models, such as Deep Neural Networks (DNNs). This is exacerbated when considering their robustness and generalizability in real-world safety-critical applications such as autonomous driving and healthcare. It is therefore important to have accurate predictions as well as uncertainty estimates in the presence of data defects and anomalies. Bayesian learning is a promising field that works with probabilistic models explicitly considering uncertainty. In this field, models such as Gaussian Processes (GPs) and more recent Bayesian Neural Networks (BNNs), which define probability distributions over functions, are used to generalize from observed data while principally accounting for the uncertainty of its generalization. Moreover, such an accurate characterization of the data uncertainty allows us to perform accurate predictions in the face of irregularities. Such a system may allow us to benefit on other practical aspects of deploying AI/ML systems, such as enabling resource efficiency in the form of resources used to collect features, energy used for inference, and time wasted on experiments. Indeed, accurate model uncertainty estimates allow us to selectively deploy resources to cases where the uncertainty is high, while being efficient in cases with high certainty. In this work, we present a robust framework for enabling uncertainty-aware AI/ML through differentiable reparameterizations of discrete variational distributions. This enables expressive distributions to be used in tractably approximating the posterior model distribution, especially in BNNs. We apply this robust framework to develop various systems that handle missing values and outliers, quantify uncertainty, detect outliers, achieve resource efficient machine learning, and continually learn novel concepts from a stream of data.