CODiT: Conformal Out-of-Distribution Detection in Time-Series Data

Abstract

Machine learning models are prone to making incorrect predictions on inputs that are far from the training distribution. This hinders their deployment in safety-critical applications such as autonomous vehicles and healthcare. The detection of a shift from the training distribution of individual datapoints has gained attention. A number of techniques have been proposed for such out-of-distribution (OOD) detection. But in many applications, the inputs to a machine learning model form a temporal sequence. Existing techniques for OOD detection in time-series data either do not exploit temporal relationships in the sequence or do not provide any guarantees on detection. We propose using deviation from the in-distribution temporal equivariance as the non-conformity measure in conformal anomaly detection framework for OOD detection in time-series data. Computing independent predictions from multiple conformal detectors based on the proposed measure and combining these predictions by Fisher's method leads to the proposed detector CODiT with guarantees on false detection in time-series data. We illustrate the efficacy of CODiT by achieving stateof-the-art results on computer vision datasets in autonomous driving. We also show that CODiT can be used for OOD detection in non-vision datasets by performing experiments on the physiological GAIT sensory dataset. Code, data, and trained models are available at https://github.com/kaustubhsridhar/time-series-OOD.

Partially Supervised Classification for Industrial System using Deep Neural Network

“…Although recent researches start to focus on open set recognition (Kaur et al, 2022;Yoshihashi et al, 2019;Geng et al, 2020…”

“…To have a fair comparison, we also combine out-ofdistribution (OOD) with TCN. That is, any OOD time series that detected by (Kaur et al, 2022) will be assigned as new class (with class index k+1), while all the in-distribution (iD) will be assigned class labels by TCN.…”

“…In a classification model that recognizes time series samples collected from industrial system, the ability to handle samples from unseen classes is crucial (Dhamija, Günther, & Boult, 2018). There are several works that try to avoid false positives by rejecting the samples from unseen classes, that sometimes is also called out-of-distribution (OOD) detection (Kaur et al, 2022). Specifically, their target is to assign a negative label to any sample from the new classes (Dhamija et al, 2018;Perera & Patel, 2019), which can be understood as classifiers that can detect outliers outside of training set distribution.…”