Transferring the Contamination Factor between Anomaly Detection Domains by Shape Similarity
Lorenzo Perini, Vincent Vercruyssen, Jesse Davis
[AAAI-22] Main Track
Abstract:
Anomaly detection attempts to find examples in a dataset that do not conform to the expected behaviour. Algorithms for this task assign an anomaly score to each example representing its degree of anomalousness. Setting a threshold on the anomaly scores enables converting these scores into a discrete prediction for each example. Setting an appropriate threshold is challenging in practice since anomaly detection is often treated as an unsupervised problem. A common approach is to set the threshold based on the dataset's contamination factor, i.e., the proportion of anomalous examples in the data. While the contamination factor may be known based on domain knowledge, it is often necessary to estimate it by labeling data. However, many anomaly detection problems involve monitoring multiple related, yet slightly different entities (e.g., a fleet of machines). Then, estimating the contamination factor for each dataset separately by labeling data would be extremely time-consuming. Therefore, this paper introduces a method for transferring the known contamination factor from one dataset (the source domain) to a related dataset where it is unknown (the target domain). Our approach does not require labeled target data and is based on modelling the shape of the distribution of the anomaly scores in both domains. We theoretically analyze how our method behaves when the (biased) target domain anomaly score distribution converges to its true one. Empirically, we demonstrate the effectiveness of our method on real-world datasets showing that it outperforms several baselines.
Introduction Video
Sessions where this paper appears
-
Poster Session 4
Fri, February 25 5:00 PM - 6:45 PM (+00:00)
Blue 2
-
Poster Session 7
Sat, February 26 4:45 PM - 6:30 PM (+00:00)
Blue 2