Clustered hierarchical anomaly and outlier detection algorithms
N Ishaq, TJ Howard, NM Daniels - 2021 IEEE International …, 2021 - ieeexplore.ieee.org
N Ishaq, TJ Howard, NM Daniels
2021 IEEE International Conference on Big Data (Big Data), 2021•ieeexplore.ieee.orgAnomaly and outlier detection is a long-standing problem in machine learning. In some
cases, anomaly detection is easy, such as when data are drawn from well-characterized
distributions such as the Gaussian. However, when data occupy high-dimensional spaces,
anomaly detection becomes more difficult. We present CLAM (Clustered Learning of
Approximate Manifolds), a manifold mapping technique in any metric space. CLAM begins
with a fast hierarchical clustering technique and then induces a graph from the cluster tree …
cases, anomaly detection is easy, such as when data are drawn from well-characterized
distributions such as the Gaussian. However, when data occupy high-dimensional spaces,
anomaly detection becomes more difficult. We present CLAM (Clustered Learning of
Approximate Manifolds), a manifold mapping technique in any metric space. CLAM begins
with a fast hierarchical clustering technique and then induces a graph from the cluster tree …
Anomaly and outlier detection is a long-standing problem in machine learning. In some cases, anomaly detection is easy, such as when data are drawn from well-characterized distributions such as the Gaussian. However, when data occupy high-dimensional spaces, anomaly detection becomes more difficult. We present CLAM (Clustered Learning of Approximate Manifolds), a manifold mapping technique in any metric space. CLAM begins with a fast hierarchical clustering technique and then induces a graph from the cluster tree, based on overlapping clusters as selected using several geometric and topological features. Using these graphs, we implement CHAODA (Clustered Hierarchical Anomaly and Outlier Detection Algorithms), exploring various properties of the graphs and their constituent clusters to find outliers. CHAODA employs a form of transfer learning based on a training set of datasets, and applies this knowledge to a separate test set of datasets of different cardinalities, dimensionalities, and domains. On 24 publicly available datasets, we compare CHAODA (by measure of ROC AUC) to a variety of state-of-the-art unsupervised anomaly-detection algorithms. Six of the datasets are used for training. CHAODA outperforms other approaches on 16 of the remaining 18 datasets. CLAM and CHAODA scale to large, high-dimensional "big data" anomaly-detection problems, and generalize across datasets and distance functions. Source code to CLAM and CHAODA are freely available on GitHub 1 .
ieeexplore.ieee.org
以上显示的是最相近的搜索结果。 查看全部搜索结果