Detecting gradual changes from data stream using MDL-change statistics

K Yamanishi, K Miyaguchi - … Conference on Big Data (Big Data …, 2016 - ieeexplore.ieee.org
2016 IEEE International Conference on Big Data (Big Data), 2016ieeexplore.ieee.org
In this paper we propose a novel methodology of sequential change detection using the
minimum description length (MDL)-change statistics. We first introduce the MDL-change
statistics as the difference between the code-lengths with change and that without change.
We give a theoretical justification for its use in the scenario of hypothesis testing. In it we
evaluate the error probabilities for the MDL-change detection to relate them to the
information-theoretic complexities of the probabilistic models and their discrepancy …
In this paper we propose a novel methodology of sequential change detection using the minimum description length (MDL)-change statistics. We first introduce the MDL-change statistics as the difference between the code-lengths with change and that without change. We give a theoretical justification for its use in the scenario of hypothesis testing. In it we evaluate the error probabilities for the MDL-change detection to relate them to the information-theoretic complexities of the probabilistic models and their discrepancy measure. We then convert the MDL-change statistics into the sequential change detection algorithm. It is designed to detect gradual changes as well as abrupt changes from big stream data. We empirically demonstrate the effectiveness of the proposed method by showing that it performs better than existing algorithms for synthetic data. We also show its validity through real problems such as SQL injection detection and failure symptom detection.
ieeexplore.ieee.org
以上显示的是最相近的搜索结果。 查看全部搜索结果