Online Anomaly Prediction for Real-Time Stream Processing

Yuanqiang HUANG; Zhongzhi LUAN; Depei QIAN; Zhigao DU; Ting CHEN; Yuebin BAI

doi:10.1587/transcom.E95.B.2034

Online Anomaly Prediction for Real-Time Stream Processing

Yuanqiang HUANG, Zhongzhi LUAN, Depei QIAN, Zhigao DU, Ting CHEN, Yuebin BAI

Full Text Views

0

Share
Cite this

Summary :

With the consideration of real-time stream processing technology, it's important to develop high availability mechanism to guarantee stream-based application not interfered by faults caused by potential anomalies. In this paper, we present a novel online prediction technique for predicting some anomalies which may occur in the near future. Concretely, we first present a value prediction which combines the Hidden Markov Model and the Mixture of Expert Model to predict the values of feature metrics in the near future. Then we employ the Support Vector Machine to do anomaly identification, which is a procedure to identify the kind of anomaly that we are about to alarm. The purpose of our approach is to achieve a tradeoff between fault penalty and resource cost. The experiment results show that our approach is of high accuracy for common anomaly prediction and low runtime overhead.

Publication: IEICE TRANSACTIONS on Communications Vol.E95-B No.6 pp.2034-2042

Publication Date: 2012/06/01

Publicized

Online ISSN: 1745-1345

DOI: 10.1587/transcom.E95.B.2034

Type of Manuscript: PAPER

Category: Network Management/Operation

Cite this

Copy

Yuanqiang HUANG, Zhongzhi LUAN, Depei QIAN, Zhigao DU, Ting CHEN, Yuebin BAI, "Online Anomaly Prediction for Real-Time Stream Processing" in IEICE TRANSACTIONS on Communications, vol. E95-B, no. 6, pp. 2034-2042, June 2012, doi: 10.1587/transcom.E95.B.2034.
Abstract: With the consideration of real-time stream processing technology, it's important to develop high availability mechanism to guarantee stream-based application not interfered by faults caused by potential anomalies. In this paper, we present a novel online prediction technique for predicting some anomalies which may occur in the near future. Concretely, we first present a value prediction which combines the Hidden Markov Model and the Mixture of Expert Model to predict the values of feature metrics in the near future. Then we employ the Support Vector Machine to do anomaly identification, which is a procedure to identify the kind of anomaly that we are about to alarm. The purpose of our approach is to achieve a tradeoff between fault penalty and resource cost. The experiment results show that our approach is of high accuracy for common anomaly prediction and low runtime overhead.
URL: https://globals.ieice.org/en_transactions/communications/10.1587/transcom.E95.B.2034/_p

Copy

@ARTICLE{e95-b_6_2034,
author={Yuanqiang HUANG, Zhongzhi LUAN, Depei QIAN, Zhigao DU, Ting CHEN, Yuebin BAI, },
journal={IEICE TRANSACTIONS on Communications},
title={Online Anomaly Prediction for Real-Time Stream Processing},
year={2012},
volume={E95-B},
number={6},
pages={2034-2042},
abstract={With the consideration of real-time stream processing technology, it's important to develop high availability mechanism to guarantee stream-based application not interfered by faults caused by potential anomalies. In this paper, we present a novel online prediction technique for predicting some anomalies which may occur in the near future. Concretely, we first present a value prediction which combines the Hidden Markov Model and the Mixture of Expert Model to predict the values of feature metrics in the near future. Then we employ the Support Vector Machine to do anomaly identification, which is a procedure to identify the kind of anomaly that we are about to alarm. The purpose of our approach is to achieve a tradeoff between fault penalty and resource cost. The experiment results show that our approach is of high accuracy for common anomaly prediction and low runtime overhead.},
keywords={},
doi={10.1587/transcom.E95.B.2034},
ISSN={1745-1345},
month={June},}

Copy

TY - JOUR
TI - Online Anomaly Prediction for Real-Time Stream Processing
T2 - IEICE TRANSACTIONS on Communications
SP - 2034
EP - 2042
AU - Yuanqiang HUANG
AU - Zhongzhi LUAN
AU - Depei QIAN
AU - Zhigao DU
AU - Ting CHEN
AU - Yuebin BAI
PY - 2012
DO - 10.1587/transcom.E95.B.2034
JO - IEICE TRANSACTIONS on Communications
SN - 1745-1345
VL - E95-B
IS - 6
JA - IEICE TRANSACTIONS on Communications
Y1 - June 2012
AB - With the consideration of real-time stream processing technology, it's important to develop high availability mechanism to guarantee stream-based application not interfered by faults caused by potential anomalies. In this paper, we present a novel online prediction technique for predicting some anomalies which may occur in the near future. Concretely, we first present a value prediction which combines the Hidden Markov Model and the Mixture of Expert Model to predict the values of feature metrics in the near future. Then we employ the Support Vector Machine to do anomaly identification, which is a procedure to identify the kind of anomaly that we are about to alarm. The purpose of our approach is to achieve a tradeoff between fault penalty and resource cost. The experiment results show that our approach is of high accuracy for common anomaly prediction and low runtime overhead.
ER -