Improved Resilience through Extended KVS-Based Messaging System

Masafumi KINOSHITA; Osamu TAKADA; Izumi MIZUTANI; Takafumi KOIKE; Kenji LEIBNITZ; Masayuki MURATA

doi:10.1587/transinf.2014NTP0015

Improved Resilience through Extended KVS-Based Messaging System

Masafumi KINOSHITA, Osamu TAKADA, Izumi MIZUTANI, Takafumi KOIKE, Kenji LEIBNITZ, Masayuki MURATA

Full Text Views

0

Share
Cite this

Summary :

In the big data era, messaging systems are required to process large volumes of message traffic with high scalability and availability. However, conventional systems have two issues regarding availability. The first issue is that failover processing itself has a risk of failure. The second issue is to find a trade-off between consistency and availability. We propose a resilient messaging system based on a distributed in-memory key-value store (KVS). Its servers are interconnected with each other and messages are distributed to multiple servers in normal processing state. This architecture can continue messaging services wherever in the messaging system server/process failures occur without using failover processing. Furthermore, we propose two methods for improved resilience: the round-robin method with a slowdown KVS exclusion and the two logical KVS counter-rotating rings to provide short-term-availability in the messaging system. Evaluation results demonstrate that the proposed system can continue service without failover processing. Compared with the conventional method, our proposed distribution method reduced 92% of error responses to clients caused by server failures.

Publication: IEICE TRANSACTIONS on Information Vol.E98-D No.3 pp.578-587

Publication Date: 2015/03/01

Publicized: 2014/12/11

Online ISSN: 1745-1361

DOI: 10.1587/transinf.2014NTP0015

Type of Manuscript: Special Section PAPER (Special Section on the Architectures, Protocols, and Applications for the Future Internet)

Category: Internet Operation and Management

Authors

Masafumi KINOSHITA
  Hitachi Ltd.,Osaka University
Osamu TAKADA
  Hitachi Ltd.
Izumi MIZUTANI
  Hitachi Ltd.
Takafumi KOIKE
  Hitachi Ltd.
Kenji LEIBNITZ
  Center of Information and Neural Networks (CiNet), National Institute of Information and Communications Technology, and Osaka University
Masayuki MURATA
  Osaka University

Keyword

KVS (key-value store), in-memory data grid, messaging queue, distribution system

Cite this

Copy

Masafumi KINOSHITA, Osamu TAKADA, Izumi MIZUTANI, Takafumi KOIKE, Kenji LEIBNITZ, Masayuki MURATA, "Improved Resilience through Extended KVS-Based Messaging System" in IEICE TRANSACTIONS on Information, vol. E98-D, no. 3, pp. 578-587, March 2015, doi: 10.1587/transinf.2014NTP0015.
Abstract: In the big data era, messaging systems are required to process large volumes of message traffic with high scalability and availability. However, conventional systems have two issues regarding availability. The first issue is that failover processing itself has a risk of failure. The second issue is to find a trade-off between consistency and availability. We propose a resilient messaging system based on a distributed in-memory key-value store (KVS). Its servers are interconnected with each other and messages are distributed to multiple servers in normal processing state. This architecture can continue messaging services wherever in the messaging system server/process failures occur without using failover processing. Furthermore, we propose two methods for improved resilience: the round-robin method with a slowdown KVS exclusion and the two logical KVS counter-rotating rings to provide short-term-availability in the messaging system. Evaluation results demonstrate that the proposed system can continue service without failover processing. Compared with the conventional method, our proposed distribution method reduced 92% of error responses to clients caused by server failures.
URL: https://globals.ieice.org/en_transactions/information/10.1587/transinf.2014NTP0015/_p

Copy

@ARTICLE{e98-d_3_578,
author={Masafumi KINOSHITA, Osamu TAKADA, Izumi MIZUTANI, Takafumi KOIKE, Kenji LEIBNITZ, Masayuki MURATA, },
journal={IEICE TRANSACTIONS on Information},
title={Improved Resilience through Extended KVS-Based Messaging System},
year={2015},
volume={E98-D},
number={3},
pages={578-587},
abstract={In the big data era, messaging systems are required to process large volumes of message traffic with high scalability and availability. However, conventional systems have two issues regarding availability. The first issue is that failover processing itself has a risk of failure. The second issue is to find a trade-off between consistency and availability. We propose a resilient messaging system based on a distributed in-memory key-value store (KVS). Its servers are interconnected with each other and messages are distributed to multiple servers in normal processing state. This architecture can continue messaging services wherever in the messaging system server/process failures occur without using failover processing. Furthermore, we propose two methods for improved resilience: the round-robin method with a slowdown KVS exclusion and the two logical KVS counter-rotating rings to provide short-term-availability in the messaging system. Evaluation results demonstrate that the proposed system can continue service without failover processing. Compared with the conventional method, our proposed distribution method reduced 92% of error responses to clients caused by server failures.},
keywords={},
doi={10.1587/transinf.2014NTP0015},
ISSN={1745-1361},
month={March},}

Copy

TY - JOUR
TI - Improved Resilience through Extended KVS-Based Messaging System
T2 - IEICE TRANSACTIONS on Information
SP - 578
EP - 587
AU - Masafumi KINOSHITA
AU - Osamu TAKADA
AU - Izumi MIZUTANI
AU - Takafumi KOIKE
AU - Kenji LEIBNITZ
AU - Masayuki MURATA
PY - 2015
DO - 10.1587/transinf.2014NTP0015
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E98-D
IS - 3
JA - IEICE TRANSACTIONS on Information
Y1 - March 2015
AB - In the big data era, messaging systems are required to process large volumes of message traffic with high scalability and availability. However, conventional systems have two issues regarding availability. The first issue is that failover processing itself has a risk of failure. The second issue is to find a trade-off between consistency and availability. We propose a resilient messaging system based on a distributed in-memory key-value store (KVS). Its servers are interconnected with each other and messages are distributed to multiple servers in normal processing state. This architecture can continue messaging services wherever in the messaging system server/process failures occur without using failover processing. Furthermore, we propose two methods for improved resilience: the round-robin method with a slowdown KVS exclusion and the two logical KVS counter-rotating rings to provide short-term-availability in the messaging system. Evaluation results demonstrate that the proposed system can continue service without failover processing. Compared with the conventional method, our proposed distribution method reduced 92% of error responses to clients caused by server failures.
ER -