Multi-FPGA systems have been receiving a lot of attention as a low cost and energy efficient system for Multi-access Edge Computing (MEC). For such purpose, a bare-metal multi-FPGA system called FiC (Flow-in-Cloud) is under development. In this paper, we introduce the FiC multi FPGA cluster which is applied partial reconfiguration (PR) FPGA design flow to support online user defined accelerator replacement while executing FPGA interconnection network and its low-level multiple FPGA management software called remote PR manager. With the remote PR manager, the user can define the FiC FPGA cluster setup by JSON and control the cluster from user application with the cooperation of simple cluster management tool / library called ficmgr on the client host and REST API service provider called ficwww on Raspberry Pi 3 (RPi3) on each node. According to the evaluation results with a prototype FiC FPGA cluster system with 12 nodes, using with online application replacement by PR and on-the-fly FPGA bitstream compression, the time for FPGA bitstream distribution was reduced to 1/17 and the total cluster setup time was reduced by 21∼57% than compared to cluster setup with full configuration FPGA bitstream.

Network throughput has become an important issue for big-data analysis on Warehouse-Scale Computing (WSC) systems. It has been reported that randomly-connected inter-switch networks can enlarge the network throughput. For irregular networks, a multi-path routing method called k-shortest path routing is conventionally utilized. However, it cannot efficiently exploit longer-than-shortest paths that would be detour paths to avoid bottlenecks. In this work, a novel routing method called k-optimized path routing to achieve high throughput is proposed for irregular networks. We introduce a heuristic to select detour paths that can avoid bottlenecks in the network to improve the average-case network throughput. Experimental results by network simulation show that the proposed k-optimized path routing can improve the saturation throughput by up to 18.2% compared to the conventional k-shortest path routing. Moreover, it can reduce the computation time required for optimization to 1/2760 at a minimum compared to our previously proposed method.

Connecting a large number of servers with high bandwidth links is one of the most crucial and challenging tasks that the Data Center Network (DCN) must fulfill. DCN faces a lot of difficulties like the effective exploitation of DC components that, if highlighted, can aid in constructing high performance, scalable, reliable, and cost-effective DCN. In this paper, we investigate the server-centric structure. We observe that current DCs use servers that mostly come with dual ports. Effective exploitation of the ports of interest for building the topology structure can help in realizing the potentialities of reducing expensive topology. Our new network topology, named “Parallel Cubes” (PCube), is a duplicate defined structure that utilizes the ports in the servers and mini-switches to form a highly effective, scalable, and efficient network structure. P-Cube provides high performance in network latency and throughput and fault tolerance. Additionally, P-Cube is highly scalable to encompass hundreds of thousands of servers with a low stable diameter and high bisection width. We design a routing algorithm for P-Cube network that utilizes the P-Cube structure to strike a balance among the numerous links in the network. Finally, numerical results are provided to show that our proposed topology is a promising structure as it outperforms other topologies and it is superior to Fat-tree, BCube and DCell by approximately 24%, 16%, 8% respectively in terms of network throughput and latency. Moreover, P-Cube extremely outperforms Fat-tree, and BCube structures in terms of total cost, complexity of cabling and power consumption.

Simultaneous multithreading (SMT) technology improves CPU throughput, but also causes unpredictable performance fluctuations for co-running workloads. Although recent major SMT processors have adopted some techniques to promote hardware support for quality-of-service (QoS), achieving both precise performance guarantees and high throughput on SMT architectures is still a challenging open problem. In this paper, we demonstrate through some comprehensive investigations on a cycle-accurate simulator that not only almost all in-core resources suffer from severe contention as workloads vary but also there is a non-linear relationship between performance and available quotas of resources. We consider these observations as the fundamental reason leading to the challenging problem above. Thus, we introduce QoSMT, a novel hardware scheme that leverages a closed-loop controlling mechanism consisting of detection, prediction and adjustment to enforce precise performance guarantees for specific targets, e.g. achieving 85%, 90% or 95% of the performance of a workload running alone respectively. We implement a prototype on GEM5 simulator. Experimental results show that the average control error is only 1.4%, 0.5% and 3.6%.

For cloud data center, Virtual Machine (VM) consolidation is an effective way to save energy and improve efficiency. However, inappropriate consolidation of VMs, especially aggressive consolidation, can lead to performance problems, and even more serious Service Level Agreement (SLA) violations. Therefore, it is very important to solve the tradeoff between reduction in energy use and reduction of SLA violation level. In this paper, we propose two Host State Detection algorithms and an improved VM placement algorithm based on our proposed Host State Binary Decision Tree Prediction model for SLA-aware and energy-efficient consolidation of VMs in cloud data centers. We propose two formulas of conditions for host state estimate, and our model uses them to build a Binary Decision Tree manually for host state detection. We extend Cloudsim simulator to evaluate our algorithms by using PlanetLab workload and random workload. The experimental results show that our proposed model can significantly reduce SLA violation rates while keeping energy cost efficient, it can reduce the metric of SLAV by at most 98.12% and the metric of Energy by at most 33.96% for real world workload.

Cloud computing, which enables users to enjoy various Internet services provided by data centers (DCs) at anytime and anywhere, has attracted much attention. In cloud computing, however, service quality degrades with user distance from the DC, which is unfair. In this study, we propose a bandwidth allocation scheme based on collectable information to improve fairness and link utilization in DC networks. We have confirmed the effectiveness of this approach through simulation evaluations.

Identifying IDC (Internet Data Center) IP addresses and analyzing the connection relationship of IDC could reflect the IDC network resource allocation and network layout which is helpful for IDC resource allocation optimization. Recent research mainly focuses on minimizing electricity consumption and optimizing network resource allocation based on IDC traffic behavior analysis. However, the lack of network-wide IP information from network operators has led to problems like management difficulties and unbalanced resource allocation of IDC, which are still unsolved today. In this paper, we propose a method for the IP identification and connection relationship analysis of IDC based on the flow connection behavior analysis. In our method, the frequent IP are extracted and aggregated in backbone communication network based on the traffic characteristics of IDC. After that, the connection graph of frequent IP (CGFIP) are built by analyzing the behavior of the users who visit the IDC servers, and IDC IP blocks are thus identified using CGFIP. Furthermore, the connection behavior characteristics of IDC are analyzed based on the connection graphs of IDC (CGIDC). Our findings show that the method can accurately identify the IDC IP addresses and is also capable of reflecting the relationships among IDCs effectively.

In data center networks, Quantized Congestion Notification (QCN) has been ratified as the standard congestion management mechanism in the link layer. Since QCN nonlinearly switches between the rate increase and decrease stages, it is very difficult to understand QCN in depth and provide theoretical guidelines on setting the buffer size of the QCN switch. This paper gives an explicit formula for the boundary of buffer occupancy of the QCN switch. Specifically, based on the fluid model of QCN, we first derive the uniformly asymptotic stability condition of the QCN system. Then, under the condition that QCN is uniformly asymptotically stable, we analyze the buffer occupancy of the QCN switch theoretically and show that the classic rule-of-thumb for buffer sizing is not suitable for QCN. Finally, simulations validate the accuracy of our theoretical results.

Query response times are critical for cluster computing applications in data centers. In this letter, we argue that to optimize the network performance, we should consider the latency of the flows suffered loss, which are called tardy flows. We propose two tardy flow scheduling algorithms and show that our work offers significant performance gains through performance analysis and simulations.

Quantized congestion notification (QCN), discussed in IEEE 802.1Qau, is one of the most promising Layer 2 congestion control methods for data center networks. Because data center networks have fundamentally symmetric structures and links are designed to have high link utilization, data center flows often pass through multiple bottleneck links. QCN reduces its transmission rate in a probabilistic manner with each congestion notification feedback reception, which might cause excessive regulation of the transmission rate in a multiple-bottleneck case because each bottleneck causes congestion feedbacks. We have already proposed QCN with bottleneck selection (QCN/BS) for multicast communications in data center networks. Although QCN/BS was originally proposed for multicast communications, it can also be applied to unicast communications with multiple bottleneck points. QCN/BS calculates the congestion level for each switch based on feedback from the switch and adjusts its transmission rate to the worst congestion level. In this paper, we preliminarily evaluate QCN/BS in unicast communications with multiple tandem bottleneck points. Our preliminary evaluation reveals that QCN/BS can resolve the excessive rate regulation problem of QCN but has new fairness problems for long-hop flows. To resolve this, we propose a new algorithm that integrates QCN/BS and our already proposed Adaptive BC_LIMIT. In Adaptive BC_LIMIT, the opportunities for rate increase are almost the same for all flows even if their transmission rates differ, enabling an accelerated convergence of fair rate allocation among flows sharing a bottleneck link. The integrated algorithm is the first congestion control mechanism that takes into account unicast flows passing through multiple tandem bottleneck points based on QCN. Furthermore, it does not require any modifications of switches used in QCN. Our simulation results show that our proposed integration of QCN/BS and Adaptive BC_LIMIT significantly mitigates the fairness problem for unicast communications with multiple bottleneck points in data center networks.

Cloud storage has become popular and is being used to hold important data. As a result, availability to become important; cloud storage providers should allow users to upload or download data even if some part of the system has failed. In this paper, we discuss distributed cloud storage that is robust against failures. In distributed cloud storage, multiple replicas of each data chunk are stored in the virtual storage at geographically different locations. Thus, even if one of the virtual storage systems becomes unavailable, users can access the data chunk from another virtual storage system. In distributed cloud storage, the placement of the virtual storage system is important; if the placement of the virtual cloud storage system means that a large number of virtual storages are possible could become unavailable from a failure, a large number of replicas of each data chunk should be prepared to maintain availability. In this paper, we propose a virtual storage placement method that assures availability with a small number of replicas. We evaluated our method by comparing it with three other methods. The evaluation shows that our method can maintain availability while requiring only with 60% of the network costs required by the compared methods.

In this paper, we propose a workload assignment policy for reducing power consumption by air conditioners in data centers. In the proposed policy, to reduce the air conditioner power consumption by raising the temperature set points of the air conditioners, the temperatures of all server back-planes are equalized by moving workload from the servers with the highest temperatures to the servers with the lowest temperatures. To evaluate the proposed policy, we use a computational fluid dynamics simulator for obtaining airflow and air temperature in data centers, and an air conditioner model based on experimental results from actual data center. Through evaluation, we show that the air conditioners' power consumption is reduced by 10.4% in a conventional data center. In addition, in a tandem data center proposed in our research group, the air conditioners' power consumption is reduced by 53%, and the total power consumption of the whole data center is exhibited to be reduced by 23% by reusing the exhaust heat from the servers.

Big data analysis and a data storing applications require a huge volume of storage and a high I/O performance. Applications can achieve high level of performance and cost efficiency by exploiting the high I/O performance of direct attached storages (DAS) such as internal HDDs. With the size of stored data ever increasing, it will be difficult to replace servers since internal HDDs contain huge amounts of data. Generally, the data is copied via Ethernet when transferring the data from the internal HDDs to the new server. However, the amount of data will continue to rapidly increase, and thus, it will be hard to make these types of transfers through the Ethernet since it will take a long time. A storage area network such as iSCSI can be used to avoid this problem because the data can be shared with the servers. However, this decreases the level of performance and increases the costs. Improving the flexibility without incurring I/O performance degradation is required in order to improve the DAS architecture. In response to this issue, we propose FlexDAS, which improves the flexibility of direct attached storage by using a disk area network (DAN) without degradation the I/O performance. A resource manager connects or disconnects the computation nodes to the HDDs via the FlexDAS switch, which supports the SAS or SATA protocols. This function enables for the servers to be replaced in a short period of time. We developed a prototype FlexDAS switch and quantitatively evaluated the architecture. Results show that the FlexDAS switch can disconnect and connect the HDD to the server in just 1.16 seconds. We also confirmed that the FlexDAS improves the performance of the data intensive applications by up to 2.84 times compared with the iSCSI.

Address configuration is a key problem in data center networks. The core issue of automatic address configuration is assigning logical addresses to the physical network according to a blueprint, namely logical-to-device ID mapping, which can be formulated as a graph isomorphic problem and is hard. Recently years, some work has been proposed for this problem, such as DAC and ETAC. DAC adopts a sub-graph isomorphic algorithm. By leveraging the structure characteristic of data center network, DAC can finish the mapping process quickly when there is no malfunction. However, in the presence of any malfunctions, DAC need human effort to correct these malfunctions and thus is time-consuming. ETAC improves on DAC and can finish mapping even in the presence of malfunctions. However, ETAC also suffers from some robustness and efficiency problems. In this paper, we present GA-MAP, a data center networks address mapping algorithm based on genetic algorithm. By intelligently leveraging the structure characteristic of data center networks and the global search characteristic of genetic algorithm, GA-MAP can solve the address mapping problem quickly. Moreover, GA-MAP can even finish address mapping when physical network involved in malfunctions, making it more robust than ETAC. We evaluate GA-MAP via extensive simulation in several of aspects, including computation time, error-tolerance, convergence characteristic and the influence of population size. The simulation results demonstrate that GA-MAP is effective for data center addresses mapping.

In a multi-tenant data center, nodes and links of tenants' virtual networks (VNs) share a single component of the physical substrate network (SN). The failure of a single SN component can thereby cause the simultaneous failures of multiple nodes and links in a single VN; this complex of failures must significantly disrupt the services offered on the VN. In the present paper, we clarify how the fault tolerance of each VN is affected by a single SN failure, especially from the perspective of VN allocation in the SN. We propose a VN allocation model for multi-tenant data centers and formulate a problem that deals with the bandwidth loss in a single VN due a single SN failure. We conduct numerical simulations (with the setting that has 1.7×108bit/s bandwidth demand on each VN, (denoted by Ci)). When each node in each VN is scattered and mapped to an individual physical server, each VN can have the minimum bandwidth loss (5.3×102bit/s (3.0×10-6×Ci)) but the maximum required bandwidth between physical servers (1.0×109bit/s (5.7×Ci)). The balance between the bandwidth loss and the required physical resources can be optimized by assigning every four nodes of each VN to an individual physical server, meaning that we minimize the bandwidth loss without over-provisioning of core switches.

Router virtualization is becoming more common as a method that uses network (NW) equipment effectively and robustly similar to server virtualization. Edge routers, which are gateways of core NWs, should be virtualized because they have many functions and resources just as servers do. To virtualize edge routers, a metro NW, which is a wide area layer-2 NW connecting each user's residential gateway to edge routers, must trace dynamic edge router re-allocation by changing the route of each Ethernet flow. Therefore, we propose a scalable centralized control architecture of a virtual layer-2 switch on a metro NW to trace virtual router re-allocation and use metro NW equipment effectively. The proposed scalable control architecture improves the centralized route control performance by processing in parallel on a flow-by-flow basis taking into account route information even in the worst case where edge routers fail. In addition, the architecture can equalize the load among parallel processes dynamically by using two proposed load re-allocation methods to increase the route control performance stably while minimizing the amount of resources for the control. We evaluate the scalability of the proposed architecture through theoretical analysis and experiments on a prototype and show that the proposed architecture increases the number of flows accommodated in a metro NW. Moreover, we evaluate the load re-allocation methods through simulation and show that they can evenly distribute the load among parallel processes. Finally, we show that the proposed architecture can be applied to not only large-scale metro NWs but also to data center NWs, which have recently become an important type of large-scale layer-2 NW.

In this paper, we propose a novel Autonomous Decentralized Control (ADC) scheme for indirectly controlling a system performance variable of large-scale and wide-area networks. In a large-scale and wide-area network, since it is impractical for any one node to gather full information of the entire network, network control must be realized by inter-node collaboration using information local to each node. Several critical network problems (e.g., resource allocation) are often formulated by a system performance variable that is an amount to quantify system state. We solve such problems by designing an autonomous node action that indirectly controls, via the Markov Chain Monte Carlo method, the probability distribution of a system performance variable by using only local information. Analyses based on statistical mechanics confirm the effectiveness of the proposed node action. Moreover, the proposal is used to implement traffic-aware virtual machine placement control with load balancing in a data center network. Simulations confirm that it can control the system performance variable and is robust against system fluctuations. A comparison against a centralized control scheme verifies the superiority of the proposal.

Data center network is composed of high-speed Ethernet extended in a limited area of a data center building, so its RTT is extremely small of µsec order. In order to regulate data center network delay large part of which is queuing delay, QCN is proposed for layer 2 congestion control in IEEE 802.1Qau. QCN controls transmission rate of the sender by congestion feedback from a congested switch. QCN adopts probabilistic feedback transmission to reduce the control overhead. When the number of flows through a bottleneck link increases, some flows might receive no feedback even in congestion phase due to probabilistic feedback transmission. In this situation, queue length might be significantly fluctuated. In this paper, we propose a new delay-based congestion detection and control method. Our proposed delay-based congestion control is cooperated with the conventional QCN so as to detect and react congestion not detected by QCN.

Tenant network provisioning in multi-tenancy data centers is time-consuming and error-prone due to the need to configure network devices with hundreds of parameter values (e.g., VLAN ID, IP address) determined according to complicated operational rules. Past works have aimed to automate such operational rule-based provisioning processes by implementing data center-specific provisioning programs, but a crucial problem is the high cost of adapting the programs to suit multiple data centers. In this paper, we aim to solve this problem by enabling to describe the provisioning processing, which has been hard-coded programs in conventional approaches, in easy-to-edit “provisioning template” files. The key component of the provisioning template is the parameter decision rule, which is a declarative abstract representation of parameter dependency and parameter assignment. We design the provisioning template so that it can handle various configuration items while preserving its editability for tenant provisioning. We design and implement the provisioning platform, and the evaluation based on a production data center shows that the provisioning platform can adopt multiple data centers with a single program, leading to less development cost compared to past approaches (i.e., program development for each data center).

This paper covers new architectures, technologies, and performance benchmarking together with prospects for high productivity and high performance computing enabled by photonics. The exponential and sustained increases in computing and data center needs are driving the demands for exascale computing in the future. Power-efficient and parallel computing with balanced system design is essential for reaching that goal as should support ∼billion total concurrencies and ∼billion core interconnections with ∼exabyte/second bisection bandwidth. Photonic interconnects offer a disruptive technology solution that fundamentally changes the computing architectural design considerations. Optics provide ultra-high throughput, massive parallelism, minimal access latencies, and low power dissipation that remains independent of capacity and distance. In addition to the energy efficiency and many of the fundamental physical problems, optics will bring high productivity computing where programmers can ignore locality between billions of processors and memory where data resides. Repeaterless interconnection links across the entire computing system and all-to-all massively parallel interconnection switch will significantly transform not only the hardware aspects of computing but the way people program and harness the computing capability. This impacts programmability and productivity of computing. Benchmarking and optimization of the configuration of the computing system is very important. Practical and scalable deployment of photonic interconnected computing systems are likely to be aided by emergence of athermal silicon photonics and hybrid integration technologies.