Journal Home Online First Current Issue Archive For Authors Journal Information 中文版

2020, Volume 21, Issue 7

Abstract

Keywords

Related Research

Back to Top

Frontiers of Information Technology & Electronic Engineering >> 2020, Volume 21, Issue 7 doi: 10.1631/FITEE.1900121

MDLB: a metadata dynamic load balancing mechanism based on reinforcement learning

Affiliation(s): National Digital Switching System Engineering & Technological R&D Center, Zhengzhou 450002, China; School of Computer Science, Fudan University, Shanghai 200433, China; Data Arena Institute, Fudan University, Shanghai 200433, China; less

Received: 2019-03-01 Accepted: 2020-07-10 Available online: 2020-07-10
HTML26 PDF 41 Collect 0

Next Previous

Abstract

With the growing amount of information and data, s have been widely used in many applications, including the Google File System, Amazon S3, Hadoop Distributed File System, and Ceph, in which load balancing of plays an important role in improving the input/output performance of the entire system. Unbalanced load on the server leads to a serious bottleneck problem for system performance. However, most existing load balancing strategies, which are based on subtree segmentation or hashing, lack good dynamics and adaptability. In this study, we propose a (MDLB) mechanism based on (RL). We learn that the algorithm and our RL-based strategy consist of three modules, i.e., the policy selection network, load balancing network, and parameter update network. Experimental results show that the proposed MDLB algorithm can adjust the load dynamically according to the performance of the servers, and that it has good adaptability in the case of sudden change of data volume.

Related Research