An Efficient Metadata Distribution Policy for Cluster File Systems
Jin Xiong, Rongfeng Tang, Sining Wu, Dan Meng, Ninghui Sun
IEEE International Conference on Cluster Computing (Cluster 2005)
Boston, Massachusetts, USA, September 27 - 30, 2005
Abstract
How to distribute the items in the file system hierarchy across a group of metadata servers is an important issue that determines the holistic metadata processing performance (HMPP) of a cluster file system which manages its metadata by a group of metadata servers. The HMPP is affected by two factors: balance degree of metadata distribution and number of branch points. Two types of well-used metadata distribution policies are the dynamic subtree policy and the random policy. Both of them emphasize one factor and neglect the other factor. As a result, their HMPP is low. In order to make good use of processing capacity of all metadata servers, we present a novel metadata distribution policy, called dynamic dir-grain (DDG) policy, which takes both factors into account. Our performance results show that this policy is potentially more efficient than the other two types of policies under real environments, as well as the conditions of creation or removal of a large hierarchy. Key words: cluster file system, metadata distribution, metadata processing performance