File Declustering for Efficient Parallel I/O on Networks of Workstations

Abstract

This paper presents the issues involved in selecting an appropriate file declustering method to serve various access patterns on distributed parallel computing environments. A good file declustering method for multiple access patterns must (1) maximize the overall local access ratios, (2) minimize the performance variance between the access patterns, and (3) balance disk loads. The variable striping file declustering (VSFD) method is proposed for optimizing the file allocation in the I/O nodes with multiple access patterns and can increase the local access ratio. In addition, VSFD can provide a uniform and consistent performance across different access patterns, because it is able to induce the smallest variance in local disk access ratios for different access patterns.