A Load-Aware Data Placement Policy on Cluster File System - Network and Parallel Computing
Conference Papers Year : 2011

A Load-Aware Data Placement Policy on Cluster File System

Yu Wang
  • Function : Author
  • PersonId : 1017394
Jing Xing
  • Function : Author
  • PersonId : 1017395
Jin Xiong
  • Function : Author
  • PersonId : 1017396
Dan Meng
  • Function : Author
  • PersonId : 1017397

Abstract

In a large-scale cluster system with many applications running on it, cluster-wide I/O access workload disparity and disk saturation on only some storage servers have been the severe performance bottleneck that deteriorates the system I/O performance. As a result, the system response time will increase and the throughput of the system will decrease drastically. In this paper, we present a load-aware data placement policy that will distribute data across the storage servers based on the load of each server and automatically migrate data from heavily-loaded servers to lightly-loaded servers. This policy is adaptive and self-managing. It operates without any prior knowledge of application access workload characteristics or the capabilities of storage servers. It can make full use of the aggregate disk bandwidth of all storage servers efficiently. Performance evaluation shows that our policy will improve the aggregate I/O bandwidth by 10%-20% compared with random data placement policy especially under mixed workloads.
Fichier principal
Vignette du fichier
978-3-642-24403-2_2_Chapter.pdf (213.17 Ko) Télécharger le fichier
Origin Files produced by the author(s)
Loading...

Dates and versions

hal-01593007 , version 1 (25-09-2017)

Licence

Identifiers

Cite

Yu Wang, Jing Xing, Jin Xiong, Dan Meng. A Load-Aware Data Placement Policy on Cluster File System. 8th Network and Parallel Computing (NPC), Oct 2011, Changsha,, China. pp.17-31, ⟨10.1007/978-3-642-24403-2_2⟩. ⟨hal-01593007⟩
81 View
48 Download

Altmetric

Share

More