%0 Conference Proceedings
%T ODCP: Optimizing Data Caching and Placement in Distributed File System Using Erasure Coding
%+ School of Computer Science and Engineering [Beijing]
%+ State Key Laboratory of Mathematical Engineering and Advanced Computing
%+ Beijing University of Technology
%A Wu, Shuhan
%A Li, Yunchun
%A Yang, Hailong
%A Luan, Zerong
%A Li, Wei
%Z Part 9: Storage
%< avec comité de lecture
%( Lecture Notes in Computer Science
%B 17th IFIP International Conference on Network and Parallel Computing (NPC)
%C Zhengzhou, China
%Y Xin He
%Y En Shao
%Y Guangming Tan
%I Springer International Publishing
%3 Network and Parallel Computing
%V LNCS-12639
%P 452-464
%8 2020-09-28
%D 2020
%R 10.1007/978-3-030-79478-1_38
%K Distributed file system
%K Erasure coding
%K Decoding latency
%K Data placement strategy
%K Cache allocation strategy
%Z Computer Science [cs]Conference papers
%X Many current distributed file systems use erasure-coding based data redundancy techniques to improve the reliability of data storage. Such techniques can significantly improve the effective storage utilization. However, there are several drawbacks to the above techniques. Firstly, they introduce non-negligible computation overhead for decoding. Secondly, traditional data caching and placement strategies become less effective in such cases. To solve the above drawbacks, this paper proposes a new data cache allocation mechanism based on simulated annealing and a new data placement strategy based on convex optimization, which effectively reduces data block transmission delay and decoding delay. We have implemented the proposed data placement strategy in the real-world distributed file system Alluxio, and evaluated the performance of our strategy. Experiment results show that our strategy can significantly reduce the file read delay compared to traditional data placement strategies.
%G English
%Z TC 10
%Z WG 10.3
%2 https://inria.hal.science/hal-03768742/document
%2 https://inria.hal.science/hal-03768742/file/511910_1_En_38_Chapter.pdf
%L hal-03768742
%U https://inria.hal.science/hal-03768742
%~ IFIP-LNCS
%~ IFIP
%~ IFIP-TC
%~ IFIP-TC10
%~ IFIP-NPC
%~ IFIP-WG10-3
%~ IFIP-LNCS-12639