A Fine-Grained Performance Bottleneck Analysis Method for HDFS - Network and Parallel Computing
Conference Papers Year : 2018

A Fine-Grained Performance Bottleneck Analysis Method for HDFS

Abstract

The performance issue of HDFS has always been a great concern due to its widely adoption in both production and research environments. However, a fine-grained performance analysis tool is missing to effectively identify the bottlenecks as well as to provide useful guidance for performance optimization. In this paper, we propose a fine-grained performance bottleneck analysis tool, which extends HTrace with fine-grained instrumentation points that are missing in Hadoop official distribution. In addition, we propose an effective trace merging method that improves the understandability of our analysis. We analyze the performance of HDFS under different kinds of workloads and get undiscovered insights.

Domains

Computer Science [cs]
Fichier principal
Vignette du fichier
477597_1_En_17_Chapter.pdf (263.53 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Hal Ifip  : Connect in order to contact the contributor

https://inria.hal.science/hal-02279548

Submitted on : Thursday, September 5, 2019-1:30:57 PM

Last modification on : Wednesday, November 3, 2021-6:39:10 AM

Long-term archiving on : Thursday, February 6, 2020-2:44:56 AM

Download

Dates and versions

hal-02279548 , version 1 (05-09-2019)

Licence

Attribution

Identifiers

Cite

Yi Liu, Yunchun Li, Honggang Zhou, Jingyi Zhang, Hailong Yang, et al.. A Fine-Grained Performance Bottleneck Analysis Method for HDFS. 15th IFIP International Conference on Network and Parallel Computing (NPC), Nov 2018, Muroran, Japan. pp.159-163, ⟨10.1007/978-3-030-05677-3_17⟩. ⟨hal-02279548⟩

Export

BibTeX XML-TEI Dublin Core DC Terms EndNote DataCite

Collections

IFIP-LNCS IFIP IFIP-TC IFIP-TC10 IFIP-NPC IFIP-WG10-3 IFIP-LNCS-11276
67 View
205 Download

Altmetric

Share

More