Mimir+: An Optimized Framework of MapReduce on Heterogeneous High-Performance Computing System - Network and Parallel Computing Access content directly
Conference Papers Year : 2018

Mimir+: An Optimized Framework of MapReduce on Heterogeneous High-Performance Computing System

Abstract

In this paper, we present an optimized data processing framework: Mimir+. Mimir+ is an implementation of MapReduce over MPI. In order to take full advantage of heterogeneous computing system, we propose the concept of Pre-acceleration to reconstruct a heterogeneous workflow and implement the interfaces of GPU so that Mimir+ can facilitate data processing through reasonable tasks and data scheduling between CPU and GPU. We evaluate Mimir+ via two benchmarks (i.e. the WordCount and large-scale matrix multiplication) on the Tianhe-2 supercomputing system. Experimental results demonstrate that Mimir+ achieves excellent acceleration effect compared with original Mimir.
Fichier principal
Vignette du fichier
477597_1_En_18_Chapter.pdf (460.69 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

hal-02279555 , version 1 (05-09-2019)

Licence

Attribution

Identifiers

Cite

Nan Hu, Zhiguang Chen, Yunfei Du, Yutong Lu. Mimir+: An Optimized Framework of MapReduce on Heterogeneous High-Performance Computing System. 15th IFIP International Conference on Network and Parallel Computing (NPC), Nov 2018, Muroran, Japan. pp.164-168, ⟨10.1007/978-3-030-05677-3_18⟩. ⟨hal-02279555⟩
39 View
111 Download

Altmetric

Share

Gmail Facebook X LinkedIn More