%0 Conference Proceedings
%T Resource-Aware Adaptive Scheduling for MapReduce Clusters
%+ Barcelona Supercomputing Center (BSC) and Technical University of Catalonia (UPC)
%+ IBM T.J. Watson Research Center
%A Polo, Jordà
%A Castillo, Claris
%A Carrera, David
%A Becerra, Yolanda
%A Whalley, Ian
%A Steinder, Malgorzata
%A Torres, Jordi
%A Ayguadé, Eduard
%Z Part 4: Green Computing and Resource Management
%< avec comité de lecture
%( Lecture Notes in Computer Science
%B 12th International Middleware Conference (MIDDLEWARE)
%C Lisbon, Portugal
%Y Fabio Kon
%Y Anne-Marie Kermarrec
%I Springer
%3 Middleware 2011
%V LNCS-7049
%P 187-207
%8 2011-12-12
%D 2011
%R 10.1007/978-3-642-25821-3_10
%K MapReduce
%K scheduling
%K resource-awareness
%K performance management
%Z Computer Science [cs]
%Z Computer Science [cs]/Networking and Internet Architecture [cs.NI]Conference papers
%X We present a resource-aware scheduling technique for MapReduce multi-job workloads that aims at improving resource utilization across machines while observing completion time goals. Existing MapReduce schedulers define a static number of slots to represent the capacity of a cluster, creating a fixed number of execution slots per machine. This abstraction works for homogeneous workloads, but fails to capture the different resource requirements of individual jobs in multi-user environments. Our technique leverages job profiling information to dynamically adjust the number of slots on each machine, as well as workload placement across them, to maximize the resource utilization of the cluster. In addition, our technique is guided by user-provided completion time goals for each job. Source code of our prototype is available at [1].
%G English
%Z TC 6
%Z WG 6.1
%2 https://inria.hal.science/hal-01597795/document
%2 https://inria.hal.science/hal-01597795/file/978-3-642-25821-3_10_Chapter.pdf
%L hal-01597795
%U https://inria.hal.science/hal-01597795
%~ IFIP-LNCS
%~ IFIP
%~ IFIP-TC
%~ IFIP-WG
%~ IFIP-TC6
%~ IFIP-WG6-1
%~ IFIP-MIDDLEWARE
%~ IFIP-LNCS-7049