Implementing Parallel Google Map-Reduce in Eden

Jost Berthold, Mischa Dieterle, Rita Loogen

Publikation: Bidrag til bog/antologi/rapportKonferencebidrag i proceedingsForskningpeer review

15 Citationer (Scopus)

Abstract

Recent publications have emphasised map-reduce as a general programming model (labelled Google map-reduce), and described existing high-performance implementations for large data sets. We present two parallel implementations for this Google map-reduce skeleton, one following earlier work, and one optimised version, in the parallel Haskell extension Eden. Eden's specific features, like lazy stream processing, dynamic reply channels, and nondeterministic stream merging, support the efficient implementation of the complex coordination structure of this skeleton. We compare the two implementations of the Google map-reduce skeleton in usage and performance, and deliver runtime analyses for example applications. Although very flexible, the Google map-reduce skeleton is often too general, and typical examples reveal a better runtime behaviour using alternative skeletons.
OriginalsprogEngelsk
TitelProceedings of the 15th International Euro-Par Conference on Parallel Processing
Antal sider13
ForlagSpringer
Publikationsdato2009
Sider990-1002
ISBN (Trykt)978-3-642-03868-6
DOI
StatusUdgivet - 2009
BegivenhedEuro-Par 2009 - Delft, Holland
Varighed: 25 aug. 200928 aug. 2009
Konferencens nummer: 15

Konference

KonferenceEuro-Par 2009
Nummer15
Land/OmrådeHolland
ByDelft
Periode25/08/200928/08/2009
NavnLecture notes in computer science
Vol/bind5704
ISSN0302-9743

Citationsformater