14
Map Reduce

MapReduce - Functional Programming

  • Upload
    tushar

  • View
    210

  • Download
    2

Embed Size (px)

DESCRIPTION

a brief introduction of Functional programming using Map Reduce algorithm.

Citation preview

Page 1: MapReduce - Functional Programming

Map Reduce

Page 2: MapReduce - Functional Programming

Map

Page 3: MapReduce - Functional Programming

Double the contents of an array

var a = [1, 2, 3];for (int I = 0; I < a.length; i++){a[ i ] = 2*a[ i ];}

Output:a = [2, 4, 6];

Page 4: MapReduce - Functional Programming

Double the contents of an arrayvar a = [1, 2, 3];

for (int i = 0; i < a.length; i++){

a[ i ] = 2*a[ i ];}

var a = [1, 2, 3];for (int i = 0; i < a.length; i++){

a[ i ] = fn(a[ i ]);}

where fn(x){ return 2*x;}

Page 5: MapReduce - Functional Programming

Double the contents of an arrayvar a = [1, 2, 3];

for (int i = 0; i < a.length; i++){a[ i ] = fn(a[ i ]);

} where fn(x){ return 2*x;}

function map(fn, a){for (int i = 0; i < a.length; i++){a[ i ] = fn(a[ i ]);

}where fn(x){ return 2*x;}

Page 6: MapReduce - Functional Programming

Double the contents of an arrayfunction map(fn, a){

for (int i = 0; i < a.length; i++){a[ i ] = fn(a[ i ]);

}where fn(x){ return 2*x;}

map(function(x){ return 2*x;},a}General Form

Page 7: MapReduce - Functional Programming

Reduce

Page 8: MapReduce - Functional Programming

Sum the contents of an array

function sum(a){ var s= 0; for(int i=0; i<a.length; i++){ s = s + a[ i ]; } return s;}

Page 9: MapReduce - Functional Programming

Sum the contents of an arrayfunction sum(a){ var s= 0; for(int i=0; i<a.length; i++){

s = s + a[ I ]; } return s;}

function sum(a){ var s= 0; for(int i=0; i<a.length; i++){

s = fn(s, a[ I ]); } return s;}

where fn(a, b){ return a+b;}

Page 10: MapReduce - Functional Programming

Sum the contents of an arrayfunction sum(a){ var s= 0; for(int i=0; i<a.length; i++){ s = fn(s, a[ I ]); } return s;}

where fn(a, b){ return a+b;}

function reduce(fn, a, init){ var s= init; for(int i=0; i<a.length; i++){ s = fn(s, a[ I ]); } return s;}

where fn(a, b){ return a+b;}

Page 11: MapReduce - Functional Programming

Sum the contents of an arrayfunction reduce(fn, a, init){ var s= init; for(int i=0; i<a.length; i++){ s = fn(s, a[ I ]); } return s;} where fn(a, b){ return a+b;}

General form

reduce(function(a,b){ retun a+b;}, a, init);

Page 12: MapReduce - Functional Programming

Benefits

In case of big arrays, code running on multiple machines, one need not to rewrite the code for computing but just replace the implementation of map and reduce as required.

Microsoft Word Document

map(function(x){ return 2*x;},a}

reduce(function(a,b){ return a+b;}, a, init);

Page 13: MapReduce - Functional Programming

What is hadoop?

Hadoop is parallel, distributive, fault tolerant and network topology aware implementation of MapReduce algorithm.

Page 14: MapReduce - Functional Programming