IKT433-G, general information

IKT433GDistributedandBigDataSystems

1/13

X X X X X X X

IKT433-G,generalinformation

Attachingsketchestothisquestion?Usethefollowingcode:

Coursecode:IKT433-GCoursename:DistributedandBigDataSystemsDate:24.05.2019Duration:4hoursResourcesallowed:Notes:PartA–Distributedsystemswillweight60%ofthewrittenexamgrade.EachassignmentwithinpartAisweightedequallyandeachsubquestionwithineachassignmentisweightedequally.

PartB–BigDatawillweight40%ofthewrittenexamgrade.EachassignmentwithinpartBisweightedequallyandeachsubquestionwithineachassignmentisweightedequally.------------------Theprofessorssometimesaskforexamanswerstobeusedforteachingpurposes,butinorderforthistotakeplace,theuniversityneedsyourconsent.DoyougranttheUniversityofAgderpermisionsuchpermission?Selectonealternative

Yes

No


2/13

X X X X X X X

1 BigData1


WhatarethetwocorecomponentsofHadoopandhowdotheywork?Fillinyouranswerhere

Words:0


3/13

X X X X X X X

2 BigData2


Inthefollowingassignment;selectoneoftheanswers,giveashortjustificationforyouranswer.InaMapReducejob,youwanteachofyouinputfilesprocessedbyasinglemaptask.HowdoyouconfigureaMapReducejobsothatasinglemaptaskprocesseseachinputfileregardlessofhowmanyblockstheinputfileoccupies?A.Increasetheparameterthatcontrolsminimumsplitsizeinthejobconfiguration.B.WriteacustomMapRunnerthatiteratesoverallkey-valuepairsintheentirefile.C.Setthenumberofmappersequaltothenumberofinputfilesyouwanttoprocess.D.WriteacustomFileInputFormatandoverridethemethodisSplittabletoalwaysreturnfalse.Fillinyouranswerhere

Words:0


4/13

X X X X X X X

3 BigData3


Inthefollowingassignment;selectoneoftheanswers,giveashortjustificationforyouranswer.YouaredevelopingacombinerthattakesasinputTextkeys,IntWritablevalues,andemitsTextkeys,IntWritablevalues.Whichinterfaceshouldyourclassimplement?A.Combiner<Text,IntWritable,Text,IntWritable>B.Mapper<Text,IntWritable,Text,IntWritable>C.Reducer<Text,Text,IntWritable,IntWritable>D.Reducer<Text,IntWritable,Text,IntWritable>E.Combiner<Text,Text,IntWritable,IntWritable>Fillinyouranswerhere

Words:0


5/13

X X X X X X X

4 BigData4


Inthefollowingassignment;selectoneoftheanswers,giveashortjustificationforyouranswer.Whatisthedifferencebetweenafailedtaskattemptandakilledtaskattempt?A.Afailedtaskattemptisataskattemptthatthrewanunhandledexception.AkilledtaskattemptisonethatwasterminatedbytheJobTracker.B.Afailedtaskattemptisataskattemptthatdidnotgenerateanykeyvaluepairs.Akilledtaskattemptisataskattemptthatthrewanexception,andthuskilledbytheexecutionframework.C.Afailedtaskattemptisataskattemptthatcompleted,butwithanunexpectedstatusvalue.Akilledtaskattemptisaduplicatecopyofataskattemptthatwasstartedaspartofspeculativeexecution.D.AfailedtaskattemptisataskattemptthatthrewaRuntimeException(i.e.,thetaskfails).Akilledtaskattemptisataskattemptthatthrewanyothertypeofexception(e.g.,IOException);theexecutionframeworkcatchestheseexceptionsandreportsthemaskilled.¨Fillinyouranswerhere

Words:0


6/13

X X X X X X X

5 BigData5


Inthefollowingassignment;selectoneoftheanswers,giveashortjustificationforyouranswer.WhenistheearliestpointatwhichthereducemethodofagivenReducercanbecalled?A.Assoonasatleastonemapperhasfinishedprocessingitsinputsplit.B.Assoonasamapperhasemittedatleastonerecord.C.Notuntilallmappershavefinishedprocessingallrecords.D.ItdependsontheInputFormatusedforthejob.Fillinyouranswerhere

Words:0


7/13

X X X X X X X

6 BigData6


Inthefollowingassignment;selectoneoftheanswers,giveashortjustificationforyouranswer.WhichdescribeshowaclientreadsafilefromHDFS?A.TheclientqueriestheNameNodefortheblocklocation(s).TheNameNodereturnstheblocklocation(s)totheclient.TheclientreadsthedatadirectoryofftheDataNode(s).B.TheclientqueriesallDataNodesinparallel.TheDataNodethatcontainstherequesteddatarespondsdirectlytotheclient.TheclientreadsthedatadirectlyofftheDataNode.C.TheclientcontactstheNameNodefortheblocklocation(s).TheNameNodethenqueriestheDataNodesforblocklocations.TheDataNodesrespondtotheNameNode,andtheNameNoderedirectstheclienttotheDataNodethatholdstherequesteddatablock(s).TheclientthenreadsthedatadirectlyofftheDataNode.D.TheclientcontactstheNameNodefortheblocklocation(s).TheNameNodecontactstheDataNodethatholdstherequesteddatablock.DataistransferredfromtheDataNodetotheNameNode,andthenfromtheNameNodetotheclient.Fillinyouranswerhere

Words:0


8/13

X X X X X X X

7 BigData7


Whatismeantbylocality,andhowwillthataffectsystemslikehadoop?Fillinyouranswerhere

Words:0


9/13

X X X X X X X

8 DistributedSystems:1


Assignment1

a:BrieflydiscussthefollowingpropertiesoftheMiddlewareCommunicationModels.b:Fromthepropertiesdiscussedabove,identifythepropertiesofRemoteProcedureCall(RPC)modelandusethosepropertiestoexplainitsfunctionality.C:ExplainhowbufferingandinterleavingcanprovideQoSinStream-OrientedCommunications.estiontext.Fillinyouranswerhere

Words:0


10/13

X X X X X X X



Assignment2:

a-Whatdowemeanbyatransparentdistributedsystem?

b-DiscusswhatismeantbyLocationTransparency,MigrationTransparency,

ConcurrencyTransparency.threekindsoftransparencyexistinginadistributedsystem?

c-Theimplementationofanamespacerequirespartitioningitintothreedifferent

layers.Explainindetailthesethreelayers.

d-Whatismeantbyfaulttolerance?

e-ExplainwhatismeantbyTimingfailuresandArbitraryfailures?

f-Akeytechniqueformaskingfaultsistouseredundancy.Explainwhatis

physicalredundancy?

g-Explainthedifferencebetweenerror,faultandfailure.

h-Whatisadistributedsystem?

i-Whatisthedifferencebetweenamulticomputerandamultiprocessor?

Fillinyouranswerhere

Words:0


11/13

X X X X X X X



Assignment3

Geneticalgorithm(GA)iswellknownoptimizationmethod.Inordertospeedupgeneticalgorithmforlargescaleoptimizationproblems,GAiscombinedwiththemultilevelschemei.

a:Explainindetailhowthegeneticalgorithmworks.

b:ExplaintheideaofthemultilevelschemewhenusedwithGA.


Words:0


Assignment4:

a-Whatdowemeanbyaconsistencymodel?

b-Giventhefollowing2examples,whereWi(x)aandRi(x)bmeanthata

writebyaprocessPitodataitemxwiththevalueaandareadfromthatitem

byPireturningbhavebeendonerespectively.

Isexample1strictlyconsistent?Why?

Example1:

P1:W(x)a

------------------------------------------------------------

P2:R(x)NILR(x)a

--------------------------------------------------------------

Example2:


12/13

X X X X X X XAttachingsketchestothisquestion?Usethefollowingcode:

P1:W(x)a

--------------------------------------------------------------

P2:W(x)b

--------------------------------------------------------------

P3:R(x)bR(x)a

--------------------------------------------------------------

P4:R(x)aR(x)b

IsExmple2causallyconsistent?


Words:0


13/13

X X X X X X X



Assignment5:

a:Explainindetailhowthetwophasecommitprotocol(2PC)worksandtheproblems(blocking)

itmayhave.

b:LamporteAlgorithmisusedtoachieveagrreementinadistributedsyste.Explainindetatilhowitworks.


Words:0

Documents

IKT433-G, general information