Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
OVERVIEWLBFS MOTIVATION INTRODUCTION
CHALLENGES ADVANTAGESOFLBFS HOWLBFSWORKS? RELATEDWORK
DESIGN SECURITYISSUES IMPLEMENTATION
SERVERIMPLEMENTATION CLIENTIMPLEMENTATION
EVALUATION SHARK
MOTIVATION
UsersrarelyconsiderrunningNFSoversloworwideareanetworks.
Ifbandwidthislow,performanceisunacceptable.
Datatransferssaturatebo@lenecklinksandcauseunacceptabledelays.
InteracAveapplicaAonsareslowinrespondingtouserinput.
RemoteloginisfrustraAng
SoluAon? Run InteracAve programs locally and manipulate remote files
throughthefilesystem. NetworkFilesystemshouldconsumelessbandwidth. LBFS.
INTRODUCTION
LBFSisusedforsloworWideareaNetworks.
ExploitssimilariAesbetweenFilesorversionsofthesamefile.
ItusesConvenAonalcomparisonandCaching.
In LBF, interacAve programs and accessing remote datathroughfilesystemrunlocally.
LBFS requires 90% less bandwidth than TradiAonalNetworkFileSystem.
Challenges? Advantages
ProvidesTradiFonalFSSchemaFcs LocalCache ExploitsCrossFilesimilariFes
VariableSizeChunks Indexeschunksbyhashvalues
HowLBFSWork? ProvidesClosetoOpenConsistency
RELATEDWORK
AFSusesservercallbackstoreducenetworktraffic.
LeasesarecallbackswithexpiraAondate.
CODA supports slow networks and even disconnected operaAonsthroughopAmisAcreplicaAon.
CODAsavesbandwidthasitavoidstransferringfilestotheserver.
BayouandOceanStoreinvesAgateconflictresoluAonforopAmisAcupdates.
SpringandWetherall: Uselargeclientandservercaches
RsyncexploitssimilariAesbetweendirectorytrees.
DESIGN
LBFSuseslargepersistentfilecacheatclient. ItassumesClienthasenoughcache.
ItExploitssimilariAesbetweenfilesandfileversions. DividesFilesintoChunks. Onlytransmitsdatachunkscontainingnewdata.
Tosavechunktransfer,LBFSreliesontheSHA‐1Hash.
LBFSUses“gzip”compression.
CentralchallengeinDesignis: Keepingtheindexareasonablesize Dealingwithshi[ingoffsets.
PROBLEMSWITHFIXEDSIZEDBLOCKS
Single byte inserAon shi[s all the blockboundaries.
OriginalFile
AUerInserFng
PossiblesoluFons: Indexfilesbythehashesofalloverlapping8KBblocksatalloffsets.
Rsync:Consideronly twofilesataAme.Existenceofafileisfoundusingfilename.
x
LBFSSoluFonforBlockSize
LBFS Onlylooksfornon‐overlappingchunksinfiles
AvoidssensiFvitytoshiUingfileoffsetsbySeZngchunkboundariesbasedonfilecontents.
Todivideafileintochunks,LBFS Examinesevery(overlapping)48‐byteregionofthefile.
LBFSUses Rabin’s fingerprints to select boundary regionscalledbreakpoints.
Fingerprintsareefficienttocomputeonaslidingwindowinafile.
RabinFingerPrints
Polynomial representaAonofdata in48‐byte regionmoduloanirreduciblepolynomial. FingerPrint=f(x)modp(x)
ProbabilityofCollision=max(|r|,|s|)/2w‐1
Boundaryregionshavethe13leastsignificantbitsoftheir fingerprint equal to an arbitrary predefinedvalue.
Methodisreasonablyfast.
ChunkBoundariesAUeraSeriesofEdits?
Figureshowsthefiledividedintovariablelengthchunkswithbreakpointsdeterminedbyhashofeach48bitregion.
EffectofinserAngsometextintothefileatchunkC4. TransferonlyC8.
EffectofinserAngadatainC5thatcontainsabreakpoint Spliengthechunksintotwonewchunks.(C9andC10) TransferonlytwonewchunksC9andC10.
Oneofthebreakpointiseliminated.C2+C3‐>C11 TransferonlyC11
PATHOLOGICALCASES VariablesizechunkscanleadtoPathologicalbehavior.
Ifevery48bytesofafilehappenedtobeabreakpoint.
VerylargechunkswouldbetoodifficulttosendinasingleRPC.
ArbitrarysizeRPCmessageswouldbesomewhatinconvenient.
Chunksizesmustbebetween2Kand64K
ArAficiallyinsertchunkboundariesiffileisfullofrepeatedsequences.
CHUNKDATABASE Thechunkdatabase Indexeschunksbyfirst64bitsofSHA‐1
hash.
Thedatabasemapskeysto(file,offset,count)triples.
LBFSneverreliesonthecorrectnessofthechunkdatabase.
Howtokeepthisdatabaseuptodate? Mustupdateitwheneverfileisupdated
CansAllhaveproblemswithlocalupdatesatserversite Crashescancorruptdatabasecontents.
FILECONSISTENCY TheLBFSclientcurrentlyperformswholefilecaching.
LBFSusesathree‐Aeredschemetodetermineifafileisuptodate. OPENAFILE:
IFLeaseNotExpired IFLeaseExpired
ClientgetsaleasefirstAmeafileisopenedforread.
ClientRenewstheexpiredleasebyrequesAngfilea@ributes.
It’sthejoboftheClienttocheckifthecachedcopyissAllcurrent.
FILEREADS LBFSUseaddiAonalcallsnotinNFS‐>GETHASHforreads
FILEWRITES LBFSServerupdatesfilesatomicallyatcloseAme.
UsesTemporaryFiles. 4RPC’sareusedinupdateprotocol:
*1.MKTMPFILE*2.CONDWRITE*3.TMPWRITE*4.COMMITTMP.
SECURITY LBFSusesthesecurityinfrastructurefromSFS. AllServershavepublickeys. AccessControl.
IMPLEMENTATION
mkdbuAlity IfFileSize<8KB TrashDirectory
EVALUATION–REPEATEDDATAINFILES
Bandwidth consumpAon and network uAlizaAon aremeasuredunderseveralcommonworkloads.
LBFSiscomparedwith: CIFS, NFSversion3and AFS.
EVALUATION(Cont)–BANDWIDTHUTILIZATION
• Used3WORKLOADS.
• (MSWord1.4MBfile,gcc‐>Compiledemacs20.7,ed‐>perl)
EVALUATION(3)–APPLICATIONPERFORMANCE
OVERVIEW
SHARK MOTIVATION INTRODUCTION CHALLENGES ADVANTAGESOFSHARK HOWSHARKWORKS?
DESIGN IMPLEMENTATION EVALUATION CONCLUSTION DISCUSSION(QUESTIONARIES)
MOTIVATION
CurrentSystems
1.ReplicaFngExecuFonEnvironment 2.MulFpleClientCopyingSameFiles
SERVER
Program1
libraries
P2P3
P4
Data
CLIENT
P1Launch
Data
P1
C2
P1
C3
P1
C6
P3
C4
P1
C5
Replicate their execuFon environment on each machine beforelaunchingdistributedapplicaFon‐>WASTERESOURCES+DEBUG
SERVER1
Data1 Data2
Data3
Data3
Data2
Data1
SERVER3
Data1
Data2
Data3
SERVER2
Client1
C3
C2C4
RequestData3
UpdateDB
UpdateDB
Data3,1 Data2,1
UpdateDB
Data3
ReplicaFngdataandservingsamecopyofdatatomulFpleclients‐>INCONSISTENCIES
INTRODUCTION
Sharkisadistributedfilesystem
Designedforlarge‐scale,wide‐areadeployment.
Scalable.
ProvidesanovelcooperaAve‐cachingmechanism
Reducestheloadonanoriginfileserver.
CHALLENGES
Scalability:Whatifalargeprogram(sayinMB’s)isbeingexecutedfromafileserverbymanyclients?
Becauseofbandwidth,servermightdeliverunacceptableperformance.
As the model is similar to P2P file systems, administraAon,accountability,andconsistencyneedtobeConsidered.
HOWSHARKWORKS?
Shark clients findnearby copies of data by using distributedindex.
Clientavoids transferring thefile/chunksof thefile fromtheserver,ifthesamedatacanbefetchedfromnearby,client.
Shark is compaAble with exisAng backup and restoreprocedures.
By shared read, Shark greatly reduces server load andimprovesclientlatency.
DESIGN–PROTOCOL(1/2)
1. Sharkserverdividesthefile intochunksbyusingRabinfingerprintalgorithm.
2. Shark interacts with the local host using an exisAngNFSv3andrunsinuserspace.
3. WhenFirstClientreadsaparAcularfile Gets file andRegisters as replica proxy for the chunks of the
fileinthedistributedindex.
4. Nowwhen2ndclientwantstoaccessthesamefile: it discovers the proxies of the file chunks by querying the
distributedindex. establishes a secure channel to (mulAple such) proxy(s) , and
downloadthefilechunksinparallel.
DESIGNPROTOCOL(2/2)
5.A[erfetching,theclientthenregistersitselfasareplicaproxyforthesechunks.
6.ServerexposestwoApi’s. Put: Client executes put to declare that it has something.
Get: clientexecutesget toget the listof clientswhohavesomething.
SECUREDATASHARING7. Data is encrypted by the sender and can be decrypted
onlybytheclientwithappropriatereadpermissions.
8.Clientscannotdownload largeamountsofdatawithoutproperreadauthorizaAon.
SECUREDATASHARING
How? Cross‐file‐systemsharing
Sharkuses tokengeneratedbythefileserverasasharedsecretbetweenclientandproxy.
Clientcanverifytheintegrityofthereceiveddata.
For a sender client (proxy client) to authorize requesterclient,requesterclientwillprovidetheknowledgeoftoken
Once authorized, receiver client will establish the readpermissionofthefile.
FILECONSISTENCY
Sharkusestwonetworkfilesystemtechniques: Leases
AFS‐stylewhole‐filecaching
WhenclientmakesareadRPCtothefileserver,itgetsareadleaseontheparAcularfile.
InSharkdefaultleaseduraAon=5mins.
ForFileCommonaliAes:ItusesLBFS.
COOPERATIVECACHING
Client(C1) Server C2 C3 C4
FileNotCached/LeaseExpired
GETTOK (fh, offset, count)
1. TF = tok (F) = HMACr(F) 2. Split data into chunks 3. Compute tokens for chunks
1. file attributes 2. file token 3. (chunktok1, offset1, size1) (chunktok2, offset2, size2) (chunktok3, offset2, size2)
Determinesif(LocalCache==Latest)
if(LocalCacheisnotlatest=>Create‘K’Threads
t1
Multicast Requesting for Chunk Fi
Chunk F1
Chunk F3
t2t3
t1t2t3
Request Chunk F2 t3
F2
F3
Chunk F2
Issues Series of Read calls to the Kernel NFS server.
Caches the tokens for future reference.
.
Fetch‘K’ChunksinParallel
PUT() -> Announuces as a Proxy for Fi
DISTRIBUTEDINDEXING
Sharkusesglobaldistributedindexforallsharkclients.
Systemmaps opaque keys onto nodes by hashing the valueontoakeyID.
AssigningID’stonodesallowslookupontheO(log(n))
SharkstoresonlysmallinformaAonaboutwhichclientsstoreswhatdata.
SharkusesCoralasitsdistributedindex.
CoralProvidesDistributedSloppyHashTable(DSHT).
Coralcacheskey/valuepairsatnodeswhoseIDsareclose.
IMPLEMENTATION
Sharkconsistsof3MainComponents: Serversidedaemon
Clientsidedaemon
Coraldaemon
Implemented in C++ andarebuiltusingSFStoolkit.
Clientsidedaemon
BiggestComponentofshark.
HandlesUserRequests Transparently
incorporateswholefilecaching.
Codeis~12,000Lines
EVALUATION(1/2)
SharkisevaluatedagainstNFSv3andSFS.
Readtestsareperformedboth
withinthecontrolledEmulabLANenvironmentand
InthewideareaonthePlanetLabv3.0test‐bed.
Theserverrequired0.9seconds tocomputechunksfora10MBrandomfile,and3.6secondsfora40MBrandomfile.
EVALUATION‐Microbenchmarks
Forthelocal‐areamicro‐benchmarks,localmachinesatNYUareusedasaSharkclient.
Inthismicro‐benchmark,Shark’schunkingmechanismreducesredundantdatatransfersbyexploiAngdatacommonaliAes.
QUESTIONS
THANKYOU