OperatingSystems

OPERATING SYSTEMS:

Lesson 11:Files

1

Jesús Carretero PérezDavid Expósito SinghJosé Daniel García SánchezFrancisco Javier García BlasFlorin Isaila

OperatingSystems

Goals

• Toknowtheconceptsoffileanddirectoryandtheircharacteristics.

• Tousefileanddirectorymanagementservicesofferedbydeoperatingsystem.

• Tounderstandafilesystemstructure.• Tounderstandthemechanismssupportingafileserverandtoapplythemtosimpleexercises.

2

OperatingSystems

Content

• Files• Attributesandoperations• Logicalview• Sharingsemantics• Representation

3

OperatingSystems

Storage

• Mainmemory.– Volatilememoryà nonpersistentdata.– Dataaccesseddirectlybyprocessor.

• Secondarymemory.– Nonvolatilememoryà persistentdata.– Organizedindatablocks.– Anabstractionneededtosimplifyaccess:File.

4

OperatingSystems

File

Byte sequence

1

block

2 3 4

1 4 3

2

5

OperatingSystems

FileSystems

• Accessingdevicesare:– Difficult:

• Physicaldetailsofdevices.• Dependentonphysicaladdresses.

– Unsafe:• Ifuseraccessestophysicalleveltherearenorestrictions.

• FileSystemisthesoftwarelayerbetweendevicesandusers.

• Goals:– Toprovidealogicalviewofdevices.– Toofferaccessprimitiveseasytouseandindependentfrom

physicaldetails.– Toprovideprotectionmechanisms.

6

OperatingSystems

Filesystem

• Offersasimplifiedlogicalviewforhandlingperipheraldevicesinformoffiles.

• Providesanabstractionmechanismhidingdetailsrelatedtostorageandinformationdistributionamongperipherals.

• Functions:– Organization.– Storage.– Retrieval.– Namemanagement.– Implementco-utilizationsemantics.– Protection.

7

OperatingSystems

Filesystem:logicalview

• Logicalview:– Files.– Directories.– Filesystemsandpartitions.

• Physicalview:– Disk->partitions+volumes->directories->files->data– Blocksorbytesplacedindevices.

8

OperatingSystems

Featuresforusers

• Permanentstorageofinformation.– Doesnotdisappearwhencomputerisswitchedoff.

• Setofinformationstructuredlogicallyfollowingapplicationcriteria.

• Logicalandstructurednames.• Dissociatedfromspecificapplicationlifecycle.• Abstractphysicalstoragedevices.• Accessedthroughoperatingsystemcallsorutilitylibraries.

9

OperatingSystems

Content

• Files• Attributesandoperations• Logicalview• Sharingsemantics• Representation

10

OperatingSystems

Fileattributes

• Name:Identifierinreadableformatforaperson.• Identifier: Uniquelyidentifiesthefile.

– Usuallynumeric.• File type:Neededinsystemsprovidingmultiplefile

formats.– Atleastusedtodifferentiate executable attribute.

• Location: Storagedeviceidentificationandpositionindevice.

• File size:Numberofbytesinfile,maximumpossiblesize,…• Protection:Accessandoperationscontrolonfile.• Time information:Creationdate,accesstype,

modificationattributes,…

11

OperatingSystems

Filenamesandextensions

• Characteristicfromeachfilesystem.– Importantforusers.

• Problem:uselogicalnamesbasedincharacterstrings.• Motivation:Usersdonotremembernameslike001223407654• Typeandlengthchangefromsystemtosystem:

– Length:fixedinMS-DOSorvariableinUNIX,Windows.– Extension:Mandatoryornot,more thanone, fixedperfiletype,…

• Casesensitive:– ExampleSYSTEMandsystemarethesamefileinWindowsbutdifferent on

GNU/Linux.• Filesystemworkswithinternalfiledescriptors.

– Onlydifferentiates someformats(executableversusnon-executable).– Example:magicnumber inUNIX.

12

OperatingSystems

Filenamesandextensions

• Directoriesmatchlogicalnamesandinternalfiledescriptors.• Extensionsmaybeusedbyapplications(html,c,cpp,...)

13

OperatingSystems

Operationsonfiles

• Create:Allocateinitialspaceandmetadata.• Erase:Freeassociatedresources.• Write:Storeinformationonfile.• Read:Retrieveinformationfromfile.

Additional operations depending on concrete file access semantics.

14

OperatingSystems

Content

• Files• Attributesandoperations• Logicalview• Sharingsemantics• Representation

15

OperatingSystems

Filestructure

• None:wordsorbytessequences(UNIX).• Simplerecordstructure:

– Lines.– Fixedlength.– Variablelength.

• Complexstructures.– Formatteddocuments(HTML,PDF,…)

• Recordstructurescanbesimulatedontopofaplainstructure.

• Whodecidesonstructure?– Internal:Operatingsystem.– External:Applications.

16

OperatingSystems

Files:LogicalView

• Setofrelatedinformationthathasbeendefinedbycreator.

• Filestructure:– Sequenceofbytes(UNIX,POSIX)

17

Position

OperatingSystems

Accessmethods

• Sequentialaccess:– Basedonaccessmodelfrommagnetictape.– Usableinsequentialandrandomaccessdevices.– Byteorrecordorientedoperations.

Current Position

Rewind (to to start) Read/Write

18

OperatingSystems

AccessMethods

• DirectAccess– Basedinaccessmodelfromdiskdevice.– Filedividedinfixedlengthrecords.– Mayspecifyrecordnumberforreadandwriteoperations.– Mayuseapositionpointertoavoidneedingtospecifypositionineveryoperation.

– Allowstobuildontopofitothermorecomplexmethods(example:sequentialindexed).

19

OperatingSystems

Content

• Files• Attributesandoperations• Logicalview• Sharingsemantics• Representation

20

OperatingSystems

Filesharing

• Severalprocessesmayaccessatthesametimetofile.

• Itisneededtodefineacoherencesemantics.– Whenaremodificationstoafileobservablebyotherprocesses.

• Options:– UNIXsemantic.– Sessionsemantic.– Immutablefilessemantic.– Versionsemantic.

21

OperatingSystems

UNIXSemantic

• Writestofileareimmediatelyvisibletoallprocesses.• Anopenfilehasanassociatedpositionpointer.• Alternativesforthepointer:

– Eachprocesskeepsitsownpositionpointer.– Possibilityfortwoprocessestosharepositionpointer.

• Implication:– Operatingsystemmustkeepauniquefileimage.– Contentionproblemsduetoexclusiveaccesstoimage.

22

OperatingSystems

Sessionsemantic

• Writesonopenfilearenotvisibletootherprocesseswiththatfilealsoopen.

• Whenafileisclosed,changesarevisibletootherprocessesthatopenthefileafterthatevent.

• Afilemaybeassociatedwithseveraldifferentimages.• Thereisnocontention.• Usecase:AFS(AndrewFileSystem).

23

OperatingSystems

Immutablesemantic

• Filemaybedeclaredasshared.– Afterthatfilecannotbemodified.

• Animmutablefiledoesnotadmitmodificationsfor:– Name.– Content.

• Simpleimplementation: read-onlysharing

24

OperatingSystems

Versionsemantics

• Updatesperformedoncopieswithversionnumber.• Onlyvisiblewhenversionsareconsolidated.• Explicitsynchronizationifimmediateupdateisrequired.

25

OperatingSystems

Accesscontrol

• Accesscontrollists.– Definealistofusersofaccessthatcanaccessafile.– Iftherearedifferentaccesstypes,thenthereisonelistperaccesscontroltype.• Ifuserisnotonthelist->protectionviolation

• Permissions.– Condensedversionrwx- rwx- rwx

• Threeaccesstypes(rwx).• Permissionsforthreecategories(user,group,others).

26

OperatingSystems

Content

• Files• Attributesandoperations• Logicalview• Sharingsemantics• Representation

27

OperatingSystems

Filerepresentation

• Operatingsystemmustkeepinformationonfiles:metadata.

• Metadataarefilesystemdependent.• Important:Anoperatingsystemmayadmitmultiplefilesystems.– Example:GNU/LinuxmymountpartitionsinExt2,NTFS,…– Simplification:sameaccessinterface(POSIX)

28

OperatingSystems

Diskspaceallocation

• Free/useddiskspacemanagement.• Spaceallocationforeachfile.

• Aspects:– Newfiles:Ismaximumspaceallocatedoncreation?– Whichallocationunitsareused?– Whichdatastructurerepresentsfileallocation?

29

OperatingSystems

Pre-allocationversusdynamicallocation

• Pre-allocation:– Allocationofmaximumpossiblefilesizeoncreation.– Maximumspaceisreserved.

• Dynamicallocation:– Spaceisallocatedasitisneeded.– Filedividedintoallocationunitsthataretakenondemand.

30

OperatingSystems

Allocationsize

• Issuestobeconsidered:– Largeallocationsizeà Informationcontiguousindisk.

• Higherperformance.– SmallallocationsizeàMetadatasizeincreases.

• Lowerstoragecapacity.

– Fixedallocationsizeà Spacereallocationissimple.– Fixedandlargeallocationsizeà incrementsspacewaste(internal

fragmentation).– Variableandlargeallocationsizeà increasesperformance,but

externalfragmentationincreasestoo.

31

OperatingSystems

AllocationMethods

• Howdiskblocksareallocatedforfilesordirectories:

– Contiguousallocation

– Linkedallocation

– Indexedallocation

32

OperatingSystems

ContiguousAllocation

• Eachfileoccupiesasetofcontiguousblocksonthedisk• Simple– onlystartinglocation(block#)andlength(numberofblocks)are

required• Randomaccesseasy• Problems

– Findspacefornewfile(first/best/worst)– Filescannotgrowmorethantheallocatedspace

• Preallocation– Buthowmuchspacedoesafileneed?

• Solution:fileasacollectionofextents– Acontiguouschunkofblocks– Whenfull,addapointertothenextextent

33

OperatingSystems

Contiguousallocation

A A A0 1 2 3 4

B5 6 7 8 9B B B B10 11 12 13 14

C C15 16 17 18 19C C C C C20 21 22 23 24C E E E25 26 27 28 29D D30 31 32 33 34

File Start Length

A 2 3B 9 5C 18 8D 30 2E 26 3

Defragneeded

34

OperatingSystems

Contiguousallocation(defragmented)

A A A B B0 1 2 3 4B B B C C5 6 7 8 9C C C C C10 11 12 13 14C E E E D15 16 17 18 19D20 21 22 23 24

25 26 27 28 29

30 31 32 33 34

File Start Length

A 0 3B 3 5C 8 8D 19 2E 16 3

35

OperatingSystems

Linkedallocation

• Eachblockcontainsapointertonextblock.• Blockallocationonebyone.• Noexternalfragmentationhappens.• Blocksdistributedacrossdisk.• Systemconsolidationtoincreaseperformanceinsequential

fileprocessing.– Increasedatalocality

• Advantages:Simple– needonlystartingaddress,Free-spacemanagementsystem– nowasteofspace.

• Problems:Norandomaccess,Spaceforpointers.

36

OperatingSystems

Linkedallocation

B0 1 2 3 4

5 6 7 8 9B

10 11 12 13 14B

15 16 17 18 19B

20 21 22 23 24

25 26 27 28 29B30 31 32 33 34

File Start Length

B 1 5

37

OperatingSystems

Linkedallocation(consolidated)

B B B B B0 1 2 3 4

5 6 7 8 9

10 11 12 13 14

15 16 17 18 19

20 21 22 23 24

25 26 27 28 29

30 31 32 33 34

File Start Length

B 0 5

38

OperatingSystems

Indexedallocation

• Tablewithallocationunitsidentifierscomposingthefile.

• Alternatives:– Blockallocation.– Extentallocation.

39

OperatingSystems

IndexedAllocation

• Advantages– Randomaccess– Dynamicaccesswithoutexternalfragmentation

• Problem– Overheadofindexblock.

• Indexblockorganization– Linkedscheme:anindexblockisonediskblockcontaining

pointerstodatablocks– Multilevelindex:ex.adiskblockcontainspointerstoblocks

containingpointerstodatablocks– Combined

• Both(UNIX)

40

OperatingSystems

Blockindexedallocation

B0 1 2 3 4

B5 6 7 8 9

B10 11 12 13 14

15 16 17 18 19

20 21 22 23 24B

25 26 27 28 29B

30 31 32 33 34

File StartB 19

1,12,26,7,33

41

OperatingSystems

Extentallocation

B B B0 1 2 3 4

5 6 7 8 9B

10 11 12 13 14

15 16 17 18 19

20 21 22 23 24B B

25 26 27 28 29B B30 31 32 33 34

File StartB 19

Start Length

1 328 414 1

42

OperatingSystems

Diskspacemanagement

• Operatingsystemmustknowwhichblocksarefreeorused.

• Alternatives:– Bitmaps:Vectorwithonebitperblock

• Summarytablewithaddressranges:Numberoffreeblocksinrange.

– Linkedlistoffreeextents.– Indexing:Indextableoffreeextents.

43

OperatingSystems

Usecase:FAT

Name

Attributes

Creation date

Name

FAT Pointer

Size

1 2 3 4 5 6 File allocation table

2 5 1 6Disk blocks

44

OperatingSystems

Usecase:UNIX

• Filetypeandprotection.• Fileowneruser.• Fileownergroup.• Filesize.• Creationdateandtime.• Lastaccessdateandtime.• Lastmodificationdateandtime.• Numberoflinks.• Directblockpointers(10).• Simpleindirectpointer.• Doubleindirectpointer.• Tripleindirectpointer.

45

OperatingSystems

Usecase:UNIX

46

OperatingSystems

Usecase:NTFS

Header

Attributes

Size

Name

Security

Data

Vclusters

1,5 KB B+ Tree

47

OperatingSystems

OPERATING SYSTEMS:

Lesson 11:Files

48

Jesús Carretero PérezDavid Expósito SinghJosé Daniel García SánchezFrancisco Javier García BlasFlorin Isaila