Upload
livijhacer-ruiz
View
226
Download
2
Tags:
Embed Size (px)
DESCRIPTION
SO
Citation preview
Fundamentos de S.O. por Mtro. Abraham Jiménez Alfaro
1
Unit OS8: File SystemUnit OS8: File System
Windows Operating System Internals Windows Operating System Internals -- by David A. Solomon and Mark E. Russinovich with Andreas Polzeby David A. Solomon and Mark E. Russinovich with Andreas Polze
Unit OS8: File SystemUnit OS8: File System8.2.8.2. Windows File SystemsWindows File Systems
Copyright NoticeCopyright Notice© 2000© 2000--2005 David A. Solomon and Mark Russinovich2005 David A. Solomon and Mark Russinovich
These materials are part of the These materials are part of the Windows Operating Windows Operating System Internals Curriculum Development Kit,System Internals Curriculum Development Kit,developed by David A. Solomon and Mark E. developed by David A. Solomon and Mark E. Russinovich with Andreas PolzeRussinovich with Andreas Polze
Microsoft has licensed these materials from DavidMicrosoft has licensed these materials from David
2
Microsoft has licensed these materials from David Microsoft has licensed these materials from David Solomon Expert Seminars, Inc. for distribution to Solomon Expert Seminars, Inc. for distribution to academic organizations solely for use in academic academic organizations solely for use in academic environments (and not for commercial use)environments (and not for commercial use)
Roadmap for Section 8.2Roadmap for Section 8.2
File Systems supported by WindowsFile Systems supported by WindowsNTFS Design GoalsNTFS Design GoalsFile System Driver ArchitectureFile System Driver Architecture
3
File System Driver ArchitectureFile System Driver ArchitectureNTFS OperationNTFS OperationWindows File System OnWindows File System On--Disk StructureDisk StructureNTFS File CompressionNTFS File Compression
Fundamentos de S.O. por Mtro. Abraham Jiménez Alfaro
2
Windows File System Windows File System --TerminologyTerminology
Sectors: Sectors: hardwarehardware--addressable blocks on a storage mediumaddressable blocks on a storage mediumTypical sector size on hard disks for x86Typical sector size on hard disks for x86--based systems is 512 bytesbased systems is 512 bytes
File system formats:File system formats:Define the way data is stored on storage mediaDefine the way data is stored on storage mediaI t fil t f t i i & it li it ti fil iI t fil t f t i i & it li it ti fil i
4
Impact a file system features: permissions & security, limitations on file size, Impact a file system features: permissions & security, limitations on file size, support for small/large files/diskssupport for small/large files/disks
Clusters:Clusters:Addressable blocks that many file system formats useAddressable blocks that many file system formats useCluster size is always a multiple of the sector sizeCluster size is always a multiple of the sector sizeCluster size tradeoff: space efficiency vs. access speedCluster size tradeoff: space efficiency vs. access speed
Metadata:Metadata:Data stored on a volume in support of file system format managementData stored on a volume in support of file system format management
Metadata includes the data that defines the placement of files and Metadata includes the data that defines the placement of files and directories on a volume, for exampledirectories on a volume, for exampleTypically not accessible to applicationsTypically not accessible to applications
Formats Supported by WindowsFormats Supported by Windows
CDCD--ROM File System (CDFS)ROM File System (CDFS)
Universal Disk Format (UDF)Universal Disk Format (UDF)
5
File Allocation Table (FAT12, FAT16, and File Allocation Table (FAT12, FAT16, and FAT32)FAT32)
New Technology File System (NTFS)New Technology File System (NTFS)
CDFSCDFS
CDFS, or, is a relatively simple format that was defined in 1988 as CDFS, or, is a relatively simple format that was defined in 1988 as the readthe read--only formatting standard for CDonly formatting standard for CD--ROM media. ROM media.
Windows 2000 implements ISO 9660Windows 2000 implements ISO 9660--compliant CDFS in compliant CDFS in \\WinntWinnt\\System32System32\\DriversDrivers\\Cdfs.sys, with long filename support Cdfs.sys, with long filename support defined by Level 2 of the ISO 9660 standarddefined by Level 2 of the ISO 9660 standard
6
Because of its simplicity, the CDFS format has a number of Because of its simplicity, the CDFS format has a number of restrictionsrestrictions
Directory and file names must be fewer than 32 characters longDirectory and file names must be fewer than 32 characters long
Directory trees can be no more than eight levels deepDirectory trees can be no more than eight levels deep
CDFS is considered a legacy format because the industry has CDFS is considered a legacy format because the industry has adopted the Universal Disk Format (UDF) as the standard for readadopted the Universal Disk Format (UDF) as the standard for read--only mediaonly media
Fundamentos de S.O. por Mtro. Abraham Jiménez Alfaro
3
UDFUDF
OSTA (Optical Storage Technology Association) defined OSTA (Optical Storage Technology Association) defined UDF in 1995 as a format to replace CDFS for magnetoUDF in 1995 as a format to replace CDFS for magneto--optical storage media, mainly DVDoptical storage media, mainly DVD--ROMROM
The Windows 2000 UDF file system implementation is ISO The Windows 2000 UDF file system implementation is ISO 1334613346--compliant and supports UDF versions 1 02 and 1 5compliant and supports UDF versions 1 02 and 1 5
7
1334613346 compliant and supports UDF versions 1.02 and 1.5compliant and supports UDF versions 1.02 and 1.5
UDF file systems have the following traits:UDF file systems have the following traits:Filenames can be 255 characters longFilenames can be 255 characters long
The maximum path length is 1023 charactersThe maximum path length is 1023 characters
Although the UDF format was designed with rewritable Although the UDF format was designed with rewritable media in mind, the Windows 2000 UDF driver media in mind, the Windows 2000 UDF driver ((\\WinntWinnt\\System32System32\\DriversDrivers\\Udfs.sys) provides readUdfs.sys) provides read--only only supportsupport
FATFAT
FAT (File Allocation Table) file systems are a legacy format that FAT (File Allocation Table) file systems are a legacy format that originated in DOS and Windows 9xoriginated in DOS and Windows 9x
Reasons why Windows supports FAT file systems:Reasons why Windows supports FAT file systems:to enable upgrades from other versions of Windowsto enable upgrades from other versions of Windows
compatibility with other operating systems in multiboot systemscompatibility with other operating systems in multiboot systems
8
compatibility with other operating systems in multiboot systemscompatibility with other operating systems in multiboot systems
as a floppy disk formatas a floppy disk format
Windows FAT file system driver is implemented in Windows FAT file system driver is implemented in \\WinntWinnt\\System32System32\\DriversDrivers\\Fastfat.sysFastfat.sys
Each FAT format includes a number that indicates the number of Each FAT format includes a number that indicates the number of bits the format uses to identify clusters on a diskbits the format uses to identify clusters on a disk
Boot sector File allocation table 2 (duplicate)
File allocation table 1 Root directory Other directories and all files
FAT format organizationFAT format organization
FAT12FAT12
FAT12's 12FAT12's 12--bit cluster identifier limits a partition bit cluster identifier limits a partition to storing a maximum of 2to storing a maximum of 21212 (4096) clusters(4096) clusters
Windows uses cluster sizes from 512 bytes to 8 KB Windows uses cluster sizes from 512 bytes to 8 KB in size, which limits a FAT12 volume size to 32 MBin size, which limits a FAT12 volume size to 32 MB
9
Windows uses FAT12 as the format for all 5Windows uses FAT12 as the format for all 5--inch inch floppy disks and 3.5floppy disks and 3.5--inch floppy disks, which store inch floppy disks, which store up to 1.44 MB of dataup to 1.44 MB of data
Fundamentos de S.O. por Mtro. Abraham Jiménez Alfaro
4
FAT16FAT16
FAT16, with a 16FAT16, with a 16--bit cluster identifier, can bit cluster identifier, can address 2address 21616 (65,536) clusters(65,536) clusters
On Windows, FAT16 cluster sizes range from 512 On Windows, FAT16 cluster sizes range from 512 bytes (the sector size) to 64 KB, which limits FAT16 bytes (the sector size) to 64 KB, which limits FAT16
10
volume sizes to 4 GBvolume sizes to 4 GB
The cluster size Windows uses depends on the size The cluster size Windows uses depends on the size of a volumeof a volume
FAT32FAT32FAT32 is the most recently defined FATFAT32 is the most recently defined FAT--based file based file system formatsystem format
it's included with Windows 95 OSR2, Windows 98, and Windows it's included with Windows 95 OSR2, Windows 98, and Windows Millennium EditionMillennium Edition
FAT32 uses 32FAT32 uses 32--bit cluster identifiers but reserves the bit cluster identifiers but reserves the high 4 bits, so in effect it has 28high 4 bits, so in effect it has 28--bit cluster identifiersbit cluster identifiers
B FAT32 l t i b l 32 KB FAT32B FAT32 l t i b l 32 KB FAT32
11
Because FAT32 cluster sizes can be as large as 32 KB, FAT32 Because FAT32 cluster sizes can be as large as 32 KB, FAT32 has a theoretical ability to address 8 TB volumeshas a theoretical ability to address 8 TB volumesAlthough Windows works with existing FAT32 volumes of larger Although Windows works with existing FAT32 volumes of larger sizes (created in other operating systems), it limits new FAT32 sizes (created in other operating systems), it limits new FAT32 volumes to a maximum of 32 GBvolumes to a maximum of 32 GBFAT32's higher potential cluster numbers let it more efficiently FAT32's higher potential cluster numbers let it more efficiently manage disks than FAT16; it can handle up to 128manage disks than FAT16; it can handle up to 128--MB volumes MB volumes with 512with 512--byte clustersbyte clusters
Unlike FAT12 and FAT16, root directory is not fixed size Unlike FAT12 and FAT16, root directory is not fixed size or locationor location
Largest file size on Windows is 4GB (largest on Win9x is 2G)Largest file size on Windows is 4GB (largest on Win9x is 2G)
NTFSNTFS
NTFS is the native file system format of WindowsNTFS is the native file system format of Windows
NTFS uses 64NTFS uses 64--bit cluster indexesbit cluster indexesTheoretical ability to address volumes of up to 16 exabytes (16 billion Theoretical ability to address volumes of up to 16 exabytes (16 billion GB)GB)
Windows 2000 limits the size of an NTFS volume to that addressable Windows 2000 limits the size of an NTFS volume to that addressable
12
with 32with 32--bit clusters, which is 128 TB (using 64bit clusters, which is 128 TB (using 64--KB clusters)KB clusters)
Why use NTFS instead of FAT? FAT is simpler, making it faster for Why use NTFS instead of FAT? FAT is simpler, making it faster for some operations, but NTFS supports:some operations, but NTFS supports:
Larger file sizes and disksLarger file sizes and disks
Better performance on large disks, large directories, and small filesBetter performance on large disks, large directories, and small files
ReliabilityReliability
SecuritySecurity
Fundamentos de S.O. por Mtro. Abraham Jiménez Alfaro
5
CIFS CIFS ––the Common Internet File Systemthe Common Internet File System
The standard Windows network file systemThe standard Windows network file systemThe file sharing protocol at the heart of CIFS is an updated version of the The file sharing protocol at the heart of CIFS is an updated version of the Server Message Block (SMB) protocolServer Message Block (SMB) protocol
dates back to the middates back to the mid--1980s 1980s in 1996/97, Microsoft submitted draft CIFS specifications to the IETF in 1996/97, Microsoft submitted draft CIFS specifications to the IETF
The SMB protocol was originally developed to run over NetBIOS (NetworkThe SMB protocol was originally developed to run over NetBIOS (Network
13
The SMB protocol was originally developed to run over NetBIOS (Network The SMB protocol was originally developed to run over NetBIOS (Network Basic Input Output System) LANsBasic Input Output System) LANs
Until Windows 2000, NetBIOS support was required for SMB transportUntil Windows 2000, NetBIOS support was required for SMB transportThe machine and service names visible in the Windows Network Neighborhood The machine and service names visible in the Windows Network Neighborhood are, basically, NetBIOS addresses (Windows 2000 and later use DNS names)are, basically, NetBIOS addresses (Windows 2000 and later use DNS names)
Windows 3.11 (WfW) introduced: Windows 3.11 (WfW) introduced: service announcement and location system called Browsing service announcement and location system called Browsing The browser service provides the list of available file and print services The browser service provides the list of available file and print services presented in the Network Neighborhoodpresented in the Network NeighborhoodWorkgroup concept was expanded to create NT DomainsWorkgroup concept was expanded to create NT Domains
File System Format CompatibilityFile System Format Compatibility
FAT12/FAT16 supported on all Microsoft OS’sFAT12/FAT16 supported on all Microsoft OS’s
FAT32:FAT32:Only Windows 2000/XP/2003Only Windows 2000/XP/2003
14
Winternals FAT32 driver for NT4Winternals FAT32 driver for NT4
NTFS:NTFS:Only Windows NTOnly Windows NT--based OS’sbased OS’s
Winternals NTFSDOS for DOS accessWinternals NTFSDOS for DOS access
Winternals NTFS for Windows 98 for Win9x/Me Winternals NTFS for Windows 98 for Win9x/Me
NTFS Design GoalsNTFS Design Goals
Overcome limitations inherent in FAT / HPFSOvercome limitations inherent in FAT / HPFSFAT (File Allocation Table) does not support large disks very FAT (File Allocation Table) does not support large disks very wellwellFAT16 (MSFAT16 (MS--DOS file system) supports only up to 2DOS file system) supports only up to 21616 clustersclustersand 2 GB disks (with 64 Kb clusters!!)and 2 GB disks (with 64 Kb clusters!!)
15
FAT / root directory represents single point of failureFAT / root directory represents single point of failureNumber of entries in root directory is limitedNumber of entries in root directory is limitedHPFS removed some of FAT‘s limitations, but still did not HPFS removed some of FAT‘s limitations, but still did not support recoverability, security, data redundancy, and faultsupport recoverability, security, data redundancy, and fault--tolerancetolerance(later versions of HPFS support up to 2TeraByte disks)(later versions of HPFS support up to 2TeraByte disks)
Fundamentos de S.O. por Mtro. Abraham Jiménez Alfaro
6
NTFS RecoverabilityNTFS Recoverability
PC disk I/O in the old days: Speed was most importantPC disk I/O in the old days: Speed was most importantNTFS changes this view NTFS changes this view –– Reliability counts most:Reliability counts most:
I/O operations that alter NTFS structure are implemented as atomic I/O operations that alter NTFS structure are implemented as atomic transactionstransactions
Change directory structure, Change directory structure,
16
extend files, allocate space for new filesextend files, allocate space for new files
Transactions are either completed or rolled backTransactions are either completed or rolled backNTFS uses redundant storage for vital FS informationNTFS uses redundant storage for vital FS information
Contrasts with FAT / HPFS onContrasts with FAT / HPFS on--disk structures, which have single disk structures, which have single sectors containing critical file system datasectors containing critical file system dataRead error in these sectors Read error in these sectors --> volume lost> volume lost
NTFS Security NTFS Security and Recoverabilityand Recoverability
NTFS security is derived from NTFS security is derived from Windows Windows object modelobject modelOpen file is implemented as file object;Open file is implemented as file object;security descriptor is stored on disk as part of the filesecurity descriptor is stored on disk as part of the fileNT security system verifies access rights when a process tries to open a handle to NT security system verifies access rights when a process tries to open a handle to any objectany objectAdministrator or file owner may set permissionsAdministrator or file owner may set permissions
17
NTFS recoverability ensures integrity of FS structureNTFS recoverability ensures integrity of FS structureNo guarantees for complete recovery of user filesNo guarantees for complete recovery of user filesLayered driver model + FTDISK driverLayered driver model + FTDISK driver
Mirroring of data Mirroring of data –– RAID level 1RAID level 1Striping of data Striping of data -- RAID level 5 (one disk with parity info)RAID level 5 (one disk with parity info)
Large Disks and Large FilesLarge Disks and Large Files
Efficient support for large files and disks in NTFSEfficient support for large files and disks in NTFSFAT16:FAT16:
1616--bit wide table stores allocation status of disk bit wide table stores allocation status of disk Up to 65.536 clusters per volume (#files !!); adjustable cluster sizeUp to 65.536 clusters per volume (#files !!); adjustable cluster size
FAT32:FAT32:N i i Wi d 2000N i i Wi d 2000
18
New in since Windows 2000New in since Windows 20004kb clusters on volumes up to 8 GB4kb clusters on volumes up to 8 GBCan relocate root directory / use backup copy of FATCan relocate root directory / use backup copy of FATRoot directory is ordinary cluster chain Root directory is ordinary cluster chain –– no limits on #entriesno limits on #entries
HPFS (support dropped in NT 4.0):HPFS (support dropped in NT 4.0):32 bits to enumerate allocation units; maximum file size: 4GB32 bits to enumerate allocation units; maximum file size: 4GBAllocates disk space in terms of physical sectors of 512 bytes;Allocates disk space in terms of physical sectors of 512 bytes;problem with some disks (1024 bit sectors)problem with some disks (1024 bit sectors)
Fundamentos de S.O. por Mtro. Abraham Jiménez Alfaro
7
Large Disks and Large Files Large Disks and Large Files (contd.)(contd.)
NTFS enumerates cluster with 64NTFS enumerates cluster with 64--bit numbersbit numbersUp to 2Up to 26464 clusters of up to 64 Kbytes sizeclusters of up to 64 Kbytes sizeMaximum file size: 2Maximum file size: 26464 bytesbytesCluster size is adjustableCluster size is adjustable
512 bytes on small disks512 bytes on small disksUsed to implement
i f M i h
19
512 bytes on small disks512 bytes on small disksMaximum of 64Kb on large disksMaximum of 64Kb on large disks
Multiple data streamsMultiple data streamsFile info: name, owner, time stamps, type implemented as attributeFile info: name, owner, time stamps, type implemented as attributeEach attribute consists of a stream Each attribute consists of a stream –– sequence of bytessequence of bytesDefault data stream has no nameDefault data stream has no nameNew streams can be added: myfile.dat:stream2New streams can be added: myfile.dat:stream2File operations manipulate all streams simultaneouslyFile operations manipulate all streams simultaneously
services for Macintoshin Windows NT Server
Other NTFS FeaturesOther NTFS Features
Multiple data streams Multiple data streams
UnicodeUnicode--based names based names
Hard links Hard links
Junctions Junctions
20
Compression and sparse files Compression and sparse files
Change logging Change logging
PerPer--user volume quotas user volume quotas
Link tracking Link tracking
Encryption Encryption
POSIX support POSIX support
DefragmentationDefragmentation
Multiple Data StreamsMultiple Data Streams
In NTFS, each unit of information associated with a file, In NTFS, each unit of information associated with a file, including its name, its owner, its time stamps, its including its name, its owner, its time stamps, its contents, and so on, is implemented as a file attribute contents, and so on, is implemented as a file attribute (NTFS object attribute)(NTFS object attribute)
Each attribute consists of a singleEach attribute consists of a single streamstream that is athat is a
21
Each attribute consists of a single Each attribute consists of a single stream,stream, that is, a that is, a simple sequence of bytessimple sequence of bytes
This generic implementation makes it easy to add more This generic implementation makes it easy to add more attributes (and therefore more streams) to a fileattributes (and therefore more streams) to a file
Because a file's data is "just another attribute" of the file and Because a file's data is "just another attribute" of the file and because new attributes can be added, NTFS files (and file because new attributes can be added, NTFS files (and file directories) can contain multiple data streamsdirectories) can contain multiple data streams
Fundamentos de S.O. por Mtro. Abraham Jiménez Alfaro
8
Multiple Data StreamsMultiple Data Streams
An NTFS file has one default data stream, which has no An NTFS file has one default data stream, which has no namename
An application can create additional, named data streams and An application can create additional, named data streams and access them by referring to their names. access them by referring to their names.
To avoid altering the Microsoft Windows I/O APIs which takeTo avoid altering the Microsoft Windows I/O APIs which take
22
To avoid altering the Microsoft Windows I/O APIs, which take To avoid altering the Microsoft Windows I/O APIs, which take a string as a filename argument, the name of the data stream a string as a filename argument, the name of the data stream is specified by appending a colon (:) to the filename e.g. is specified by appending a colon (:) to the filename e.g. myfile:stream2myfile:stream2
Unicode NamesUnicode Names
Like Windows as a whole, NTFS is fully Unicode Like Windows as a whole, NTFS is fully Unicode enabled, using Unicode characters to store enabled, using Unicode characters to store names of files, directories, and volumesnames of files, directories, and volumes
23
Hard LinksHard Links
A hard link allows multiple paths to refer to the same file A hard link allows multiple paths to refer to the same file or directoryor directory
If you create a hard link named C:If you create a hard link named C:\\UsersUsers\\DocumentsDocuments\\Spec.doc Spec.doc that refers to the existing file C:that refers to the existing file C:\\My DocumentsMy Documents\\Spec.doc, the Spec.doc, the two paths link to the same ontwo paths link to the same on--disk file and you can make disk file and you can make changes to the file using either pathchanges to the file using either path
24
changes to the file using either pathchanges to the file using either path
can create hard links can create hard links with the Windoqs API with the Windoqs API CreateHardLinkCreateHardLink functionfunctionor the or the lnln POSIX functionPOSIX function
Fundamentos de S.O. por Mtro. Abraham Jiménez Alfaro
9
JunctionsJunctions
Junctions, also called symbolic links, allow a Junctions, also called symbolic links, allow a directory to redirect file or directory pathname directory to redirect file or directory pathname translation to an alternate directorytranslation to an alternate directory
If the path C:If the path C:\\Drivers is a junction that redirects to Drivers is a junction that redirects to
25
pp jjC:C:\\WinntWinnt\\System32System32\\Drivers, an application reading Drivers, an application reading C:C:\\DriversDrivers\\Ntfs.sys actually reads Ntfs.sys actually reads C:C:\\WinntWinnt\\SystemSystem\\DriversDrivers\\Ntfs.sysNtfs.sys
Junctions are a useful way to lift directories that are Junctions are a useful way to lift directories that are deep in a directory tree to a more convenient depth deep in a directory tree to a more convenient depth without disturbing the original tree's structure or without disturbing the original tree's structure or contentscontents
JunctionsJunctions
You can create junctions with the You can create junctions with the junctionjunction tool tool from Sysinternals or the from Sysinternals or the linkdlinkd tool from the tool from the Resource KitsResource Kits
26
Change LoggingChange Logging
Many types of applications, such as incremental Many types of applications, such as incremental backup utilities, need to monitor a volume for backup utilities, need to monitor a volume for changeschanges
An obvious way to watch for changes is toAn obvious way to watch for changes is to
27
An obvious way to watch for changes is to An obvious way to watch for changes is to perform a full scanperform a full scan
Very performance inefficientVery performance inefficient
There is a way for an application to “wait” on a There is a way for an application to “wait” on a directory and be told of notificationsdirectory and be told of notifications
An application can miss changes since it must An application can miss changes since it must specify a buffer to hold themspecify a buffer to hold them
Fundamentos de S.O. por Mtro. Abraham Jiménez Alfaro
10
Change LoggingChange Logging
With Windows 2000, NTFS introduces With Windows 2000, NTFS introduces the change log, which is a sparse the change log, which is a sparse metadata file that records file system metadata file that records file system events (not enabled by default)events (not enabled by default)
As the file exceeds its maximum onAs the file exceeds its maximum on diskdisk
28
As the file exceeds its maximum onAs the file exceeds its maximum on--disk disk size, NTFS frees the disk space for the size, NTFS frees the disk space for the oldest portions marking them emptyoldest portions marking them empty
An application uses Win32 APIs to read An application uses Win32 APIs to read eventsevents
The log file is shared, and generally large The log file is shared, and generally large enough that an application won’t miss enough that an application won’t miss changes even during heavy file system changes even during heavy file system activityactivity
PerPer--User Volume QuotasUser Volume Quotas
NTFS quotaNTFS quota--management support allows for permanagement support allows for per--user user specification of quota enforcementspecification of quota enforcement
Can be configured to log an event indicating the occurrence to Can be configured to log an event indicating the occurrence to the system Event Log if a user surpasses his warning limitthe system Event Log if a user surpasses his warning limit
If a user attempts to use more volume storage then her quotaIf a user attempts to use more volume storage then her quota
29
If a user attempts to use more volume storage then her quota If a user attempts to use more volume storage then her quota limit permits, NTFS can log an event to the system Event Log limit permits, NTFS can log an event to the system Event Log and fail the application file I/O that would have caused the and fail the application file I/O that would have caused the quota violation with a "disk full" error codequota violation with a "disk full" error code
User disk space is tracked on a perUser disk space is tracked on a per--volume basis by volume basis by summing the summing the logicallogical sizes of all the files and directories sizes of all the files and directories that have the user as the owner in their security that have the user as the owner in their security descriptorsdescriptors
Link TrackingLink Tracking
Several types of symbolic file links are used by layered applicationsSeveral types of symbolic file links are used by layered applicationsShell shortcuts allow users to place files in their shell namespace (on Shell shortcuts allow users to place files in their shell namespace (on their desktop, for example) that link to files located in the file system their desktop, for example) that link to files located in the file system namespacenamespace
Object linking and embedding (OLE) links allow documents from one Object linking and embedding (OLE) links allow documents from one application to be transparently embedded in the documents of otherapplication to be transparently embedded in the documents of other
30
application to be transparently embedded in the documents of other application to be transparently embedded in the documents of other applicationsapplications
In the past, these links were difficult to manageIn the past, these links were difficult to manageIf someone moved a link source (what a link points to), the link brokeIf someone moved a link source (what a link points to), the link broke
Windows now has a linkWindows now has a link--tracking service, TrkWks (it runs in tracking service, TrkWks (it runs in services.exe), tags link sources with a unique object ID services.exe), tags link sources with a unique object ID
NTFS can return the name of a file given a link, so if the link moves NTFS can return the name of a file given a link, so if the link moves the service can query each of a system’s volume for the object IDthe service can query each of a system’s volume for the object ID
A distributed linkA distributed link--tracking service, TrkSvr, works to track link source tracking service, TrkSvr, works to track link source movement across systemsmovement across systems
Fundamentos de S.O. por Mtro. Abraham Jiménez Alfaro
11
EncryptionEncryption
While NTFS implements security for files and directories, the While NTFS implements security for files and directories, the security is ineffective if the physical security of the computer is security is ineffective if the physical security of the computer is compromisedcompromised
Can install a parallel copy of WindowsCan install a parallel copy of Windows
NTFSDOSNTFSDOS
31
Encrypting File System (EFS)Encrypting File System (EFS)Like compression, its operation is transparentLike compression, its operation is transparent
Also like compression, encryption is a file and directory attributeAlso like compression, encryption is a file and directory attribute
Files that are encrypted can be accessed only by using the private Files that are encrypted can be accessed only by using the private key of an account's EFS private/public key pair, and private keys are key of an account's EFS private/public key pair, and private keys are locked using an account's passwordlocked using an account's password
While you might think that its implemented as a file system filter While you might think that its implemented as a file system filter driver, it’s a driver that’s tightly connected to NTFSdriver, it’s a driver that’s tightly connected to NTFS
POSIX SupportPOSIX Support
POSIX support requires two file system features:POSIX support requires two file system features:Primary group in security descriptorPrimary group in security descriptor
CaseCase--sensitive namessensitive names
32
DefragmentationDefragmentation
Fragmentation: A file is fragmented if its data Fragmentation: A file is fragmented if its data occupies discontiguous clustersoccupies discontiguous clusters
33
Fundamentos de S.O. por Mtro. Abraham Jiménez Alfaro
12
DefragmentationDefragmentationA common myth is that NTFS doesn’t fragment, but it A common myth is that NTFS doesn’t fragment, but it doesdoes
Defragmentation APIs have been present since NT 4Defragmentation APIs have been present since NT 4
Windows 2000 introduced a nonWindows 2000 introduced a non--schedulable graphical schedulable graphical defragmenterdefragmenter
A command line interface was added in Windows XPA command line interface was added in Windows XP
34
Compression and Sparse FilesCompression and Sparse Files
NTFS supports transparent compression of filesNTFS supports transparent compression of filesWhen a directories is marked compressed it means any files When a directories is marked compressed it means any files or subdirectories are marked compressedor subdirectories are marked compressed
Compression is performed on 16Compression is performed on 16--cluster blocks of a filecluster blocks of a file
U E l th t t l t fil ( tU E l th t t l t fil ( t
35
Use Explorer or the compact tool to compress files (compact Use Explorer or the compact tool to compress files (compact shows compression rations for compressed files)shows compression rations for compressed files)
Sparse files are an applicationSparse files are an application--controlled form of controlled form of compression that define parts of a file as empty compression that define parts of a file as empty –– those those areas don’t occupy any disk spaceareas don’t occupy any disk space
Applications use Windows APIs to define empty areasApplications use Windows APIs to define empty areas
NTFS File System DriverNTFS File System Driver
I/O manager
NTFS driver
Fault tolerant driver
Flush thelog file
Write thecache
Log fileservice
Log the transactionRead/write the file
Read/writea mirroredor stripedvolume
36
Cache manager
Virtual memorymanager
driver
Disk driver
Access the mappedfile or flush the cache
log file
Load datafrom disk
intomemory
Read/writethe disk
Fundamentos de S.O. por Mtro. Abraham Jiménez Alfaro
13
Components related to NTFSComponents related to NTFSCache ManagerCache Manager
System wide caching System wide caching for NTFS and other file systems driversfor NTFS and other file systems driversIncluding network file system drivers (server and redirectors)Including network file system drivers (server and redirectors)
Cached files are mapped into virtual memoryCached files are mapped into virtual memorySpecialized interface from Cache Manager to NT virtual memory Specialized interface from Cache Manager to NT virtual memory
37
managermanagerMemory manager calls NTFS to access disk driver and obtain fileMemory manager calls NTFS to access disk driver and obtain file
Log File ServiceLog File Service2 copies of transaction logs2 copies of transaction logsTransaction log is flushed to disk before writeTransaction log is flushed to disk before write--data is sent to diskdata is sent to diskCache manager performs actual flush operationCache manager performs actual flush operation
NTFS & File ObjectsNTFS & File Objects
File object
Handle tableProcess
Fil bj t
Streamcontrolblocks
FilecontrolblockObject
38
File objectData
attribute
User-definedattribute
Master filetable
NTFS datastructures
(used to managethe on-diskstructure) NTFS
database(on disk)
Objectmanager
datastructures
App accesses files asNT objects by handles.Object Manager and securitysubsystem verify access rights
NTFS OnNTFS On--Disk StructureDisk Structure
Volumes correspond to logical partitions on diskVolumes correspond to logical partitions on diskFault tolerant volumes may span multiple disksFault tolerant volumes may span multiple disks
Windows Windows 2000 2000 Disk Administrator utilityDisk Administrator utility
Volume consists of series of files + unallocated spaceVolume consists of series of files + unallocated spaceFAT volume: some areas specially formatted for file systemFAT volume: some areas specially formatted for file systemNTFS volume: all data are stored as ordinary filesNTFS volume: all data are stored as ordinary files
39
NTFS volume: all data are stored as ordinary filesNTFS volume: all data are stored as ordinary files
NTFS refers internally to clustersNTFS refers internally to clustersCluster factor: #sectors/cluster; varies with volume size; Cluster factor: #sectors/cluster; varies with volume size; (integral number of physical sectors; always a power of 2)(integral number of physical sectors; always a power of 2)
Logical Cluster Numbers (LCNs): Logical Cluster Numbers (LCNs): refer to physical locationrefer to physical locationLCNs are contiguous enumeration of all clusters on a volumeLCNs are contiguous enumeration of all clusters on a volume
Fundamentos de S.O. por Mtro. Abraham Jiménez Alfaro
14
NTFS Cluster SizeNTFS Cluster Size
Default cluster size is diskDefault cluster size is disk--size dependentsize dependent512 bytes for small disks (up to 512 MB)512 bytes for small disks (up to 512 MB)1 KB for disks up to 1 GB1 KB for disks up to 1 GB2 KB for disks between 1 and 2 GB2 KB for disks between 1 and 2 GB4 KB for disks larger than 2 GB4 KB for disks larger than 2 GB
40
Tradeoff: disk fragmentation versus wasted spaceTradeoff: disk fragmentation versus wasted spaceNTFS refers to physical locations via LCNsNTFS refers to physical locations via LCNs
Physical cluster = LCN * clusterPhysical cluster = LCN * cluster--factorfactor
Virtual Cluster Numbers (VCNs): Virtual Cluster Numbers (VCNs): Enumerates clusters belonging to a file; mapped to LCNsEnumerates clusters belonging to a file; mapped to LCNsLCNs are not necessarily physically contiguousLCNs are not necessarily physically contiguous
Master File TableMaster File TableAll data stored on a volume is contained All data stored on a volume is contained in a file in a file MFT: Heart of NTFS volume structureMFT: Heart of NTFS volume structure
Implemented as array of file recordsImplemented as array of file recordsOne row for each file on the volumeOne row for each file on the volume(including one row for MFT itself)(including one row for MFT itself)Metadata files store file system structure Metadata files store file system structure
MFT
MFT copy (partial)
Log file
Volume fileNTFS
41
yyinformationinformation(hidden files; $MFT; $Volume...)(hidden files; $MFT; $Volume...)More than one MFT record for highly More than one MFT record for highly fragmented filesfragmented filesNfi.exe Utility from OEM Support Tools Nfi.exe Utility from OEM Support Tools allows to dump MFT content allows to dump MFT content (see (see support.microsoft.com/support/support.microsoft.com/support/kb/articles/Q253/0/66.aspkb/articles/Q253/0/66.asp))
Volume file
Attribute def. table
Root directory
Bitmap file
Boot file
Bad cluster file
User files and dirs.
...
metadatafile
NTFS operationNTFS operation
Mounting a volumeMounting a volume1.1. NTFS looks in boot file for physical address of MFT ($MFT)NTFS looks in boot file for physical address of MFT ($MFT)2.2. 2nd entry in MFT points to copy of MFT ($MFTMirr)2nd entry in MFT points to copy of MFT ($MFTMirr)
used to locate metadata files if MFT is corruptedused to locate metadata files if MFT is corrupted
33 MFT t i MFT t i VCNMFT t i MFT t i VCN tt LCN i i fLCN i i f
42
3.3. MFT entry in MFT contains VCNMFT entry in MFT contains VCN--toto--LCN mapping infoLCN mapping info4.4. NTFS obtains from MFT addresses of metadata filesNTFS obtains from MFT addresses of metadata files
NTFS opens these filesNTFS opens these files
5.5. NTFS performs recovery operationsNTFS performs recovery operations6.6. File system is now ready for user accessFile system is now ready for user access
Fundamentos de S.O. por Mtro. Abraham Jiménez Alfaro
15
NTFS metadataNTFS metadata
NTFS writes to log file ($LogFile)NTFS writes to log file ($LogFile)Record all commands that change volume structureRecord all commands that change volume structure
Root directory:Root directory:When NTFS tries to open a file, it starts search in the root directoryWhen NTFS tries to open a file, it starts search in the root directoryOnce the file is found, NTFS stores the file‘s MFT file referenceOnce the file is found, NTFS stores the file‘s MFT file referenceSubsequent read/write ops. may access file‘s MFT record directlySubsequent read/write ops. may access file‘s MFT record directly
Bit fil ($Bit )Bit fil ($Bit )
43
Bitmap file ($Bitmap):Bitmap file ($Bitmap):stores allocation state volume; each bit represents one clusterstores allocation state volume; each bit represents one cluster
Boot file ($Boot):Boot file ($Boot):Stores bootstrap codeStores bootstrap codeHas to be located at special disk addressHas to be located at special disk addressRepresented as file by NTFS Represented as file by NTFS --> file ops. possible (!) (no editing)> file ops. possible (!) (no editing)
NTFS metadata (contd.)NTFS metadata (contd.)
BadBad--cluster file ($BadClus)cluster file ($BadClus)Records bad spots on the diskRecords bad spots on the disk
Volume file ($Volume)Volume file ($Volume)Contains: volume name NTFS versionContains: volume name NTFS version
44
Contains: volume name, NTFS versionContains: volume name, NTFS versionBit, which indicates whether volume is corruptedBit, which indicates whether volume is corrupted
Attribute Definition Table ($AttrDef)Attribute Definition Table ($AttrDef)Defines attribute types supported on the volumeDefines attribute types supported on the volumeIndicates whether they can be indexed, recovered, Indicates whether they can be indexed, recovered, etc.etc.
File Records & File Records & File Reference NumbersFile Reference Numbers
File on NTFS volume is identified by File on NTFS volume is identified by file referencefile referenceFile number == index in MFTFile number == index in MFT
Sequence number File number
063 47
45
Sequence number Sequence number –– used by NTFS for consistency checking;used by NTFS for consistency checking;incremented each time a reference is reincremented each time a reference is re--usedused
File Records:File Records:File is collection of attribute/value pairs (one of which is data)File is collection of attribute/value pairs (one of which is data)Unnamed data attributeUnnamed data attributeOther attributes: filename, time stamp, security descriptor,...Other attributes: filename, time stamp, security descriptor,...Each file attribute is stored as separate stream of bytes within a fileEach file attribute is stored as separate stream of bytes within a file
Fundamentos de S.O. por Mtro. Abraham Jiménez Alfaro
16
File Records (contd.)File Records (contd.)
NTFS doesn‘t read/write files:NTFS doesn‘t read/write files:It reads/writes attribute streamsIt reads/writes attribute streamsOperations: create, delete, read (byte range), write (byte range)Operations: create, delete, read (byte range), write (byte range)Read/write normally operate on unnamed data attributeRead/write normally operate on unnamed data attribute
46
FilenameStandard
informationSecurity
descriptor Data
Master File Table
MFT record for a small file
Windows optimization: Security descriptorsare stored in a central file and referenced by each file record (saves disk space)
Standard Attributes for NTFS FilesStandard Attributes for NTFS Files
AttributeAttribute DescriptionDescription
Standard Standard informationinformation
File attributes: readFile attributes: read--only, archive, etc; time stamps; only, archive, etc; time stamps; creation/modification time; hard link countcreation/modification time; hard link count
FilenameFilename Name in Unicode characters; multiple filename Name in Unicode characters; multiple filename attributes possible (POSIX links!!); short names forattributes possible (POSIX links!!); short names for
47
attributes possible (POSIX links!!); short names for attributes possible (POSIX links!!); short names for access by MSaccess by MS--DOS and 16DOS and 16--bin Win applicationsbin Win applications
Security descriptorSecurity descriptor Specifies who owns the file and who can access itSpecifies who owns the file and who can access it
datadata Contents of the file; a file has one default unnamed data Contents of the file; a file has one default unnamed data attribute; directory has no default data attrib.attribute; directory has no default data attrib.
Index root, indexIndex root, index Three attributes used to implement filename allocation, Three attributes used to implement filename allocation, bitmap index for large directories (dirs. only)bitmap index for large directories (dirs. only)
Attribute listAttribute list List of attributes that make up the file and first reference List of attributes that make up the file and first reference of the MFT record in which the attribute is located (for of the MFT record in which the attribute is located (for files which require multiple MFT file records)files which require multiple MFT file records)
Attributes (contd.)Attributes (contd.)
Each attribute in a file record has a name and a valueEach attribute in a file record has a name and a valueNTFS identifies attributes:NTFS identifies attributes:
Uppercase name starting with $: $FILENAME, $DATAUppercase name starting with $: $FILENAME, $DATA
Attribute‘s value: Byte streamAttribute‘s value: Byte streamThe filename for $FILENAMEThe filename for $FILENAME
48
$$The data bytes for $DATAThe data bytes for $DATA
Attribute names correspond to numeric typecodesAttribute names correspond to numeric typecodesFile attributes in an MFT record are ordered by typecodesFile attributes in an MFT record are ordered by typecodes
Some attribute types may appear more than once (e.g. Filename)Some attribute types may appear more than once (e.g. Filename)
Fundamentos de S.O. por Mtro. Abraham Jiménez Alfaro
17
FilenamesFilenamesPOSIX:POSIX:
CaseCase--sensitive, trailing periods & spacessensitive, trailing periods & spacesNTFS namespace equiv. to POSIX spaceNTFS namespace equiv. to POSIX space
Win32:Win32:Long filenames, unicode namesLong filenames, unicode namesMultiple dots, embedded spaces, beginning Multiple dots, embedded spaces, beginning dotsdots
Namespaces
49
MSMS--DOS:DOS:8.3 names, case does not matter8.3 names, case does not matter
NTFS generates MSNTFS generates MS--DOS names for Win32 DOS names for Win32 files automaticallyfiles automatically
Fully functional aliases for NTFS namesFully functional aliases for NTFS namesStored in same directory as long names; dir /xStored in same directory as long names; dir /x
POSIXsubsystem
Win32subsystem
MS-DOS Win16 clients
MSMS--DOS filenames in NTFSDOS filenames in NTFS
NTFS name and MSNTFS name and MS--DOS name are stored in same file record and refer to same DOS name are stored in same file record and refer to same filefile
Renaming changes both filenamesRenaming changes both filenames
NTFS filenameStandard info MS-DOS filename Security desc. Data
MFT file record with MS-DOS filename attribute
50
Renaming changes both filenamesRenaming changes both filenamesOpen, read, write, delete work with both names equallyOpen, read, write, delete work with both names equally
POSIX hardlinks are implemented in similar wayPOSIX hardlinks are implemented in similar wayDeleting a file with multiple names only decreases link countDeleting a file with multiple names only decreases link count
Generation of MSGeneration of MS--DOS names:DOS names:1.1. Remove all illegal chars; remove all but one period; truncate to 6 charsRemove all illegal chars; remove all but one period; truncate to 6 chars2.2. Append ~1 to name; truncate extension to 3 chars; all uppercaseAppend ~1 to name; truncate extension to 3 chars; all uppercase3.3. Increment ~1 if filename duplicates an existing name in directoryIncrement ~1 if filename duplicates an existing name in directory
Resident & Nonresident AttributesResident & Nonresident Attributes
Small files:Small files:All attributes and values fit into MFTAll attributes and values fit into MFT
Attribute with value in MFT is called „resident“Attribute with value in MFT is called „resident“
All attributes start with header (always resident)All attributes start with header (always resident)
Header contains offset to attr. value and length of valueHeader contains offset to attr. value and length of value
51
NTFS filenameStandard info Security desc. Data
MYFILE.DAT„RESIDENT“
Offset: 8hLength: 14h
header
value
Fundamentos de S.O. por Mtro. Abraham Jiménez Alfaro
18
Attributes (contd.)Attributes (contd.)
Small directory:Small directory:index root attribute contains index of file references index root attribute contains index of file references for files and subdirectoriesfor files and subdirectories
NTFS filenameStandard info Index rootSecurity desc. Empty
52
Index of files
file1, file2, file3,...
MFT file record for a small directory
• If file attribute does not fit into MFT:• NTFS allocates separate cluster (run, extent) to store the values• NTFS allocates additional runs if an attribute‘s value later grows• Those attributes are called „non-resident“• Header of non-resident attribute contains location info
Large files & directoriesLarge files & directories
Only attributes that can grow can be nonOnly attributes that can grow can be non--residentresident
NTFS filenameStandard info HPFS extended attr.Security desc. Data
MFT record for large file with 2 data runs
53
Filename & standard info are always residentFilename & standard info are always resident
Index of files for directories forms B+ tree Index of files for directories forms B+ tree
NTFS filenameStandard info Index rootSecurity desc. Bitmap
file4, file8
MFT file record for a large directorywith nonresident filename index
Index allocation
file1, file2, file3 file5, file6
Index of filesVCN-to-LCN
mappings
Large files (contd.)Large files (contd.)NTFS keeps track of runs by means of VCNNTFS keeps track of runs by means of VCN(Virtual Cluster Numbers)(Virtual Cluster Numbers)
Logical Cluster Numbers represent an entire volumeLogical Cluster Numbers represent an entire volumeVirtual Cluster Numbers represent clusters belonging to one fileVirtual Cluster Numbers represent clusters belonging to one fileAttribute lists may extend over multiple runs (not only data)Attribute lists may extend over multiple runs (not only data)
54
NTFS filenameStandard info Security desc. Data
DataData
VCN-to-LCN mappings for anonresident data attribute
StartinStarting VCNg VCN
StartinStartingg LCNLCN
Number of Number of clustersclusters
00 13551355 44
44 15881588 44
VCN 0 1 2 3
LCN 1355 1356 1357 1358DataData
VCN 4 5 6 7
LCN 1588 1589 1590 1591
Fundamentos de S.O. por Mtro. Abraham Jiménez Alfaro
19
Data CompressionData Compression
NTFS supports compressionNTFS supports compression
PerPer--file, perfile, per--directory, perdirectory, per--volume basisvolume basis
NTFS compression is performed on user data only, NTFS compression is performed on user data only, not NTFS metadatanot NTFS metadata
55
Inspect files/volume via Winndows API:Inspect files/volume via Winndows API:GetVolumeInformation(), GetCompressedFileSize()GetVolumeInformation(), GetCompressedFileSize()
Change settings for files/directories:Change settings for files/directories:DeviceIoControl() DeviceIoControl()
with flags with flags FSCTL_GET_COMPRESSION, FSCTL_SET_COMPRESSIONFSCTL_GET_COMPRESSION, FSCTL_SET_COMPRESSION
Compression of sparse filesCompression of sparse files
NTFS zeroes all file contents on creation (C2 req.)NTFS zeroes all file contents on creation (C2 req.)Many sparse files contain large amount of zeroMany sparse files contain large amount of zero--bytesbytes
These bytes occupy space on disk These bytes occupy space on disk –– unless files are compressedunless files are compressed
NTFS filenameStandard info Security desc. Data
56
NTFS filenameStandard info Security desc. Data
DataData
StartinStartingg VCNVCN
StartinStarting LCNg LCN
Number of Number of clustersclusters
00 13551355 1616
3232 15881588 1616
4848 9696 1616
128128 324324 1616
VCN 0 1 2 3 .... 15
LCN 1355 1356 1357 1358 .... 1370
DataDataVCN 32 33 34 35 ... 47
LCN 1588 1589 1590 1591 .... 1603Certain ranges of VCNs have noCertain ranges of VCNs have nodisk allocation (16disk allocation (16--31, 6431, 64--127)127)
Compressing Nonsparse DataCompressing Nonsparse DataNTFS divides the file‘s unprocessed data intoNTFS divides the file‘s unprocessed data intocompression unitscompression units 16 clusters long16 clusters longCertain sequence might not compress muchCertain sequence might not compress much
NTFS determines for each compression unit whether it will shrink NTFS determines for each compression unit whether it will shrink by at least on clusterby at least on clusterIf data does not compress, NTFS allocates cluster space and simply If data does not compress, NTFS allocates cluster space and simply writes datawrites data
57
If data compresses at least one cluster, NTFS allocates only the If data compresses at least one cluster, NTFS allocates only the clusters needed for compressed dataclusters needed for compressed data
When writing data, NTFS ensures that each run begins on virtual 16When writing data, NTFS ensures that each run begins on virtual 16--cluster boundarycluster boundary
NTFS reads/writes at least one compression unit when accessing a NTFS reads/writes at least one compression unit when accessing a filefileReadRead--ahead + asynch. decompression improves performanceahead + asynch. decompression improves performance
Fundamentos de S.O. por Mtro. Abraham Jiménez Alfaro
20
Data runs of a compressed fileData runs of a compressed file0 15
19 20 21 22
VCN
LCN
Compressed data
16 31
23 24 25 26
Compressed data
32 4727 28 29 30
58
97 98 99 100 101 102
Noncompressed data
48 63
Compressed data
103 104 105 106 107 108 109 110 111 112
113 114 115 116 117 118 119 120 121 122
StartinStarting VCNg VCN
StartinStarting LCNg LCN
No. of No. of clusterclusterss
00 1919 441616 2323 883232 9797 16164848 113113 1010
MFT record for a compressed file
Windows Windows -- NTFS ExtensionsNTFS Extensions
Disk quotas on perDisk quotas on per--user basesuser basesSecurity descriptors (ACLs) can be stored once but referenced in Security descriptors (ACLs) can be stored once but referenced in multiple filesmultiple filesNative support for properties (OLE), including indexingNative support for properties (OLE), including indexingReparse pointsReparse points implementation of symbolic linksimplementation of symbolic links
59
Reparse points Reparse points –– implementation of symbolic linksimplementation of symbolic linksMount points for arbitrary file system volumesMount points for arbitrary file system volumes
Support for sparse filesSupport for sparse filesDistributed link tracking (via global object Ids)Distributed link tracking (via global object Ids)
Renaming the target file will no longer break links Renaming the target file will no longer break links (shortcuts...)(shortcuts...)
Add disk space to an NTFS volume without rebootAdd disk space to an NTFS volume without rebootNo decompressing when transmitting files over networkNo decompressing when transmitting files over network
File System Driver ArchitectureFile System Driver Architecture
Local File System Drivers (Local FSDs):Local File System Drivers (Local FSDs):Ntfs.sys, Fastfat.sys, Udfs,sys, Cdfs,sysNtfs.sys, Fastfat.sys, Udfs,sys, Cdfs,sysResponsible for registering with the I/O manager and volume Responsible for registering with the I/O manager and volume recognition/integrity checksrecognition/integrity checksFSD creates device objects for each mounted file system formatFSD creates device objects for each mounted file system formatI/O k ti b t l ‘ d i bj tI/O k ti b t l ‘ d i bj t
60
I/O manager makes connection between volume‘s device objects I/O manager makes connection between volume‘s device objects (Created by storage device) and the FSD‘s device object(Created by storage device) and the FSD‘s device objectLocal FSDs use cache manager to improve file access performanceLocal FSDs use cache manager to improve file access performanceDismount operation permits the system to disconnect FSD from Dismount operation permits the system to disconnect FSD from volume object volume object
When media is changed or when application requires raw When media is changed or when application requires raw device accessdevice accessI/O manager reinitiated volume mount operation on next access I/O manager reinitiated volume mount operation on next access to mediato media
Fundamentos de S.O. por Mtro. Abraham Jiménez Alfaro
21
Layered Drivers Layered Drivers --I/O System ArchitectureI/O System Architecture
Environmentsubsystem or
DLL
Services
I/O manager
1)1)Call I/O serviceCall I/O service
2)The I/O manager creates an IRP, 2)The I/O manager creates an IRP, initializes first stack location and initializes first stack location and calls file system drivercalls file system driver
User modeUser modeKernel modeKernel mode
7)Return I/O pending status7)Return I/O pending status
61
calls file system drivercalls file system driver
3)File system driver fills in a 23)File system driver fills in a 2ndnd
IRP stack location and callsIRP stack location and callsthe disk driverthe disk driver
IRP
File systemdriver
Diskdriver
IRP
4)Send IRP data to device4)Send IRP data to device(or queue IRP), and return(or queue IRP), and return
6)Return I/O pending status6)Return I/O pending status
5)Return I/O pending status5)Return I/O pending status
Optimization: associated IRPsOptimization: associated IRPsmay work in parallel on a singlemay work in parallel on a singleI/O requestI/O request
File System Driver Architecture File System Driver Architecture (contd.)(contd.)
Remote File System Drivers (Remote FSDs):Remote File System Drivers (Remote FSDs):ClientClient--side FSD translates I/O requests from side FSD translates I/O requests from applications into network file system protocol commandsapplications into network file system protocol commandsServerServer--side FSD listens for network commands and side FSD listens for network commands and issues I/O requests to local FSDissues I/O requests to local FSD
Application
I/O manager
user mode
kernel mode
62
Windows clientWindows client--side remote FSD: LANMan Redirectorside remote FSD: LANMan RedirectorImplemented as port/miniport driver Implemented as port/miniport driver Includes Windows service WorkstationIncludes Windows service Workstation
ServerServer--side FSD server: LANMan Serverside FSD server: LANMan ServerIncludes Windows service ServerIncludes Windows service ServerCIFS CIFS –– common internet file system (enhancement common internet file system (enhancement of Server Message Block protocol)of Server Message Block protocol)
Remote FSD(redirector)
Local FSD
Remote FSD(server)
Storage devicedrivervolume
client server
Windows Remote File Drivers: Windows Remote File Drivers: Server Message Block (SMB) protocolServer Message Block (SMB) protocol
SMB is a client server, SMB is a client server, requestrequest--response protocol.response protocol.
Addl. info athttp://anu.samba.org/cifs/docs/what is smb html
63
The only exception to the requestThe only exception to the request--response nature of SMB response nature of SMB is when the client has requested opportunistic locks (oplocks) and is when the client has requested opportunistic locks (oplocks) and the server subsequently has to break an already granted oplock the server subsequently has to break an already granted oplock because another client has requested a file open with a mode that because another client has requested a file open with a mode that is incompatible with the granted oplock. is incompatible with the granted oplock.
In this case, the server sends an unsolicited message to the client In this case, the server sends an unsolicited message to the client signaling the oplock break.signaling the oplock break.
cifs/docs/what-is-smb.html
Fundamentos de S.O. por Mtro. Abraham Jiménez Alfaro
22
SMB and the OSI modelSMB and the OSI model
64
Clients connect to servers using TCP/IP (actually NetBIOS over Clients connect to servers using TCP/IP (actually NetBIOS over TCP/IP as specified in RFC1001 and RFC1002), NetBEUI or TCP/IP as specified in RFC1001 and RFC1002), NetBEUI or IPX/SPX. IPX/SPX. SMB was also sent over the DECnet protocol. SMB was also sent over the DECnet protocol. Digital (now HP) did this for their PATHWORKS productDigital (now HP) did this for their PATHWORKS product
SMB characteristicsSMB characteristics
NetBIOS NamesNetBIOS NamesIf SMB is used over TCP/IP, DECnet or NetBEUI, then NetBIOS names must If SMB is used over TCP/IP, DECnet or NetBEUI, then NetBIOS names must be used in a number of cases. be used in a number of cases. NetBIOS names are up to 15 characters long, and are usually the name of the NetBIOS names are up to 15 characters long, and are usually the name of the computer that is running NetBIOS. computer that is running NetBIOS. Microsoft, and some other implementers, insist that NetBIOS names be in Microsoft, and some other implementers, insist that NetBIOS names be in upper case, especially when presented to servers as the CALLED NAME. upper case, especially when presented to servers as the CALLED NAME.
P t l f ti lit (C t l)P t l f ti lit (C t l)
65
Protocol functionality (Core protocol):Protocol functionality (Core protocol):connecting to and disconnecting from file and print shares connecting to and disconnecting from file and print shares opening and closing files opening and closing files opening and closing print files opening and closing print files reading and writing files reading and writing files creating and deleting files and directories creating and deleting files and directories searching directories searching directories getting and setting file attributes getting and setting file attributes Locking and unlocking byte ranges in files Locking and unlocking byte ranges in files
SMB characteristics (contd.)SMB characteristics (contd.)SecuritySecurity
The SMB model defines two levels of security: The SMB model defines two levels of security: Share level. Share level.
Each share can have a password, and a client only needs that Each share can have a password, and a client only needs that password to access all files under that share. password to access all files under that share. This was the first security model that SMB had and is the only security This was the first security model that SMB had and is the only security
66
y y yy y ymodel available in the Core and CorePlus protocols. model available in the Core and CorePlus protocols.
User Level.User Level.Protection is applied to individual files in each share and is based on Protection is applied to individual files in each share and is based on user access rights. user access rights. Each user (client) must log in to the server and be authenticated by the Each user (client) must log in to the server and be authenticated by the server.server.When it is authenticated, the client is given a UID which it must present When it is authenticated, the client is given a UID which it must present on all subsequent accesses to the server. on all subsequent accesses to the server. This model has been available since LAN Manager 1.0. This model has been available since LAN Manager 1.0.
Fundamentos de S.O. por Mtro. Abraham Jiménez Alfaro
23
SMB Clients and Servers SMB Clients and Servers
Clients:Clients:Included in WfW 3.x, Win 95, Win98, Win ME and Windows Included in WfW 3.x, Win 95, Win98, Win ME and Windows NT/2000/XP/Server 2003/Vista. NT/2000/XP/Server 2003/Vista. smbclient from Samba, smbfs for Linux, SMBlib smbclient from Samba, smbfs for Linux, SMBlib
Servers:Servers:Microsoft Windows for Workgroups 3.x, Win95, Win98, Win ME, Microsoft Windows for Workgroups 3.x, Win95, Win98, Win ME, Wi dWi d NT/2000/XP/S 2003/Vi tNT/2000/XP/S 2003/Vi t
67
Windows Windows NT/2000/XP/Server 2003/VistaNT/2000/XP/Server 2003/VistaSambaSamba (Linux, Solaris, SunOS, HP(Linux, Solaris, SunOS, HP--UX, ULTRIX, DEC OSF/1, Digital UX, ULTRIX, DEC OSF/1, Digital UNIX, Dynix (Sequent), IRIX (SGI), SCO Open Server, DGUNIX, Dynix (Sequent), IRIX (SGI), SCO Open Server, DG--UX, UX, UNIXWARE, AIX, BSDI, NetBSD, NEXTSTEP, A/UX)UNIXWARE, AIX, BSDI, NetBSD, NEXTSTEP, A/UX)The PATHWORKS family of servers from Digital The PATHWORKS family of servers from Digital LAN Manager for OS/2, SCO, etc LAN Manager for OS/2, SCO, etc VisionFS from SCO VisionFS from SCO Advanced Server for UNIX from AT&T (NCR?) Advanced Server for UNIX from AT&T (NCR?) LAN Server for OS/2 from IBM LAN Server for OS/2 from IBM
Further ReadingFurther Reading
Mark E. Russinovich and David A. Solomon, Mark E. Russinovich and David A. Solomon, Microsoft Windows Internals, 4th Edition, Microsoft Microsoft Windows Internals, 4th Edition, Microsoft Press, 2004.Press, 2004.
68
File Systems supported by Windows (from pp. 689)File Systems supported by Windows (from pp. 689)File System Driver Architecture (from pp. 694)File System Driver Architecture (from pp. 694)
NTFS Design Goals and FeaturesNTFS Design Goals and Features (from pp. 717)(from pp. 717)NTFS OnNTFS On--Disk Structure (from pp. 732)Disk Structure (from pp. 732)
Source Code ReferencesSource Code References
Windows Research Kernel sources do not Windows Research Kernel sources do not include NTFSinclude NTFS
A raw file system driver is included in A raw file system driver is included in \\basebase\\ntosntos\\rawraw
69
\\basebase\\ntosntos\\rawraw
Also see Also see \\basebase\\ntosntos\\fstrl (File System Runfstrl (File System Run--Time Time Library)Library)