of 57/57
Unix File Systems Computer Forensics COEN 252

Computer Forensics COEN 252. File systems can be extent-based ◦ E.g. NTFS ◦ Storage space is allocated in extents, large sets of contiguous blocks ◦

  • View

  • Download

Embed Size (px)

Text of Computer Forensics COEN 252. File systems can be extent-based ◦ E.g. NTFS ◦ Storage space is...

Unix File Systems

Unix File SystemsComputer ForensicsCOEN 252Unix File SystemFile systems can be extent-basedE.g. NTFSStorage space is allocated in extents, large sets of contiguous blocksMetadata often located in a single, central database such as the MFT in NTFSOr, file systems can be inode-basedUNIX and derivativesMore flexible, but more overhead Unix file systems usually come in two layers:Disk layoutVeneer layerAllows splitting kernel into file-system dependent and non-file system dependent part

Allocation of disk space to files is done with blocks.Choice of block size is fundamentalBlock size small: Needs to store much location informationBlock size large: Disk capacity wasted in partially used blocks (at the end of file)Typical Unix block sizes are 4KB and 8KBDisk BlocksInodes are fixed sized metadata describing the layout of a file Inode structure:i_mode (directory IFDIR, block special file (IFBLK), character special file (IFCHR), or regular file (IFREG)i_nlink i_uid (user id)i_gid (group id)i_size (file size in bytes)i_addr (an array that holds addresses of blocks)i_mtime (modification time & date)i_atime (access time & date)

InodesMetadata in Inode is space-limitedNumber of block addresses in a single inode only suffices for small filesUse (single and double) indirect inodes to find space for all blocks in a fileInodesInodes

Unix File SystemUnix File Systems use inode numbers to refer internally to files.Classical Unix used a file table to mediate between users and their open files.File table have references to the inodes of open files.General designNeed to find free blocks: e.g. block bitmapNeed to find free inodes: e.g. inode bitmapInodes are located often in a designated area of the disk partition. (Inode Table)Data blocks often located in a designated area of the disk partition.Since this can create long actuator movements between metadata lookup and data block lookup, use Block Groups in EXT2Divide partition in groups, where each group has this structureUnix File SystemOn-Disk Layout.Superblock contains data on the file system.

Unix File SystemLanguage Chart

Unix File SystemsDisk Layout not uniform.Ext2 (Linux) file system layout.

EXT DetailsOverview

EXT DetailsExt superblock:Located 1024 B from start of the file system.Backups of superblock are usually stored in the first block of each block group.Contains basic information:Block sizeTotal number of blocksNumber of reserved blocksEXT Details: EXT SuperBlockByteDescription0-3BNumber of inodes in file system4-7BNumber of blocks in file system8-11BNumber of blocks reserved to prevent file system from filling up12-15BNumber of unallocated blocks.16-19BNumber of unallocated inodes.20-23BBlock where block group 0 starts24-27BBlock size. (Saved as the number of places to shift 1,024 to the left).28-31BFragment size. (Saved as the number of places to shift 1,024 to the left).32-35BNumber of blocks in each group.36-39BNumber of fragments in each block group40-43BNumber of inodes in each block group.44-47BLast mount time.48-51BLast written time.52-53BCurrent mount time.54-55BMaximum mount countEXT Details: EXT SuperBlockByteDescription56-57BSignature 0xef5358-59BFile system state60-61BError handling method62-63BMinor Version64-67BLast consistency check time.68-71BInterval between forced consistency checks72-75BCreator OS76-79BMajor version80-81BUID that can use reserved blocks.82-83BGID that can use reserved blocks.84-87BFirst non-reserved inode in file system88-89BSize of each inode structure90-91BBlock group that this superblock is part of (if this is the backup copy)92-95BCompatibility feature flags96-99BIncompatibile feature flagsEXT Details: EXT SuperBlockByteDescription100-103Read only feature flags104-119File system ID120-135Volume name136-199Path were last mounted on200-203Algorithm usage bitmap204Number of blocks to preallocate for files.205Number of blocks to preallocate for directories208-223Journal ID224-227Journal Inode228-231Journal device232-235Head of orphan inode list236-1023UnusedEXT DetailsGroup Descriptor TableIn the block following superblockDescribes all block groups in the systemEXT DetailsGroup Descriptor Table Entries0-3 starting block address of block bitmap4-7 starting block address of inode bitmap8-11 starting block address of inode table12-13 number of unallocated blocks in group14-15 number of unallocated inodes in group16-17 number of directories in groupEXT DetailsTotal number of blocks includes Reserved area and all groups.Blocks per group determines size of group.

EXT DetailsBlock Group Descriptor TableLocated in block following the superblockBasic layout of a block group:

Block bitmap takes exactly one block.Inode bitmap manages allocation status of inodes.

EXT DetailsNumber of blocks = bits in bitmap = bits in a block (namely the bitmap block).Size of block determines number of blocks in a block group!Inode bitmap starting address contained in block descriptor table.Size of Inode bitmap given by #inodes per group divided by 8.Block group descriptor table gives starting block for inode table.Size of inode table = 128B * number of inodes.EXT DetailsBoot CodeIf exists, will be in the 1024B before the superblock.Many Linux systems have a boot loader in the MBR.In this case, there will be no additional boot code.EXT DetailsData stored in blocks.Typical block sizes are 1,024B; 2048B; or 4096BAllocation status of a block determined by the groups block bitmap

EXT DetailsAnalyzing content:Locate any blockRead its contentsDetermine its allocation statusFirst block starts in the first sector of the file system. Block size is given by superblock.EXT DetailsDetermining allocation status:Determine the block group to which the block belongs.Locate the groups entry in the group descriptor table to find out where the block bitmap is stored.Process the block bitmap to find out whether this block is allocated or not.EXT DetailsTo find all unallocated blocks:Systematically go through the block bitmap and look for 0 bit entries.Status of reserved sectors at the beginning is less clear since there are no bitmap entries for them.EXT DetailsMetadata is stored in the inode data structure.All inodes have the same size specified in the superblock.Inodes have addresses starting with 1.Inodes in each group are in a table with address given by the group descriptor.group = (inode 1) / INODES_PER_GROUPEXT DetailsInodes 1 10 are typically reserved.Superblock has the value of the first non-reserved inode.Inode 1 keeps track of bad blocks.Inode 2 contains the root directoryJournal uses Inode 8First user file in Inode 11, typically for lost+foundEXT DetailsInode can store the address of the first 12 data blocks of a file.For larger files, we use double indirect and triple indirect block pointers

EXT DetailsAllocation AlgorithmsBlock group:Non-directories are allocated in the same block group as parent directory, if possible.Directory entries are put into underutilized groups.Contents of allocated inode are cleared and MAC times set to the current system time.Deleted files have their inode link value decremented. If the link value is zero, then it is unallocated. If a process still has the file open, it becomes an orphan file and is linked to the superblock.

EXT DetailsInode Structure0-1 File Mode (type and permissions)2-3 Lower 16 bits of user ID4-7 Lower 32 bits of size in bytes8-11 Access Time12-15 Change Time16-19 Modification Time20-23 Deletion Time

EXT DetailsInode Structure24-25 Lower 16 bits of group ID26-27 Link count28-31 Sector count32-35 Flags36-39 Unused40 87 12 direct block pointers88-91 1 single indirect block pointer92-95 1 double indirect block pointerEXT DetailsInode Structure96-99 1 indirect block pointer100 103 Generation number (NFS)104 107 Extended attribute block108 111 Upper 32 bits of size / Directory ACL112 115 Block address of fragment116 Fragment index in block

EXT DetailsInode Structure117 Fragment Size118 119 Unused120 121 Upper 16 bits of user ID122 123 Upper 16 bits of group ID124 127 UnunsedEXT DetailsInode StructurePermission flags of the file mode field0x001 Other execute permission0x002 Other write permission0x004 Other read permission0x008 Group execute permission0x010 Group write permission0x020 Group read permission0x040 User execute permission0x080 User write permission0x100 User read permissionEXT DetailsInode StructureFlags for bits 9 11 of the file mode field0x200 Sticky bit (save text image)0x400 Set Group ID0x800 Set User IDEXT DetailsInode StructureFile mode field These are values not flags0x1000 FIFO0x2000 Character device0x4000 Directory0x6000 Block device0x8000 Regular file0xA000 Symbolic link0xC000 Unix socket

EXT DetailsTime ValuesAre stored as seconds since January 1, 1970, Universal Standard TimeGet ready for the Year 2038 problem.EXT DetailsLinux updates (in general)A-time, when the content of file / directory is read.For a file:If a process reads the file.When the file is copied.When the file is moved to a new volume.But not if the file is moved within a volume.For a directoryWhen a directory listing is done.When a file or subdirectory is opened.EXT DetailsLinux updates (in general)M-time, when the content of file / directory is modified.For a file:If file contents change.For a directoryWhen a file is created or deleted inside the directory.When a file is copied, the M-time is not changed.However, when a file is copied to a network drive, the network server might consider it a new file and reset the M-time to the current time.EXT DetailsLinux updates (in general)C-time corresponds to the last inode change.When file / directory is created.When permissions change.When contents change.D-time is set only if a file is deleted.When a file is created, then D-time is set to 0.EXT DetailsUnallocated inodes contain temporary data.M-, C-, D-time values might show when the file was deleted.Users can change A- and M-time with the touch command.EXT DetailsLinux fills slack space (unused bytes of block) with zeroes.Data from deleted files will only exist in unallocated blocks.File size and allocated blocks will probably be wiped from unallocated inode entries.EXT DetailsLinux file hiding technique:Have a process open a file for reading or writing.Delete the file name.Link count for the inode is zero, but inode is not unallocated.The file system should add the orphan inode to a list in the superblock.EXT DetailsDirectory StructureA directory entry consists ofA variable length name.The inode number with the metadata of the entry.The original byte allocation is as follows:0-3 Inode value4-5 Length of entry6-7 Length of name8- Name in ASCIIEXT DetailsDirectory StructureThe improved byte allocation is as follows:0-3 Inode value4-5 Length of entry6 Length of name (up to 255 now)7 File type0 unknown1 regular file2 directory3 character device4 block device5 FIFO6 Unix Socket7 Symbolic link8- Name in ASCIIEXT DetailsThe record entry length allows the file system to find the next entry in a directory.If a directory entry is deleted, then the previous entries length is increased.

EXT DetailsWhen FS is created, a Linux user can decide to use hash trees instead.Directory entries are no longer in an unsorted list.A directory using a hash tree contains multiple blocks, the nodes in the tree.First block contains the . and .. directory entries.EXT DetailsLinksHard link: an additional file/directory name.Implemented by another directory entry pointing to the same inode.Link count in inode is incremented.Directory link count is 2 + number of subdirectoriesFile system cannot distinguish between the first and the second name of file. EXT DetailsLinksSoft link: an additional file/directory name.Implemented by a directory entry pointing to another inode.Inode points to a file, that contains the path to the original file.

EXT DetailsMount Point ExampleFS1 has directory /dir1.If FS2 is mounted on /dir1 and a user changed into /dir1, then only FS2 is shown.

EXT DetailsEXT hiding technique uses a directory (containing the files to be hidden) as a mount point.Forensics tools tend to not give mount points.Consequentially, this hiding technique falls flat for forensics tools.EXT3EXT3 journal located at inode 8 (typically)Journal records transactionsBlock updates about to occur.Log of update after the fact.Two modes:Only metadata blocks are journaled.Metadata and data blocks are journaled.

EXT DetailsExt3 Journal gives additional information about recent events.


Maintaining consistency in file systems after a crash is difficult1970s: Possible to read complete file system metadata and piece it togetherLog Structured File System (Osterhoot, Mendelbaum, 1990)Maintain all data, including metadata in a single logOnly write to the end of the logCurrently not used, but idea appears always attractiveJournaling File SystemMaintain a log of all metadata changes.After a crash, find point in journal where the file system was consistent, then roll forward from thereForensic potential of journaling file systems has not yet been realized

Journaling File SystemsThe Coroners Toolkit (TCT)http://www.porcupine.org/forensics/tct.htmlgrave-robberpcat, ils, icat, fileunrm and lazarusMactimeTCTUtils (Brian Carrier)http://www.digital-evidence.org/index.htmlSleuth Kit (Brian Carrier)http://www.sleuthkit.org/sleuthkit/desc.phpAutopsy Forensics BrowserGUI for TCT and TCTUtilshttp://www.sleuthkit.org/autopsy/desc.phpCommercial tools (Encase, FTK, )

Unix / Linux Forensic Tools