Upload
others
View
14
Download
0
Embed Size (px)
Citation preview
SM L08 -Enhanced Primary Storage Compression and De-duplication on UNIX and Linux with Storage Foundation 6.0 Lab Student’s Guide
Introduction
This hands on lab contains exercises that will allow the student to carry out new features in
Storage Foundation 6.0 for Unix and Linux. At the end of
these labs,
participants
should be
able to do
the following:
Identify new CLIs, layouts, and reporting features within Storage Foundation 6.0.
Understand how to enable, tune, report, and run De-duplication within a file system.
Understand how to enable, tune, report, and run Compression within a file system.
Identify use-cases, architectures, and storage optimization opportunities on primary storage.
Run and interpret the new defragmentation options within file systems, files, and whitespace.
Lab Agenda Lab No. Time Lab Name
Lab 1 5 Identify new disk layout version in SF 6.0
Lab 2 10 Create a new file system
Lab 3 25 Compression Exercise
Lab 4 25 De-duplication Exercise
Lab 5 10 Human Friendly CLI Exercise
Lab Layout
Storage Foundation 6.0 Server Name: rhel61 OS: RedHat Enterprise Linux 6.1 Credentials: root/rootroot
Lab Student’s Guide
Symantec Vision 2012 – MGM, Las Vegas SM L08 - Lab Student’s Guide / Last Update: Sunday, May 06, 2012 2
Lab Exercise 1 Lab: Identify new disk layout version in SF 6.0 5 minutes
1. We’ll start by calling out the new disk layout in SF 6.0 that is a requirement for Compression and De-duplication features. While Storage Foundation 6.0 can run on previous layouts, version 170 is required to enable new features.
2. After logging into the console, right-click and open a terminal
3. Display the created disk and diskgroup Command: vxdisk list
4. Display the disk layout of the existing disk, sf60dg. The layout version used by Storage Foundation 6.0, thus indicating compatibility with Compression and De-duplication is: 170 Command: vxprint -l
Lab Student’s Guide
Symantec Vision 2012 – MGM, Las Vegas SM L08 - Lab Student’s Guide / Last Update: Sunday, May 06, 2012 3
Lab Student’s Guide
Symantec Vision 2012 – MGM, Las Vegas SM L08 - Lab Student’s Guide / Last Update: Sunday, May 06, 2012 4
Lab Exercise 2 Lab: Create a new file system 10 minutes
1. Now that we have viewed and verified that we have the proper disk layout to take advantage of the new storage optimization features of SF 6.0, we will create a file system to test with.
2. First we must create a volume. Using the same terminal opened in Exercise 1, create a new 18g volume named sf60vol that will use the existing disk group sf60dg. Command: vxassist -g sf60dg make sf60vol 18g Command: vxprint -g sf60dg
3. Create a new filesystem, sf60fs, and mount it to our volume sf60vol. Notice the new VxFS layout as well called out as version 9. Command: mkfs -t vxfs /dev/vx/dsk/sf60dg/sf60vol Command: mkdir /sf60fs Command: mount -t vxfs /dev/vx/dsk/sf60dg/sf60vol /sf60fs
Lab Student’s Guide
Symantec Vision 2012 – MGM, Las Vegas SM L08 - Lab Student’s Guide / Last Update: Sunday, May 06, 2012 5
4. Verify the file system is present and the proper size. Command: df -H /sf60fs
5. Run an initial fragmentation report to show Fragmentation Index = 0. Command: /opt/VRTS/bin/fsadm -E /sf60fs
Lab Student’s Guide
Symantec Vision 2012 – MGM, Las Vegas SM L08 - Lab Student’s Guide / Last Update: Sunday, May 06, 2012 6
Lab Exercise 3 Lab: Compression Exercise 25 minutes
Now that we have our file system configured and verified, let’s copy some data and analyze the various features and impacts of VxCompress.
1. Create a new directory within our /sf60fs/ file system to hold the files we want to compress. Command: cd /sf60fs Command: mkdir comp Command: cd comp
2. Test files have been provided in the /sf/labData directory on your virtual machine. Let’s start small by taking all the .txt files from /sf/labData and copying them into our new /sf60fs/comp directory. Command: cp /sf/labData/*.txt .
3. Now, let’s turn these small files into larger files (~1 GB) each. The commands are listed below and are available on your desktop for cut-and-paste as “grow_file.txt” and “grow_file2.txt”.
a. Optional: Feel free to play around with the sizes and counts of these files. The end goal is to have a data set to show compression ratios while still performing within the lab environment.
Lab Student’s Guide
Symantec Vision 2012 – MGM, Las Vegas SM L08 - Lab Student’s Guide / Last Update: Sunday, May 06, 2012 7
Command: for i in {1..25}; do cat file.txt file.txt > tmp.txt && mv -f tmp.txt file.txt; done Command: for i in {1..25}; do cat file2.txt file2.txt > tmp.txt && mv -f tmp.txt file2.txt; done
4. Make a copy of each file so we can check contents later. Command: cp file.txt save.txt Command: cp file2.txt save2.txt
5. At this point we are ready to begin compression. Before doing so, let’s use the new vxcompress command to investigate the current compression status our test files. Since no compression has been run, each file will show Physical=Logical for the size, and 0 for %Compressed, %Extents, and Storage Savings. Command vxcompress -l .
Lab Student’s Guide
Symantec Vision 2012 – MGM, Las Vegas SM L08 - Lab Student’s Guide / Last Update: Sunday, May 06, 2012 8
6. Now let’s compress our two targeted files: File.txt and Save.txt with the default parameters using the new vxcompress command. Optional: Using Worksheet 1 mark down the statistics and configuration of your compression exercise. Command: vxcompress file.txt Command: vxcompress save.txt
7. Lets re-investigate the files using vxcompress and identify the differences in the Logical and Phyiscal sizes. As well as the %Compressed, %Extents, and Storage Savings. Here you can also take note of two new entries for Alg-Str and BSize indicating the compression algorithm and block size used, respectively. Both the algorithm and block size are tune-ables to generate larger compression rates with increased time, or smaller rates more quickly. At the end of this exercise you can play with the parameters and note the differences in your Worksheet.
Lab Student’s Guide
Symantec Vision 2012 – MGM, Las Vegas SM L08 - Lab Student’s Guide / Last Update: Sunday, May 06, 2012 9
Optional: Dig deeper and show how compressed files interact with the following commands and mark down the results in Worksheet 1. Command: /opt/VRTS/bin/fsadm -HS compressed /sf60fs Command: df -H . Command: du -h . Command: ls -alh .
8. We can dig deep into the files and show the compressed extents of file.txt. Here you can see each extent within the file, its physical size and its logical size. Command: fsmap -p file.txt | more
9. A key differentiator in file compression using Storage Foundation 6.0 is the fact that the extension of the file does
not change. This allows end-users and applications to read and modify compressed files just as they one an
uncompressed file. When reading a file, only those blocks to be read are uncompressed in memory, not on disk.
While the initial read may result in performance impact for heavily read files, any subsequent reads will come from
memory and provide a performance improvement. Note: The primary use case for compression is on older and
archived files. Live files can be compressed but the customer will have to plan for any performance degradation
accordingly.
Let’s look at how this interacts by reading the file.
Command: head -10 file.txt
Lab Student’s Guide
Symantec Vision 2012 – MGM, Las Vegas SM L08 - Lab Student’s Guide / Last Update: Sunday, May 06, 2012 10
10. To verify that nothing has been expanded during the read, investigate file.txt using vxcompress. You will see that
100% of its extents are still compressed.
Command: vxcompress -l file.txt
11. To verify that the overall content of the compressed file has not changed, let’s compare file.txt to our copied version,
save.txt. Using the cmp command we can see how the file system does not interact any differently when dealing
with compressed and non-compressed files.
Command: cmp file.txt save.txt
The results are empty, showing no difference between our compressed file.txt and the pre-compression copy,
save.txt.
12. Now, just as a typical user would, let’s modify some text within the compressed file. To do this, you can use vi or,
you can use dd to write some blocks within the file. The dd command are listed below and are available on your
desktop for cut-and-paste as “dd_file_txt.txt”
Command: dd if=/dev/zero of=file.txt bs=2048k count=100 seek=500 conv=notrunc
After modifying the file, let’s use vxcompress again to show a file in a mixed state of compression.
Optional: Mark down in Worksheet 1 the details of our compression targets after modification.
Optional: Compare file.txt and save.txt again and compare the results
13. The rest of these exercises are Optional and allow you to run vxcompress with non-default parameters and
compare the differences in savings. You can also run some defrag commands and see what, if any, vxcompress has
on the fragmentation of the drive.
Lab Student’s Guide
Symantec Vision 2012 – MGM, Las Vegas SM L08 - Lab Student’s Guide / Last Update: Sunday, May 06, 2012 11
More data of different file types are also available in /sf/labData for continued experimentation. Copy any of the
files into /sf60fs/comp and run tests as desired.
a. Uncompress and re-compress using new parameters. Note: Once a file is compressed, all future
compressions of extents within that file will use the same algorithm and block size. To change the algorithm
and/or the block size used, the entire file will have to be un-compressed.
Example Command: vxcompress -u file.txt
Example Command: vxcompress -b 64 file.txt
Note: Due to “0”’s written in the previous exercise, file.txt will not show 100% compression.
Example Command: vxcompress -b 256 -n 2 save.txt
Example Command: vxcompress -b 512 -t gzip-9 file.txt
Lab Student’s Guide
Symantec Vision 2012 – MGM, Las Vegas SM L08 - Lab Student’s Guide / Last Update: Sunday, May 06, 2012 12
You’re doing well you’re nearly halfway through the lab. The Compression exercise is complete. We’ll now move on to the De-duplication exercise.
Lab Student’s Guide
Symantec Vision 2012 – MGM, Las Vegas SM L08 - Lab Student’s Guide / Last Update: Sunday, May 06, 2012 13
Lab Exercise 4 Lab: De-duplication exercise 25 minutes
1. Let’s follow the same pattern as the compression exercise and create a new directory under /sf60fs/ to start our de-duplication tasks. Command: cd /sf60fs Command: mkdir dedup Command: cd dedup Command: cp ../comp/file2.txt . Command: cp ../comp/save2.txt .
2. To see the state prior to de-duplication, use fsadm to investigate the shared extents. This should show Space_Saved=0 as no de-duplication has been run Command: /opt/VRTS/bin/fsadm -S shared /sf60fs/
3. Before performing any de-duplication tasks, it has to be enabled on the root inode of our file system.. Command: fsdedupadm enable /sf60fs
Lab Student’s Guide
Symantec Vision 2012 – MGM, Las Vegas SM L08 - Lab Student’s Guide / Last Update: Sunday, May 06, 2012 14
4. Before performing an actual de-duplication, let’s utilize the dryrun parameter to get an idea of the savings. Optional: Mark the expected savings called out through the dryrun in Worksheet 2 for tracking. Command: fsdedupadm dryrun /sf60fs/
Optional: Check the status of the dryrun using fsdedupadm
Command: fsdedupadm status -v /sf60fs/
Note: Depending on what work was done in previous exercises, you may see up to 100% savings as we have been performing tasks on copies of the same data.
5. Now, just as a typical user would, let’s modify some text within the compressed file. To do this, you can use vi or,
you can use dd to write some blocks within the file. The dd command are listed below and are available on your
desktop for cut-and-paste as “dd_file2_txt.txt” and “dd_save2_txt.txt”.
Command: dd if=/dev/zero of=file2.txt bs=2048k count=100 seek=500 conv=notrunc
Command: dd if=/dev/zero of=save2.txt bs=2048k count=200 seek=1250 conv=notrunc
Lab Student’s Guide
Symantec Vision 2012 – MGM, Las Vegas SM L08 - Lab Student’s Guide / Last Update: Sunday, May 06, 2012 15
6. Now that we have modified our files, that were once identical, in different places, let’s perform an actual de-duplication. Command: fsdedupadm start /sf60fs/
7. Optional: Check the Status of the de-duplication run. Command: fsdedupadm status -v /sf60fs/
Optional: Once de-duplication is complete, mark the results in Worksheet 2 and compare the actual savings to the dryrun savings identified. Note: The files were modified in the exercise above, so the savings may be less than the dryrun.
8. Once complete, let’s investigate the savings using fsadm. Command: /opt/VRTS/bin/fsadm -HS shared /sf60fs
Lab Student’s Guide
Symantec Vision 2012 – MGM, Las Vegas SM L08 - Lab Student’s Guide / Last Update: Sunday, May 06, 2012 16
Optional: Compare the shared extent and savings numbers to the fsadm numbers shown at the beginning of this exercise.
9. Investigate how de-duplicated files interact with the following commands and mark down the results in Worksheet 1. Command: df -H . Command: du -h . Command: ls -alh .
10. The rest of these exercises are Optional and allow you to run vxcompress with non-default parameters and compare the differences in savings. You can also run some defrag commands and see what, if any, vxcompress has on the fragmentation of the drive. More data of different file types are also available in /sf/labData for continued experimentation. Copy any of the files into /sf60fs/comp and run tests as desired.
a. Run some more dd commands or use vi to edit the de-duplicated data and see how this impacts the shared extents.
b. Disable and remove de-duplication from /sf60fs/ and re-enable with non-default parameters and compare how different settings impact de-duplication savings and performance.
c. Run a defragmentation report and compare to the fragmentation index to the initial report done in the first exercise.
Lab Student’s Guide
Symantec Vision 2012 – MGM, Las Vegas SM L08 - Lab Student’s Guide / Last Update: Sunday, May 06, 2012 17
Lab Exercise 5 Lab: Explore the new “Human Readable” CLI on base unix commands 10 minutes
1. Compare “df” outputs from the following commands: Command: df Command: df -H
2. Call out specific units of measure for human read-ability within certain vx* commands.
a. Get the Highest unit of measure available Command: vxprint -u H
b. Get the results in MB Command: vxprint -u M
c. Get the results in GB Command: vxprint -u G
d. Get the results in PB Command: vxprint -u P
SM L08 -Enhanced Primary Storage Compression and De-duplication on UNIX and Linux with Storage Foundation 6.0 Lab Student’s Guide
Lab Student’s Guide
Symantec Vision 2012 – MGM, Las Vegas SM L08 - Lab Student’s Guide / Last Update: Sunday, May 06, 2012 19
Worksheet 1
Parameters
Used
Initial
Size
Optimized
Size
Savings % “fsadm” “df” “du” “ls”
Exercise 1.7: Default Compression File.txt:
File.txt:
File.txt:
File.txt:
Save.txt: Save.txt: Save.txt: Save.txt:
Exercise 1.12: Write to compressed file File.txt:
File.txt:
File.txt:
File.txt:
Save.txt: Save.txt: Save.txt: Save.txt:
Exercise 1.13
optional
Data:
Data:
Data:
Lab Student’s Guide
Symantec Vision 2012 – MGM, Las Vegas SM L08 - Lab Student’s Guide / Last Update: Sunday, May 06, 2012 20
Worksheet 2
Parameters
Used
Initial
Size
Optimized Size Savings %
Exercise 2.4: De-duplication dry run
Exercise 2.7: De-duplication
Exercise 2.9: De-duplication interaction “fsadm” “df” “du” “ls”
File.txt:
File.txt:
File.txt:
File.txt:
Save.txt: Save.txt: Save.txt: Save.txt:
Exercise 2.10
optional
Data:
Data:
Data: