Upload
eros
View
28
Download
0
Tags:
Embed Size (px)
DESCRIPTION
SaddleHill. A SCSI I/O Generator By Owen Parry. Project Motivation. To create an application that exercises new controller firmware and hardware. Provide the ability to rapidly add features. Provide more tolerance for hardware/firmware failures. - PowerPoint PPT Presentation
Citation preview
SaddleHillA SCSI I/O GeneratorByOwen Parry
Project MotivationTo create an application that exercises new controller firmware and hardware.Provide the ability to rapidly add features.Provide more tolerance for hardware/firmware failures.To Develop a mechanism that allows multiple hosts to randomly access and efficiently share the SATA affiliations.Current methods seek to avoid the Serial ATA limitation; using single initiator in a SAS domain; limiting communication with the disk drive to only 1 initiator at a time.Need simple and decentralized strategy for an embedded environment.Achieve long-term max-min fairness.Avoid violating I/O time limits.
BackgroundSCSI targeted at the enterprise storage market. Used primarily to attach hard disk drives.High performanceRPM: 10K, 15KSeek Time: 3.2 7.4 msGreater reliability.MTBF: 1.2 M HrCapacity, 18 300GbExpensive: $160 - $1400Multiple host support.ATA targeted at the desktop market.Medium performanceRPM: 5400, 7200Seek Time: 8.9 9.5 msMediocre reliabilityMTBF: 500 K hrCapacity, 40GB 1TbCheap: $75 - $300Single host.
BackgroundSerial Attached SCSI is the new Transport protocol replacing parallel SCSI.SAS Advantages.Faster Data Rates.SAS-1:300 MB/sSAS-2:600MB/sLarger Drive counts.Typical Domain size 12816K addresses using fan-out expandersIncreased data integrity.Configuration flexibility.Supports Serial ATA Drives.
BackgroundProblem with SATA in SAS topologyArchitecture only allows a single host.SAS uses mutual exclusion called an Affiliation. The first initiator to open a connection may own the affilation indefinitely.Vendors want to simultaneously issue commands to SATA disks from multiple initiators.
Related WorkUnable to locate other works in the storage area.Closely Related ResearchWireless LANsBandwidth sharing schemes:Maxmin Fair Scheduling in Wireless Networks, Leandros Tassiulas and Saswati Sarkar.Channel time sharing schemes:Proportional Fairness in Wireless LANs and Ad Hoc Networks, Li Bin Jian, and Soung Chang LiewTime-based fairness improves Performance in Multi-rate WLANs, Godfrey Tan and John Guttag.
SaddleHill Design / Implementation
text
text
SCSI IO System
SCSI Target
Service Delivery Subsystem
SCSI Initiator
Linux Operating System
SAS Initiator Port
SaddleHill Driver
SaddleHill
SaddleHill Design / ImplementationBuilt using Trolltechs Qt 4.2.3Compiled for x86_64 bit systems.Comprised of four logical blocks.
SaddleHill Design / ImplementationMainWindowLists PCI SAS Initiators devices.Lists SAS Target devices.Displays Live Test Statistics.Displays Application messages.Accepts user input.
SaddleHill Design / ImplementationManagement UnitManages SaddleHills physical I/O Data buffers, and Initiator operational buffers.Address conversion: Virtual to Physical; Physical to Virtual.Maintains a list of SAS Initiators and Targets.Maintain the application message log.Maintain the model objects (system device, message, statistics) which are used by the GUI to gather and display information to user.Distributes device configurations.Starts/Stops I/O tests. Calculates I/O and Throughput rates.
SaddleHill Design / ImplementationIO EngineInitializes SAS targets.Maintains disk SAS Addresses, and Target ID.Generates, Issues, and Completes SCSI Commands. e.g. Read10, Write10, Write And Verify10, Inquiry, Read Capacity etc.Comprises three threads to perform each of the above tasks.Maintains statistics:Number of I/Os issuedNumber of I/Os completedError countAmount of Data Transferred.I/O Response times.
SaddleHill Design / ImplementationHardware Abstraction LayerSaddleHillDriverRegisters with linux kernel as a character device.Registers with PCI core.Allocates blocks of physical memory. Currently 16 MB.Reserves the physical memory to prevent swapping.Provides the facilities to map PCI SAS I/O control registers to user space.Provides the facilities to map the physical memory to user space.Provides PCI Device configuration information to user space application.HAL (User Level)Implements the MPI specificationInitializes the SAS AdapterConverts Requests from IO Engine to the MPI specific format.Sends requests to and receive replies to/from the Initiator via the PCI control registers.Processes MPI Replies and completes request to IO Engine.Manages STP Affiliations.Maintains test statisticsNumber of I/Os issues.Number of I/Os completed.Error Count.Amount of Data Transferred.I/O Response Times.Affiliation ownership times.Affiliation synchronization count.
Affiliation SynchronizationUses idea put forward in Proportional Firness in Wireless LANs and Ad Hoc Networks.Fix the maximum transmission time.Contend fairly among the initiators for the mutex.ImplementationAffiliation AcquisitionAcquisition started be reception of new I/OCalculate back-off.Use uniform distribution random number generator to choose back-off time within contention window size.Generate SCSI Inquiry commandSleep for length of back-offIssue InquiryFailed synchronization attempt doubles contention window size.Start timer on successful acquisitionAffiliation ReleaseResource released if no I/Os are waiting to be sent.Resource released after ownership timer expires.There are no preemptions.I/Os are placed into a waiting state during the release and acquisition process.I/Os outstanding at the time of release are allowed to completed.The truncated binary exponential back-off strategy is used to calculate the back-off times.
Finding the Back-Off StrategyConsidered strategies for back-off included:No Back-offFixed WindowBEBTBEBLogarithmicTest Strategy:Read10, Write10 commandsSingle Block TransfersSame LBADrive Caching EnabledNCQ enabledDrive Q-Depth = 83Gb SATA diskMultiple Initiators
Finding the Back-Off StrategySTP Ownership Times
Chart14
17736101770986
11873801220190
15747731760360
16632581719267
17453181746005
17459071745360
Disk 1
Disk 2
Algorithm
Time (ms)
STP Resource Ownership Time
Sheet1
Two Adapters - Run Time = 60 seconds
BackOff AlgorithmAvg. RespLargest RespNum Sync. RequestsRsc Time
No BackOff57753248929378
Fixed Window14340576720589
BEB311496918232691
TBEB149557118326690
LOG58821103229042
TLOG59792102029133
No BackOff59813253129262
Fixed Window14428457920058
BEB2491349716820045
TBEB129581118328216
LOG58817103428542
TLOG631097104028225
Four Adapters - Run Time 60 seconds
No BackOff1883150376016036
Fixed Window4516572859080
BEB5631189415911797
TBEB387537923211549
LOG2913654173912344
TLOG1892786160614259
No BackOff2802794409813062
Fixed Window34533979911092
BEB3781127017213585
TBEB383557324812013
LOG1622521156216152
TLOG1853690155315313
No BackOff2772646402912702
Fixed Window6316025907108
BEB490106581849874
TBEB353762924014646
LOG2644551165013745
TLOG2474035164713443
No BackOff1643110369016562
Fixed Window33148108911064
BEB2921230718417475
TBEB288475921715039
LOG2062625155215343
TLOG2253810164114275
Two Adapters - Run Time 60 minutes
No BackOff699401495851773610
Fixed Window163696246601187380
BEB1543826776951574773
TBEB1287102123431663258
LOG571657858891745318
TLOG571212856431745907
No BackOff699101503961770986
Fixed Window155615145961220190
BEB1203963576841760360
TBEB1196537121401719267
LOG571229855851746005
TLOG571140856311745360
Four Adapters - Run Time 60 minutes
No BackOff2389134308174867054
Fixed Window38193625810618523
BEB3543363410843822688
TBEB3961086214271790866
LOG2176247128800850684
TLOG2496267100385857229
No BackOff2348822306691876004
Fixed Window41581295874586537
BEB3913951810626775765
TBEB371904414418819852
LOG2045998127432879152
TLOG235736199478883262
No BackOff23411161307522869000
Fixed Window415104715850588228
BEB3912705510542752990
TBEB372884614381827747
LOG2035665127281879839
TLOG2577460100843848736
No BackOff2268612304844891607
Fixed Window41693185802584862
BEB3344110810788854700
TBEB398890014336780650
LOG2106959128301865056
TLOG233568499443886117
Sheet1
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
Largest IO Response Time
Sheet2
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Count
Synchronization Requests
Sheet3
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
STP Resource Ownership Time
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
Average IO Response Times
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
Largest IO Response Time
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 2
Disk 4
Algorithm
Count
Synchronization Requests
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
STP Resource Ownership Time
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
Average IO Response Times
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
Average IO Response Times
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
Largest IO Response Time
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Num. Attempts
Synchronization Requests
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
STP Resource Ownership Time
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
Average IO Response Times
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
Largest IO Response TIme
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Number of Attempts
Synchronization Requests
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
STP Resource Ownership Times
Chart12
2926229378
2005820589
2004532691
2821626690
2854229042
2822529133
Disk 1
Disk 2
Algorithm
Time (ms)
STP Resource Ownership Time
Sheet1
Two Adapters - Run Time = 60 seconds
BackOff AlgorithmAvg. RespLargest RespNum Sync. RequestsRsc Time
No BackOff57753248929378
Fixed Window14340576720589
BEB311496918232691
TBEB149557118326690
LOG58821103229042
TLOG59792102029133
No BackOff59813253129262
Fixed Window14428457920058
BEB2491349716820045
TBEB129581118328216
LOG58817103428542
TLOG631097104028225
Four Adapters - Run Time 60 seconds
No BackOff1883150376016036
Fixed Window4516572859080
BEB5631189415911797
TBEB387537923211549
LOG2913654173912344
TLOG1892786160614259
No BackOff2802794409813062
Fixed Window34533979911092
BEB3781127017213585
TBEB383557324812013
LOG1622521156216152
TLOG1853690155315313
No BackOff2772646402912702
Fixed Window6316025907108
BEB490106581849874
TBEB353762924014646
LOG2644551165013745
TLOG2474035164713443
No BackOff1643110369016562
Fixed Window33148108911064
BEB2921230718417475
TBEB288475921715039
LOG2062625155215343
TLOG2253810164114275
Two Adapters - Run Time 60 minutes
No BackOff699401495851773610
Fixed Window163696246601187380
BEB1543826776951574773
TBEB1287102123431663258
LOG571657858891745318
TLOG571212856431745907
No BackOff699101503961770986
Fixed Window155615145961220190
BEB1203963576841760360
TBEB1196537121401719267
LOG571229855851746005
TLOG571140856311745360
Four Adapters - Run Time 60 minutes
No BackOff2389134308174867054
Fixed Window38193625810618523
BEB3543363410843822688
TBEB3961086214271790866
LOG2176247128800850684
TLOG2496267100385857229
No BackOff2348822306691876004
Fixed Window41581295874586537
BEB3913951810626775765
TBEB371904414418819852
LOG2045998127432879152
TLOG235736199478883262
No BackOff23411161307522869000
Fixed Window415104715850588228
BEB3912705510542752990
TBEB372884614381827747
LOG2035665127281879839
TLOG2577460100843848736
No BackOff2268612304844891607
Fixed Window41693185802584862
BEB3344110810788854700
TBEB398890014336780650
LOG2106959128301865056
TLOG233568499443886117
Sheet1
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
Largest IO Response Time
Sheet2
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Count
Synchronization Requests
Sheet3
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
STP Resource Ownership Time
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
Average IO Response Times
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
Largest IO Response Time
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 2
Disk 4
Algorithm
Count
Synchronization Requests
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
STP Resource Ownership Time
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
Average IO Response Times
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
Average IO Response Times
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
Largest IO Response Time
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Num. Attempts
Synchronization Requests
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
STP Resource Ownership Time
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
Average IO Response Times
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
Largest IO Response TIme
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Number of Attempts
Synchronization Requests
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
STP Resource Ownership Times
Finding the Back-Off StrategySynchronization Requests
Chart8
25312489
7967
168182
183183
10341032
10401020
Disk 1
Disk 2
Algorithm
Count
Synchronization Requests
Sheet1
Two Adapters - Run Time = 60 seconds
BackOff AlgorithmAvg. RespLargest RespNum Sync. RequestsRsc Time
No BackOff57753248929378
Fixed Window14340576720589
BEB311496918232691
TBEB149557118326690
LOG58821103229042
TLOG59792102029133
No BackOff59813253129262
Fixed Window14428457920058
BEB2491349716820045
TBEB129581118328216
LOG58817103428542
TLOG631097104028225
Four Adapters - Run Time 60 seconds
No BackOff1883150376016036
Fixed Window4516572859080
BEB5631189415911797
TBEB387537923211549
LOG2913654173912344
TLOG1892786160614259
No BackOff2802794409813062
Fixed Window34533979911092
BEB3781127017213585
TBEB383557324812013
LOG1622521156216152
TLOG1853690155315313
No BackOff2772646402912702
Fixed Window6316025907108
BEB490106581849874
TBEB353762924014646
LOG2644551165013745
TLOG2474035164713443
No BackOff1643110369016562
Fixed Window33148108911064
BEB2921230718417475
TBEB288475921715039
LOG2062625155215343
TLOG2253810164114275
Two Adapters - Run Time 60 minutes
No BackOff699401495851773610
Fixed Window163696246601187380
BEB1543826776951574773
TBEB1287102123431663258
LOG571657858891745318
TLOG571212856431745907
No BackOff699101503961770986
Fixed Window155615145961220190
BEB1203963576841760360
TBEB1196537121401719267
LOG571229855851746005
TLOG571140856311745360
Four Adapters - Run Time 60 minutes
No BackOff2389134308174867054
Fixed Window38193625810618523
BEB3543363410843822688
TBEB3961086214271790866
LOG2176247128800850684
TLOG2496267100385857229
No BackOff2348822306691876004
Fixed Window41581295874586537
BEB3913951810626775765
TBEB371904414418819852
LOG2045998127432879152
TLOG235736199478883262
No BackOff23411161307522869000
Fixed Window415104715850588228
BEB3912705510542752990
TBEB372884614381827747
LOG2035665127281879839
TLOG2577460100843848736
No BackOff2268612304844891607
Fixed Window41693185802584862
BEB3344110810788854700
TBEB398890014336780650
LOG2106959128301865056
TLOG233568499443886117
Sheet1
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
Largest IO Response Time
Sheet2
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Count
Synchronization Requests
Sheet3
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
STP Resource Ownership Time
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
Average IO Response Times
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
Largest IO Response Time
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 2
Disk 4
Algorithm
Count
Synchronization Requests
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
STP Resource Ownership Time
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
Average IO Response Times
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
Average IO Response Times
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
Largest IO Response Time
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Num. Attempts
Synchronization Requests
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
STP Resource Ownership Time
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
Average IO Response Times
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
Largest IO Response TIme
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Number of Attempts
Synchronization Requests
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
STP Resource Ownership Times
Chart10
149585150396
46604596
76957684
1234312140
8588985585
8564385631
Disk 1
Disk 2
Algorithm
Num. Attempts
Synchronization Requests
Sheet1
Two Adapters - Run Time = 60 seconds
BackOff AlgorithmAvg. RespLargest RespNum Sync. RequestsRsc Time
No BackOff57753248929378
Fixed Window14340576720589
BEB311496918232691
TBEB149557118326690
LOG58821103229042
TLOG59792102029133
No BackOff59813253129262
Fixed Window14428457920058
BEB2491349716820045
TBEB129581118328216
LOG58817103428542
TLOG631097104028225
Four Adapters - Run Time 60 seconds
No BackOff1883150376016036
Fixed Window4516572859080
BEB5631189415911797
TBEB387537923211549
LOG2913654173912344
TLOG1892786160614259
No BackOff2802794409813062
Fixed Window34533979911092
BEB3781127017213585
TBEB383557324812013
LOG1622521156216152
TLOG1853690155315313
No BackOff2772646402912702
Fixed Window6316025907108
BEB490106581849874
TBEB353762924014646
LOG2644551165013745
TLOG2474035164713443
No BackOff1643110369016562
Fixed Window33148108911064
BEB2921230718417475
TBEB288475921715039
LOG2062625155215343
TLOG2253810164114275
Two Adapters - Run Time 60 minutes
No BackOff699401495851773610
Fixed Window163696246601187380
BEB1543826776951574773
TBEB1287102123431663258
LOG571657858891745318
TLOG571212856431745907
No BackOff699101503961770986
Fixed Window155615145961220190
BEB1203963576841760360
TBEB1196537121401719267
LOG571229855851746005
TLOG571140856311745360
Four Adapters - Run Time 60 minutes
No BackOff2389134308174867054
Fixed Window38193625810618523
BEB3543363410843822688
TBEB3961086214271790866
LOG2176247128800850684
TLOG2496267100385857229
No BackOff2348822306691876004
Fixed Window41581295874586537
BEB3913951810626775765
TBEB371904414418819852
LOG2045998127432879152
TLOG235736199478883262
No BackOff23411161307522869000
Fixed Window415104715850588228
BEB3912705510542752990
TBEB372884614381827747
LOG2035665127281879839
TLOG2577460100843848736
No BackOff2268612304844891607
Fixed Window41693185802584862
BEB3344110810788854700
TBEB398890014336780650
LOG2106959128301865056
TLOG233568499443886117
Sheet1
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
Largest IO Response Time
Sheet2
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Count
Synchronization Requests
Sheet3
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
STP Resource Ownership Time
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
Average IO Response Times
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
Largest IO Response Time
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 2
Disk 4
Algorithm
Count
Synchronization Requests
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
STP Resource Ownership Time
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
Average IO Response Times
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
Average IO Response Times
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
Largest IO Response Time
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Num. Attempts
Synchronization Requests
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
STP Resource Ownership Time
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
Average IO Response Times
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
Largest IO Response TIme
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Number of Attempts
Synchronization Requests
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
STP Resource Ownership Times
Finding the Back-Off StrategyAverage I/O Response
Chart3
5957
144143
24931
129149
5858
6359
Disk 1
Disk 2
Algorithm
Time (ms)
Average IO Response Times
Sheet1
Two Adapters - Run Time = 60 seconds
BackOff AlgorithmAvg. RespLargest RespNum Sync. RequestsRsc Time
No BackOff57753248929378
Fixed Window14340576720589
BEB311496918232691
TBEB149557118326690
LOG58821103229042
TLOG59792102029133
No BackOff59813253129262
Fixed Window14428457920058
BEB2491349716820045
TBEB129581118328216
LOG58817103428542
TLOG631097104028225
Four Adapters - Run Time 60 seconds
No BackOff1883150376016036
Fixed Window4516572859080
BEB5631189415911797
TBEB387537923211549
LOG2913654173912344
TLOG1892786160614259
No BackOff2802794409813062
Fixed Window34533979911092
BEB3781127017213585
TBEB383557324812013
LOG1622521156216152
TLOG1853690155315313
No BackOff2772646402912702
Fixed Window6316025907108
BEB490106581849874
TBEB353762924014646
LOG2644551165013745
TLOG2474035164713443
No BackOff1643110369016562
Fixed Window33148108911064
BEB2921230718417475
TBEB288475921715039
LOG2062625155215343
TLOG2253810164114275
Two Adapters - Run Time 60 minutes
No BackOff699401495851773610
Fixed Window163696246601187380
BEB1543826776951574773
TBEB1287102123431663258
LOG571657858891745318
TLOG571212856431745907
No BackOff699101503961770986
Fixed Window155615145961220190
BEB1203963576841760360
TBEB1196537121401719267
LOG571229855851746005
TLOG571140856311745360
Four Adapters - Run Time 60 minutes
No BackOff2389134308174867054
Fixed Window38193625810618523
BEB3543363410843822688
TBEB3961086214271790866
LOG2176247128800850684
TLOG2496267100385857229
No BackOff2348822306691876004
Fixed Window41581295874586537
BEB3913951810626775765
TBEB371904414418819852
LOG2045998127432879152
TLOG235736199478883262
No BackOff23411161307522869000
Fixed Window415104715850588228
BEB3912705510542752990
TBEB372884614381827747
LOG2035665127281879839
TLOG2577460100843848736
No BackOff2268612304844891607
Fixed Window41693185802584862
BEB3344110810788854700
TBEB398890014336780650
LOG2106959128301865056
TLOG233568499443886117
Sheet1
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
Largest IO Response Time
Sheet2
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Count
Synchronization Requests
Sheet3
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
STP Resource Ownership Time
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
Largest IO Response Time
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 2
Disk 4
Algorithm
Count
Synchronization Requests
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
STP Resource Ownership Time
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
Average IO Response Times
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
Average IO Response Times
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
Largest IO Response Time
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Num. Attempts
Synchronization Requests
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
STP Resource Ownership Time
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
Average IO Response Times
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
Largest IO Response TIme
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Number of Attempts
Synchronization Requests
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
STP Resource Ownership Times
MBD0019B0B4.xls
Chart1
188280277164
451345631331
563378490292
387383353288
291162264206
189185247225
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
Average IO Response Times
Sheet1
Two Adapters - Run Time = 60 seconds
BackOff AlgorithmAvg. RespLargest RespNum Sync. RequestsRsc Time
No BackOff57753248929378
Fixed Window14340576720589
BEB311496918232691
TBEB149557118326690
LOG58821103229042
TLOG59792102029133
No BackOff59813253129262
Fixed Window14428457920058
BEB2491349716820045
TBEB129581118328216
LOG58817103428542
TLOG631097104028225
Four Adapters - Run Time 60 seconds
No BackOff1883150376016036
Fixed Window4516572859080
BEB5631189415911797
TBEB387537923211549
LOG2913654173912344
TLOG1892786160614259
No BackOff2802794409813062
Fixed Window34533979911092
BEB3781127017213585
TBEB383557324812013
LOG1622521156216152
TLOG1853690155315313
No BackOff2772646402912702
Fixed Window6316025907108
BEB490106581849874
TBEB353762924014646
LOG2644551165013745
TLOG2474035164713443
No BackOff1643110369016562
Fixed Window33148108911064
BEB2921230718417475
TBEB288475921715039
LOG2062625155215343
TLOG2253810164114275
Two Adapters - Run Time 60 minutes
No BackOff699401495851773610
Fixed Window163696246601187380
BEB1543826776951574773
TBEB1287102123431663258
LOG571657858891745318
TLOG571212856431745907
No BackOff699101503961770986
Fixed Window155615145961220190
BEB1203963576841760360
TBEB1196537121401719267
LOG571229855851746005
TLOG571140856311745360
Four Adapters - Run Time 60 minutes
No BackOff2389134308174867054
Fixed Window38193625810618523
BEB3543363410843822688
TBEB3961086214271790866
LOG2176247128800850684
TLOG2496267100385857229
No BackOff2348822306691876004
Fixed Window41581295874586537
BEB3913951810626775765
TBEB371904414418819852
LOG2045998127432879152
TLOG235736199478883262
No BackOff23411161307522869000
Fixed Window415104715850588228
BEB3912705510542752990
TBEB372884614381827747
LOG2035665127281879839
TLOG2577460100843848736
No BackOff2268612304844891607
Fixed Window41693185802584862
BEB3344110810788854700
TBEB398890014336780650
LOG2106959128301865056
TLOG233568499443886117
Sheet1
No BackOffNo BackOff
Fixed WindowFixed Window
BEBBEB
TBEBTBEB
LOGLOG
TLOGTLOG
Disk 1
Disk 2
Algorithm
Time (ms)
Largest IO Response Time
Sheet2
No backoffNo backoff
Fixed WindowFixed Window
BEBBEB
TBEBTBEB
LOGLOG
TLOGTLOG
Disk 1
Disk 2
Algorithm
Count
Synchronization Requests
Sheet3
No BackOffNo BackOff
Fixed WindowFixed Window
BEBBEB
TBEBTBEB
LOGLOG
TLOGTLOG
Disk 1
Disk 2
Algorithm
Time (ms)
STP Resource Ownership Time
No BackOffNo BackOffNo BackOffNo BackOff
Fixed WindowFixed WindowFixed WindowFixed Window
BEBBEBBEBBEB
TBEBTBEBTBEBTBEB
LOGLOGLOGLOG
TLOGTLOGTLOGTLOG
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
Average IO Response Times
No backOffNo backOffNo backOffNo backOff
Fixed WindowFixed WindowFixed WindowFixed Window
BEBBEBBEBBEB
TBEBTBEBTBEBTBEB
LOGLOGLOGLOG
TLOGTLOGTLOGTLOG
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
Largest IO Response Time
No BackOffNo BackOffNo BackOffNo BackOff
Fixed WindowFixed WindowFixed WindowFixed Window
BEBBEBBEBBEB
TBEBTBEBTBEBTBEB
LOGLOGLOGLOG
TLOGTLOGTLOGTLOG
Disk 1
Disk 2
Disk 2
Disk 4
Algorithm
Count
Synchronization Requests
No BackOffNo BackOffNo BackOffNo BackOff
Fixed WindowFixed WindowFixed WindowFixed Window
BEBBEBBEBBEB
TBEBTBEBTBEBTBEB
LOGLOGLOGLOG
TLOGTLOGTLOGTLOG
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
STP Resource Ownership Time
No BackoffNo Backoff
Fixed WindowFixed Window
BEBBEB
TBEBTBEB
LOGLOG
TLOGTLOG
Disk 1
Disk 2
Algorithm
Time (ms)
Average IO Response Times
No BackOffNo BackOff
Fixed WindowFixed Window
BEBBEB
TBEBTBEB
LOGLOG
TLOGTLOG
Disk 1
Disk 2
Algorithm
Time (ms)
Average IO Response Times
No BackoffNo Backoff
Fixed WindowFixed Window
BEBBEB
TBEBTBEB
LOGLOG
TLOGTLOG
Disk 1
Disk 2
Algorithm
Time (ms)
Largest IO Response Time
No BackoffNo Backoff
Fixed WindowFixed Window
BEBBEB
TBEBTBEB
LOGLOG
TLOGTLOG
Disk 1
Disk 2
Algorithm
Num. Attempts
Synchronization Requests
No BackoffNo Backoff
Fixed WindowFixed Window
BEBBEB
TBEBTBEB
LOGLOG
TLOGTLOG
Disk 1
Disk 2
Algorithm
Time (ms)
STP Resource Ownership Time
No BackoffNo BackoffNo BackoffNo Backoff
Fixed WindowFixed WindowFixed WindowFixed Window
BEBBEBBEBBEB
TBEBTBEBTBEBTBEB
LOGLOGLOGLOG
TLOGTLOGTLOGTLOG
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
Average IO Response Times
No BackoffNo BackoffNo BackoffNo Backoff
Fixed WindowFixed WindowFixed WindowFixed Window
BEBBEBBEBBEB
TBEBTBEBTBEBTBEB
LOGLOGLOGLOG
TLOGTLOGTLOGTLOG
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
Largest IO Response TIme
No BackoffNo BackoffNo BackoffNo Backoff
Fixed WindowFixed WindowFixed WindowFixed Window
BEBBEBBEBBEB
TBEBTBEBTBEBTBEB
LOGLOGLOGLOG
TLOGTLOGTLOGTLOG
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Number of Attempts
Synchronization Requests
No BackoffNo BackoffNo BackoffNo Backoff
Fixed WindowFixed WindowFixed WindowFixed Window
BEBBEBBEBBEB
TBEBTBEBTBEBTBEB
LOGLOGLOGLOG
TLOGTLOGTLOGTLOG
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
STP Resource Ownership Times
Chart2
6969
163155
154120
128119
5757
5757
Disk 1
Disk 2
Algorithm
Time (ms)
Average IO Response Times
Sheet1
Two Adapters - Run Time = 60 seconds
BackOff AlgorithmAvg. RespLargest RespNum Sync. RequestsRsc Time
No BackOff57753248929378
Fixed Window14340576720589
BEB311496918232691
TBEB149557118326690
LOG58821103229042
TLOG59792102029133
No BackOff59813253129262
Fixed Window14428457920058
BEB2491349716820045
TBEB129581118328216
LOG58817103428542
TLOG631097104028225
Four Adapters - Run Time 60 seconds
No BackOff1883150376016036
Fixed Window4516572859080
BEB5631189415911797
TBEB387537923211549
LOG2913654173912344
TLOG1892786160614259
No BackOff2802794409813062
Fixed Window34533979911092
BEB3781127017213585
TBEB383557324812013
LOG1622521156216152
TLOG1853690155315313
No BackOff2772646402912702
Fixed Window6316025907108
BEB490106581849874
TBEB353762924014646
LOG2644551165013745
TLOG2474035164713443
No BackOff1643110369016562
Fixed Window33148108911064
BEB2921230718417475
TBEB288475921715039
LOG2062625155215343
TLOG2253810164114275
Two Adapters - Run Time 60 minutes
No BackOff699401495851773610
Fixed Window163696246601187380
BEB1543826776951574773
TBEB1287102123431663258
LOG571657858891745318
TLOG571212856431745907
No BackOff699101503961770986
Fixed Window155615145961220190
BEB1203963576841760360
TBEB1196537121401719267
LOG571229855851746005
TLOG571140856311745360
Four Adapters - Run Time 60 minutes
No BackOff2389134308174867054
Fixed Window38193625810618523
BEB3543363410843822688
TBEB3961086214271790866
LOG2176247128800850684
TLOG2496267100385857229
No BackOff2348822306691876004
Fixed Window41581295874586537
BEB3913951810626775765
TBEB371904414418819852
LOG2045998127432879152
TLOG235736199478883262
No BackOff23411161307522869000
Fixed Window415104715850588228
BEB3912705510542752990
TBEB372884614381827747
LOG2035665127281879839
TLOG2577460100843848736
No BackOff2268612304844891607
Fixed Window41693185802584862
BEB3344110810788854700
TBEB398890014336780650
LOG2106959128301865056
TLOG233568499443886117
Sheet1
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
Largest IO Response Time
Sheet2
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Count
Synchronization Requests
Sheet3
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
STP Resource Ownership Time
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
Largest IO Response Time
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 2
Disk 4
Algorithm
Count
Synchronization Requests
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
STP Resource Ownership Time
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
Average IO Response Times
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
Average IO Response Times
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
Largest IO Response Time
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Num. Attempts
Synchronization Requests
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
STP Resource Ownership Time
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
Average IO Response Times
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
Largest IO Response TIme
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Number of Attempts
Synchronization Requests
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
STP Resource Ownership Times
MBD0019B0B4.xls
Chart1
188280277164
451345631331
563378490292
387383353288
291162264206
189185247225
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
Average IO Response Times
Sheet1
Two Adapters - Run Time = 60 seconds
BackOff AlgorithmAvg. RespLargest RespNum Sync. RequestsRsc Time
No BackOff57753248929378
Fixed Window14340576720589
BEB311496918232691
TBEB149557118326690
LOG58821103229042
TLOG59792102029133
No BackOff59813253129262
Fixed Window14428457920058
BEB2491349716820045
TBEB129581118328216
LOG58817103428542
TLOG631097104028225
Four Adapters - Run Time 60 seconds
No BackOff1883150376016036
Fixed Window4516572859080
BEB5631189415911797
TBEB387537923211549
LOG2913654173912344
TLOG1892786160614259
No BackOff2802794409813062
Fixed Window34533979911092
BEB3781127017213585
TBEB383557324812013
LOG1622521156216152
TLOG1853690155315313
No BackOff2772646402912702
Fixed Window6316025907108
BEB490106581849874
TBEB353762924014646
LOG2644551165013745
TLOG2474035164713443
No BackOff1643110369016562
Fixed Window33148108911064
BEB2921230718417475
TBEB288475921715039
LOG2062625155215343
TLOG2253810164114275
Two Adapters - Run Time 60 minutes
No BackOff699401495851773610
Fixed Window163696246601187380
BEB1543826776951574773
TBEB1287102123431663258
LOG571657858891745318
TLOG571212856431745907
No BackOff699101503961770986
Fixed Window155615145961220190
BEB1203963576841760360
TBEB1196537121401719267
LOG571229855851746005
TLOG571140856311745360
Four Adapters - Run Time 60 minutes
No BackOff2389134308174867054
Fixed Window38193625810618523
BEB3543363410843822688
TBEB3961086214271790866
LOG2176247128800850684
TLOG2496267100385857229
No BackOff2348822306691876004
Fixed Window41581295874586537
BEB3913951810626775765
TBEB371904414418819852
LOG2045998127432879152
TLOG235736199478883262
No BackOff23411161307522869000
Fixed Window415104715850588228
BEB3912705510542752990
TBEB372884614381827747
LOG2035665127281879839
TLOG2577460100843848736
No BackOff2268612304844891607
Fixed Window41693185802584862
BEB3344110810788854700
TBEB398890014336780650
LOG2106959128301865056
TLOG233568499443886117
Sheet1
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
Largest IO Response Time
Sheet2
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Count
Synchronization Requests
Sheet3
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
STP Resource Ownership Time
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
Average IO Response Times
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
Largest IO Response Time
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 2
Disk 4
Algorithm
Count
Synchronization Requests
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
STP Resource Ownership Time
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
Average IO Response Times
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
Average IO Response Times
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
Largest IO Response Time
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Num. Attempts
Synchronization Requests
00
00
00
00
00
00
Disk 1
Disk 2
Algorithm
Time (ms)
STP Resource Ownership Time
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
Average IO Response Times
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
Largest IO Response TIme
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Number of Attempts
Synchronization Requests
0000
0000
0000
0000
0000
0000
Disk 1
Disk 2
Disk 3
Disk 4
Algorithm
Time (ms)
STP Resource Ownership Times
Finding the Back-Off StrategyNo Back-OffToo many synchronization attemptsDepending on topology configuration will favor some initiatorsFixed WindowThere is no way to chose the appropriate window size.BEBViolates the I/O time limits in long test runs.LogarithmicAchieves near perfect max-min fairness in resource ownership in both short and long terms.Large number of synchronization requests. Unacceptable in large topologies.The Truncated Binary Exponential Strategy was chosen for the implementation of the synchronization algorithmClosely achieves long-term max-min fairnessLow number of synchronization attempts.
PerformanceTransaction processing profile was used. Small Block Transfer (1-16 Blocks)Concerned with I/O Rates rather than throughput.Single InitiatorSame IO size ~2250 IOPS.Random IO sizes ~902 IOPS.Dual InitiatorsSame IO sizes ~ 2075 IOPS. 8% Performance decrease.Random IO sizes ~ 786 IOPS. 12% Performance decreaseQuad InitiatorsSame IO sizes ~ 1975 IOPS. 14% Performance decreaseRandom IO sizes ~745 IOPS. 17% Performance decrease
Performance
Future DirectionsDue to the challenges of SATA in enterprise storage environments. Vendors are employing varying strategies to deal with the SATA problem. These include:Completely removing SATA from topologies.Building special hardware that increase the affiliation resources. The STP Resource sharing algorithm will be moved to the SAS Initiator port.Requires a change in the mechanism that acquires an releases affiliations.Utilize the SAS CLOSE(CLEAR AFFILIATION) primitive when tearing down connections.Simply convert and issue host IO.
SaddleHillShort TermSupport SAS-2 InitiatorSupport additional SBC and SPC commandsSupport SSC and MMC SCSI command setsFW Upgrade SupportInitiator Configuration ModificationLong TermBuild into a automated firmware unit test system.
ConclusionAll project goals achievedUser-Level SCSI I/O generatorSynchronization algorithm that meets the simplicity, fairness and decentralization objectives.
QUESTIONS?