Embed Size (px)
Citation preview
2
Legal Stuff
• Notice• IBM may have patents or pending patent applications covering subject matter
described in this document. The furnishing of this document does not give you any
license to these patents. You can send license inquiries, in writing to: IBM Director
of Licensing, IBM Corporation, North Castle Drive, Armonk, NY 10504-
1785 U.S.A.
• Any references in this information to non-IBM Web sites are provied for convenience
only and do not in any manner serve as an endorsement of those Web sites. The
materials at those Web sites are not part of the materials for this IBM product and
use of those Web sites is at your own risk.
• Trademarks
• The following terms are trademarks of the International Business Machines
Corporation in the United States, other countries, or both: FICON® IBM®
RedbooksTM System z10TM z/OS® zSeries® z10TM
• Other Company, product, or service names may be trademarks or service marks of
others.
3
Agenda
Frames, Sequences and Exchanges
IDAW vs MIDAW
Buffer to Buffer Credit
Ficon Recovery
4
Fibre Channel Frame
The basic building block is the FRAME
4 24 1 – 2048
(2112)
4 4
Identifies beginning of a frame
Source/Destination link addresses
Exchange ID
Protocol Type (FICON – 1B/1C, zHPF=08)
Start or End of Sequence
Start or End of Exchange
Protects Frame
Identifies end of a frame
SOF Header Payload CRC EOF
5
Sequences and IUs
• Each Upper Layer Protocol (ULP) defines the contents and format of it’s own Information Units (IUs)
• Commands
• Data
• Status
• Control
• Etc
• Ficon IUs can be up to 8K (8192) in size
• 8160 (8K-32) bytes of data• 32 bytes contain Ficon Header information
• 4 frames are needed for the largest IU
• The collection of frame(s) that make up a IU are called a Sequence
• A Sequence may be as small as a single Frame
6
Ficon IU Examples
1 Frame IU to transfer a Read CCW
3 Frame IU to transfer 4K of data
SOF HeaderFicon
HeaderCRC EOF
SOF Header CCW Data (2016 Bytes) CRC EOFFiconHeader
SOF Header CCW Data (2048 Bytes) CRC EOF
SOF Header Ficon CRC CRC EOFCCW Data (32 Bytes)
7
Exchanges
• Fibre Channel Architecture defines an Exchange as
• “A mechanism for identifying and managing an operation betweentwo ports“
• All IUs (a.k.a. Sequences) that make up a single I/O operation
are part of an Exchange
8
Ficon Exchanges
• In Ficon, each concurrent I/O operation uses two Exchanges
• One unidirectional Exchange for IUs from the Channel to the CU
• A different unidirectional Exchange for IUs from the CU to the Channel
• The PAIR is commonly know as a “Ficon Exchange”
9
How many Exchanges do I need?
• Little’s Law states:• The number of “things” in a system can be determined by multiplying the
average arrival rate of those “things” by the average time each “thing” stays
in the system
• Applied to Ficon:• The average number of Exchanges active at any given time =
Average I/O rate * Average response time
• Example: 5000 Ficon I/Os / Second on a given channel with 0.4ms service time1 needs 2 Active Exchanges (pairs) at any given time
1 The amount of time the I/O is active in the channel
10
Agenda
Frames, Sequences and Exchanges
IDAW vs MIDAW
Buffer to Buffer Credit
Ficon Recovery
11
What Problem Does MIDAW Solve?
• Extended Format data sets have a small suffix appended to the data.
• Data and Suffix are in DISCONTIGUOUS virtual storage.
• Data and Suffix are in the same physical record on the DASD volume.
• This combination results in less than optimal channel, control unit, and link efficiencies.
12
Non-EF vs EF Performance before MIDAWs
DB2 table scan with 4K pages prior to MIDAWs
13
EF Data in VIRTUAL Storage
User Data4K
User Data4K
User Data4K
User Data4K
Suffix32
Suffix32
Suffix32
Suffix32
In virtual storage the User Data is contiguous
14
EF Data in REAL Storage
User Data½ of 4K
Suffix32
Suffix32
Suffix32
Suffix32
User Data½ of 4K
User Data2/2 of 4K
User Data2/2 of 4K
User Data2/2 of 4K
User Data2/2 of 4K
In REAL storage the User Data is scattered about
User Data½ of 4K
User Data½ of 4K
15
EF Data from REAL Storage to DASD volume
User Data½ of 4K
Suffix32
Suffix32
Suffix32
Suffix32
Data
User Data½ of 4K
User Data2/2 of 4K
User Data2/2 of 4K
User Data2/2 of 4K
User Data2/2 of 4K
Count/Key DataCount/Key
Suffix
DataCount/Key
Suffix
DataCount/Key
Suffix
User Data½ of 4K
User Data½ of 4K
DASD Track
16
How It’s Done With IDAW and Data Chaining
0017546020XX
00465E181000CD,IDAWrite
0017544020CCXX
00465E101000CD,IDAWrite
0017542020CCXX
00465E081000CD,IDAWrite
0017540020CCXX
00465E001000CD,IDAWrite
User Data½ of 4K
Suffix32
Suffix32
Suffix32
Suffix32
User Data½ of 4K
User Data2/2 of 4K
User Data2/2 of 4K
User Data2/2 of 4K
User Data2/2 of 4K
User Data½ of 4K
0BF43000
0D06E800
097F8000
0E015800
0C78D000
02FE9000
03C5B800
03EC4000
User Data½ of 4K
17
Sequence View Of the FC Link with IDAWChannel Write Command + 4096 bytes of data + CRC (3 Frames)
Command Update + 32 byte Suffix + CRC (1 Frame)
Command Response (1 Frame)
Write Command + 4096 bytes of data + CRC (3 Frames)
Command Update + 32 byte Suffix + CRC (1 Frame)
Write Command + 4096 bytes of data + CRC (3 Frames)
Command Update + 32 byte Suffix + CRC (1 Frame)
Write Command + 4096 bytes of data (3 Frames)
Command Update + 32 byte Suffix + CRC (1 Frame)
Ending Status (1 Frame)
Control
Unit
Status Accept
18
What is MIDAW?
• Modified InDirect Address Word
• A new method of gathering and scattering data into & from non-contiguous system z storage locations during an I/O operation.
• Designed to improve performance of certain applications• DB2 sequential workloads that use Media Manager to process small
records with Extended Format data sets
• Reduces the number of CCWs in a channel program for these workloads
19
How It’s Done With MIDAW
465E4481020MIDAWrite
465E301020CC,MIDAWrite
465E181020CC,MIDAWrite
465E001020CC,MIDAWrite
User Data½ of 4K
Suffix32
Suffix32
Suffix32
Suffix32
User Data½ of 4K
User Data2/2 of 4K
User Data2/2 of 4K
User Data2/2 of 4K
User Data2/2 of 4K
User Data½ of 4K
User Data½ of 4K
20
800
800
20
800
800
00175420
0C78D000
02FE9000
00175400
03C5B800
03EC4000… …
20
Sequence View Of the FC Link with MIDAW
ChannelWrite Command + 4128 bytes of data + CRC (3 Frames)
Command Response (1 Frame)
Write Command + 4128 bytes of data + CRC (3 Frames)
Write Command + 4128 bytes of data + CRC (3 Frames)
Write Command + 4128 bytes of data (3 Frames)
Ending Status (1 Frame)
Control Unit
Status Accept
21
IDAW vs MIDAW comparison
MIDAWIDAWMIDAWIDAW
2259Sequences
221317Frames
CU to ChannelChannel to CU
22
6 x 4 k1 2 x 4 k 2 4 x 4 k
3 2 x 4 k
n o m id a w s
w i t h m id a w s
1 8 4
2 6 03 1 3 3 2 7
8 5 9 5 1 0 1 1 0 40
5 0
1 0 0
1 5 0
2 0 0
2 5 0
3 0 0
3 5 0
4 0 0
R E A D M B / se c
s u m m a r y o f F IC O N E x p r e s s 4 c h a n n e l
M ID A W s m e a s u r e m e n ts
n o m id a w s w it h m id a w s
* This performance data was measured in a controlled environment running an I/O driver program under z/OS. The actual
throughput or performance that any user will experience will vary depending upon considerations such as the amount of
multiprogramming in the user’s job stream, the I/O Configuration, the storage configuration, and the workload processed.
See also: http://www.redbooks.ibm.com/redpapers/pdfs/redp4201.pdf
23
Agenda
Frames, Sequences and Exchanges
IDAW vs MIDAW
Buffer to Buffer Credit
Ficon Recovery
24
• The greater the BB Credit….
• A. The faster frames can be sent
• B. The farther apart the two ports can be
• C. The larger the frames can be
• D. None of the above
What is Buffer-to-Buffer Credit?
25
• The greater the BB Credit….
• A.
• B. The farther apart the two ports can be
• C.
• D.
What is Buffer-to-Buffer Credit?
26
• BB Credit is the number of FRAME buffers a port provides for it’s NEAREST neighbor
• BB Credit does NOT have to be symmetrical
What is Buffer-to-Buffer Credit?
27
• BB credit value determines the DISTANCE two nodes can be apart and still maintain full link frame rate
• BB credit is on a FRAME basis, not frame SIZE basis
• A 1 byte frame consumes 1 credit
• A 2K byte frame consumes 1 credit
• Number of credits needed determined by:
• Raw Link Speed
• Distance between two adjacent nodes
• Average frame size
What is Buffer-to-Buffer Credit?
28
• Each time a frame is sent, the sender decrements its available credit count by 1
• Each time a frame receiver clears a frame buffer it sends a “R_RDY”
• Special 4 byte character – NOT a frame !
• Reception of a R_RDY causes the available credit count to be incremented by 1
What is Buffer-to-Buffer Credit?
29
BB-Credit Animations
30
20
Channel
“Perfect” Switch
Control Unit
Initial Conditions
Number of B-B
Credits the
switch advertised
to the channel
during link init
NOTE: In these animations, both the frames and R_RDY’s are numbered. This is for illustrative purposes only. In reality, neither the frames nor the R_RDY’s are numbered. The arrival of an R_RDY only informs the receiver that A frame buffer has been freed, not WHICH frame buffer has been freed.
31
20
32
19
1
33
18
12
34
17
3 12
35
16
3 14 2
36
15
5 3 14 2
37
14
5 3 16 4 2
38
13
7 5 3 16 4 2
39
12
78 5 3 16 4 2
40
11
9 78 5 36 4 2 1
1
41
9 78 5 36 410
10
12
1 2
42
9
9 78 56 41011 3 12
31 2
43
8
9 78 56101112 3 14 2
1 2 3 4
44
7
9 78 610111213 5 3 14 2
1 2 3 4 5
45
6
9 781011121314 5 3 16 4 2
1 2 3 4 5 6
46
5
9 8101112131415 7 5 3 16 4 2
1 2 3 4 5 6 7
47
4
9 8101112131415 7 5 3 16 4 2
1 2 3 4 5 6 7
16
8
48
4
1011121314151617 9 78 5 3 16 4 2
2 3 4 5 6 7 8 9
49
4
1112131415161718 9 78 5 3 16 4 210
3 4 5 6 7 8 9 10
50
4
1213141516171819 9 78 5 36 4 21011
4 5 6 7 8 9 10 11 1
51
This Page Intentionally Left Blank
52
20
Suppose the switch is too far away from the channel for the B-B credit it advertised to the channel
53
19
1
54
18
12
55
17
3 12
56
16
3 14 2
57
15
5 3 14 2
58
14
5 3 16 4 2
59
13
7 5 3 16 4 2
60
12
78 5 3 16 4 2
61
11
78 5 3 16 4 29
62
10
78 5 3 16 4 2910
63
9
78 5 3 16 4 291011
64
8
78 5 3 16 4 29101211
65
7
78 5 3 16 4 2910111213
66
6
78 5 3 16 4 291011121314
67
5
78 5 3 16 4 29101112131415
68
4
78 5 3 16 4 2910111213141516
1
69
3
78 5 3 16 4 291011121314151617
1 2
70
2
78 5 3 16 4 29101112131415161718
1 2 3
71
1
78 5 3 16 4 2910111213141516171819
1 2 3 4
72
0
78 5 3 16 4 291011121314151617181920
1 2 3 4 5
73
0
78 5 3 16 4 291011121314151617181920
1 2 3 4 5 6
74
0
78 5 3 16 4 291011121314151617181920
1 2 3 4 5 6 7
75
0
78 5 3 16 4 291011121314151617181920
1 2 3 4 5 6 7 8
76
0
78 5 3 16 4 291011121314151617181920
1 2 3 4 5 6 7 8 9
77
0
78 5 3 16 4 291011121314151617181920
1 2 3 4 5 6 7 8 9 10
78
0
78 5 36 4 291011121314151617181920
1 2 3 4 5 6 7 8 9 1011 1
79
0
78 5 36 491011121314151617181920
1 2 3 4 5 6 7 8 9 101112 1 2
80
0
78 56 491011121314151617181920
1 2 3 4 5 6 7 8 9 10111213 1 2 3
81
0
78 5691011121314151617181920
1 2 3 4 5 6 7 8 9 1011121314 1 2 3 4
82
0
78 691011121314151617181920
1 2 3 4 5 6 7 8 9 101112131415 1 2 3 4 5
83
0
7891011121314151617181920
2 3 4 5 6 7 8 9 101112
21
13141516 1 2 3 4 5 6
84
0
891011121314151617181920
3 4 5 6 7 8 9 101112
2122
1314151617 1 2 3 4 5 6 7
85
0
91011121314151617181920212322
4 5 6 7 8 9 101112131415161718 1 2 3 4 5 6 7 8
86
0
1011121314151617181920212324 22
5 6 7 8 9 10111213141516171819 1 2 3 4 5 6 7 8 9
87
0
21
1 2 3 4 5 6 7 8 9 10
2324 2225
6 7 8 9 1011121314151617181920
11121314151617181920
88
0
21
2 3 4 5 6 7 8 9 10
2324 222526
7 8 9 1011121314151617181920
121314151617181920
11
89
0
21
3 4 5 6 7 8 9 10
2324 22252627
8 9 1011121314151617181920
1314151617181920
1112
90
0
21
4 5 6 7 8 9 10
2324 2225262728
9 1011121314151617181920
14151617181920
111213
91
0
212324 222526272829
1011121314151617181920
151617181920
5 6 7 8 9 1011121314
92
0
212324 22252627282930
11121314151617181920
1617181920
6 7 8 9 101112131415
93
0
212324 2225262728293031
121314151617181920
17181920
7 8 9 10111213141516
94
0
212324 222526272829303132
1314151617181920
181920
8 9 1011121314151617
95
0
212324 22252627282930313233
14151617181920
1920
9 101112131415161718
96
0
212324 2225262728293031323334
151617181920
20
10111213 151617181914
97
0
212324 222526272829303132333435
1617181920 11121314151617181920
98
0
212324 22252627282930313233343536
181920 2117 121314151617181920
99
0
212324 22252627282930313233343536
181920 13141516171819202122
37
100
0
212324 22252627282930313233343536
1920 14151617181920212223
3738
101
0
212324 22252627282930313233343536
20 15161718192021222324
373839
102
0
212324 22252627282930313233343536
1718192021 25222324
37383940
16
103
0
212324 22252627282930313233343536
17181920212223242526
37383940
104
0
212324 22252627282930313233343536
181920222324252627
37383940
21
105
0
212324 22252627282930313233343536
192022232425262728
37383940
21
106
0
212324 22252627282930313233343536
202223242526272829
37383940
21
107
0
212324 22252627282930313233343536
222324252627282930
37383940
21
108
0
2324 22252627282930313233343536
222324252627282930
37383940
21 31 21
109
0
2324252627282930313233343536
3132
37383940
22232425262728293021 2221
110
This Page Intentionally Left Blank
111
20
20
Suppose there are multiple senders to one receiver
Each sender attempts to send at 100% link speed
112
19
19
1
1
113
18
18
12
12
114
17
17
3 12
3 12
115
16
16
3 14 2
3 14 2
116
15
15
5 3 14 2
5 3 14 2
117
14
14
5 3 16 4 2
5 3 16 4 2
118
13
13
7 5 3 16 4 2
7 5 3 16 4 2
119
12
12
78 5 3 16 4 2
78 5 3 16 4 2
120
11
11
78 5 3
1
6 4 29
78 5 3
1
6 4 29
1
121
10
10
78 5 36 4910
78 5 36 4910
2
11
2
1
1
122
9
9
78 56 491011
78 56 491011
112
3
3
2
1
1 2
123
8
8
78 569101211
78 569101211
1122
3
3
4
4
1 2
1 2
124
7
7
11223
45
34
5
78 6910111213
78 6910111213
1 2
1 2
3
125
6
6
11223
45
4
5
3
6
6
7891011121314
7891011121314
1 2 3
1 2 3
126
5
5
1122334
4
56
5
6
7
89101112131415
89101112131415
7
1 2 3 4
1 2 3
127
4
4
11223344
56
78
5
6
78
910111213141516
910111213141516
1 2 3 4
1 2 3 4
128
4
3
112233445
56
78
9
6
789
1011121314151617
1011121314151617
52 3 4
1 2 3 4
129
3
3
1122334455
6
78
9
6
789
10
10
1112131415161718
1112131415161718
52 3 4
52 3 4
130
3
2
1223344556
789
10
6
78
910
11
11
1213141516171819
1213141516171819
5 63 4
52 3 4
131
2
2
2233445566
78
910
11
12
789
10
1112
1314151617181920
1314151617181920
5 63 4
5 63 4
132
2
1
2334455667
89
10
111213
1415161718192021
1415161718192021
5 6 74
5 63 478
910
11
12
13
133
1
1
3344556677
5 6 748
910
11
12
1314
1516171819202122
89
10
11121314
1516171819202122
5 6 74
134
1
0
3445566778
9
10
11121314
8
910
11
12
1314
15
15
1617181920212223
1617181920212223
5 6 7 8
5 6 74
135
0
0
4455667788
910
11
12
131415
9
10
1112131415
16
1718192021222324
1718192021222324
5 6 7 8
5 6 7 8
16
136
0
0
4556677889
10
1112131415
16
910
11
12
13141516
17
17
18192021222324
1819202122232425
96 7 8
5 6 7 8
137
0
0
5566778899
10
11
12
13141516
17
18
10
1112131415
16
17
19202122232425
18
19202122232425
96 7 8
96 7 8
138
0
0
56677889910
1112131415
16
171819
20212223242526
9 107 8
19
202122232425
10
11
12
13141516
17
18
96 7 8
139
0
0
667788991010
2122232425
19
11
12
13141516
17
18
20
1112131415
16
171819
212223242526
9 107 8
9 107 8
26
20
140
0
0
6778899101011
19
11
12
13141516
17
18
20
2223242526
21
12131415
16
171819
20
2223242526
21
27
9 10 118
9 107 8
141
0
0
778899101011
12131415
16
171819
2021
22
2324252627
11
19
12
13141516
17
18
2021
22
23242526
9 10 118
27
9 10 118
142
0
0
788991010111112
131415
16
171819
2021
22
23
2425262728
19
12
13141516
17
18
2021
22
23
24252627
9 10 118
9 10 11 12
143
0
0
8899101011111212
19
13141516
17
18
2021
22
2324
25262728
131415
16
171819
2021
22
23
24
25262728
9 10 11 12
9 10 11 12
144
0
0
89910101111121213
1415
16
171819
20212223
2425
26272829
1310 11 12
19
13141516
17
18
2021
22
2324
25262728
9 10 11 12
145
0
0
991010111112121313
19
141516
17
18
2021
22
232425
26272829
1310 11 12
26272829
1310 11 12
1415
16
171819
20212223
2425
146
0
0
9101011111212131314
15
16
171819
20
21
22
23242526
26272829
1310 11 12
27282930
13 1011 12
19
141516
17
18
2021
22
232425
147
This Page Intentionally Left Blank
148
20
20
More real life example: Two senders sending at 30% - 50% link rate to one receiver
149
19
19
1
1
150
18
18
12
12
151
17
18
3 12
12
152
16
18
3 14 2
12
153
16
17
3 14 2
3 12
154
16
16
3 14 2
3 14 2
155
16
16
3 14 2
3 14 2
156
15
16
3 14 2
3 14 2
5
157
15
16
3 14 2
3 14 2
5
158
15
16
34 2
34 2
5
1
1
1
159
15
15
34
34
52
11
2
5
1
1
160
14
14
112
3
2
2
4
34
5
5
6
6
1
1
161
14
14
5
5
6
6
1122
34
21
1 2
34
162
14
14
11223
4
3
4
5
5
6
6
21
1 2
3
163
14
14
11223
4
4
3
31 2
5
5
6
6
21 3
164
14
13
1122334
4
4
5
5
6
6
31 2
21 3
7
165
14
13
11223344
5
4
5
6
67
31 2
421 3
166
13
13
112233445
5
5
6
67
421 3
431 2
7
167
14
13
112233445
5
6
67
7
542 3
431 2
168
13
14
12233445
6
67
7
5
5
8
542 3
432
169
13
13
223344556
6
6
7
78
543
5432
8
9
170
12
14
23344556
6543
543
7
78
8
9
6
6
10
171
13
14
33445566
7
78
8
910
654
543 6
172
12
15
3445566
7
78
8
91011
654
54 6
173
13
15
445566
78
8
91011
7
7
65
54 6
174
13
15
45566
78
8
91011
7
7
65
5 6
9
175
14
15
5566
8
8
91011
77
7
7
6
5 6
9
176
13
15
56677
8
8
91011
7
7
6
5 6
9
12
177
13
16
56677
8
91011
8
87
7
6
6
9
12
178
14
16
667788
9
8
1011
87
76
9
12
179
This Page Intentionally Left Blank
180
Fat Pipe / Skinny Pipe
20
8 Gig Link 4 Gig Link
181
19
1
182
18
12
183
17
3 12
184
16
3 14 2
185
15
5 3 14 2
186
14
5 3 16 4 2
187
13
7 5 3 16 4 2
188
12
78 5 3 16 4 2
189
11
9 78 5 36 4 2 1
190
10
9 78 5 36 410 1
2
1
191
9
1
3
9 78 56 41011
1
2
192
8
12
4
3
9 78 56101112
1 2
193
7
123
45
9 78 610111213
1 2
4
3
194
6
123
45
6
9 781011121314
1 2 3
195
5
1234
456
5
7
9 8101112131415
1 2 3
196
4
1234
456
5
7
8
910111213141516
1 2 3 4
197
3
12345
56
7
8
9
1011121314151617
1 2 3 4
198
3
12345
56
7
8
9
10
1112131415161718
52 3 4
199
2
6 12345
7
8
9
10
11
1213141516171819
52 3 4
200
2
6 2345
7
8
9
10
1112
20 13141516171819
5 63 4
201
1
6 23457
8
9
10
1112
13
2021 141516171819
53 4 6
202
1
6 3457
8
9
10
1112
1314
202122 1516171819
5 74 6
203
0
6 34578
9
10
1112
131415
20212223 16171819
5 74 6
204
0
6 4578
9
10
1112
131415
16
20212223 171819
5 7 86
24
205
0
9 6 4578
10
1112
131415
16
17
20212223 181924
5 7 86
206
0
9 6 578
10
1112
131415
16
17
18
23 20212223 1924
97 86
207
0
10 9 6 578
1112
131415
16
17
18
23 20212223
19
24
97 86
208
0
10 9 678
1112
131415
16
17
18
19
2325 21222324
97 8 10
20
209
0
11 10 9 678
12
131415
16
17
18
19
20
2325
21
222324
97 8 10
210
0
11 10 9 78
12
131415
16
17
18
19
20
21
26 2325
22
2324
9 118 10
211
0
12 11 10 9 78
131415
16
17
18
19
20
21
22
26 2325
23
24
9 118 10
212
0
12 11 10 9 8
131415
16
17
18
19
20
21
22
23
26 2325
24
27
9 11 1210
213
0
13 12 11 10 9 8
1415
16
17
18
19
20
21
22
23
24
26 232527
9 11 1210
214
0
13 12 11 10 9
1415
16
17
18
19
20
21
22
23
24
2628 2527
23
1311 1210
215
0
2628 2527
1311 1210
14 13 12 11 10 9
15
16
17
18
19
20
21
22
23
2423
216
0
14 13 12 11 10
15
16
17
18
19
20
21
22
23
2423
25 2628
25
27
1311 12 14
217
• Good “Rule of thumb”
Number of credits needed = 1 + Link speed in Gb/s * Distance in Km
Average Frame Size in Kb
How much credit do I need?
Example: 30 km at 4 Gb/s
1 + 4 * 30 = 61
2
Example: 10 km at 8 Gb/s
1 + 8 * 10 = 41
2
218
How does credit affect Bandwidth?
4 20 36 52 68 84 100 116 132 148
Distance (Km)
10
20
30
40
50
60
70
80
90
100
110
Perc
ent D
ata
Rate
4Gb32 Credits
15 km
20 km
30 km
219
Average Frame Size vs Block Size
Average Frame Size for 28K I/O
600
800
1000
1200
1400
1600
1800
2000
1 2 3 4 5 6 7 8 9 10 11 12
28K X N
Average Frame Size for 4K I/O
600
800
1000
1200
1400
1600
1800
2000
1 2 3 4 5 6 7 8 9 10 11 12
4K X N
Av
era
ge
Fra
me
Siz
e
220
Agenda
Frames, Sequences and Exchanges
IDAW vs MIDAW
Buffer to Buffer Credit
Ficon Recovery
221
Error Detection and Recovery
• Device Level
• Error scope is one single I/O operation
• Link Level
• Error scope is all active I/O on the affected link
• Channel to Switch Link � All I/O active on the channel
• Switch to CU Link � All I/O on that one destination link
• Channel Internal Errors
222
Sequence Number = 0
Sequence Number = 2
Sequence Number = 1
Sequence Number = 3
Sequence Number = 4
Read
Read
Read
Read
Status Accept
CmdResp
Sequence Number = 0
Sequence Number = 1
Sequence Number = 2
Sequence Number = 3
Sequence Number = 4
Data
Data
Data
Data
Sequence Number = 5 Status
Device Level Errors
223
Device Level Errors
Sequence Number = 0
Sequence Number = 2
Sequence Number = 1
Sequence Number = 3
Sequence Number = 4
Read
Read
Read
Read
Status Accept
CmdResp
Sequence Number = 0
Sequence Number = 1
Sequence Number = 2
Sequence Number = 3
Sequence Number = 4
Data
Data
Data
Data
Sequence Number = 5 Status
Channel
Checks
CU
Checks
224
Device Level Errors
Sequence Number = 0
Sequence Number = 2
Sequence Number = 1
Sequence Number = 3
Sequence Number = 4
Read
Read
Read
Read
Status Accept
CmdResp
Sequence Number = 0
Sequence Number = 1
Sequence Number = 2
Sequence Number = 3
Sequence Number = 4
Data
Data
Data
Data
Sequence Number = 5 Status
Channel
Times
CU
Times
225
Device Level Errors
Sequence Number = 0
Sequence Number = 2
Sequence Number = 1
Sequence Number = 3
Sequence Number = 4
Read
Read
Read
Read
Status Accept
CmdResp
Sequence Number = 0
Sequence Number = 1
Sequence Number = 2
Sequence Number = 3
Sequence Number = 4
Data
Data
Data
Data
Sequence Number = 5 Status
Damaged
Or
Discarded
226
Device Level Errors
Sequence Number = 0
Sequence Number = 2
Sequence Number = 1
Sequence Number = 3
Sequence Number = 4
Read
Read
Read
Read
Status Accept
CmdResp
Sequence Number = 0
Sequence Number = 1
Sequence Number = 2
Sequence Number = 4
Data
Data
Data
Data
Sequence Number = 5 Status
Channel Detects Sequence Number
Error
Error Causes Abort
227
Device Level Errors
Purge Path Extended
Purge Path (Extended) Response
Selective Reset (with request for retry)
Status
Or
DACK *
Resend Failing Cmd
Or
LACK
* DACK will result in an IFCC (IOS050I) interrupt to software
228
Link Recovery
• Three initiators:• Link Between Channel and Switch ‘Fails’
• State Change from Switch
• Timeout
• For Remote Links, Channel Attempts a ‘Ping’
• If channel gets a response – all is ok
• If ‘no one home’ response, repeat Ping in 4 seconds• Still ‘no one home’ response – declare link dead
• 5 Link fails in 5 Minutes � Flapping Link Threshold
IOS001E dddd,INOPERATIVE PATHS pp pp pp
IOS2001I dddd,INOPERATIVE PATHS
STATUS FOR PATH(S) pp,pp,pp....
LOGICAL PATH IS REMOVED OR NOT ESTABLISHED (A0)LINK RECOVERY THRESHOLD EXCEEDED FOR LOGICAL PATH (06)
229
Internal Errors
• Firmware detected
• “Shouldn’t be able to get here from there”
• Hardware detected
• Parity Error
• Cross Check
• SAP Detected
• Timeouts
• Out of context messages
230
Internal Error Recovery
• Channel Hardware Reset
• Firmware Reloaded
• Channel is Re-Initialized
• Any in-progress Operations Terminated
231
Thank You For Your Time And Attention
![Beamline Instrumentation: From Insertion Devices to ... · ALBA IVU-2 X=0 it the n mm 7X 7mm coil around sGcm Data 0 5 0 5] 0 0 0 0 10 20] d m m m m m m m m m m m m 0 5 0 5] 0 0 0](https://img.dokumen.tips/doc/110x75/5eb5e7e92d051d4c2f251fc6/beamline-instrumentation-from-insertion-devices-to-alba-ivu-2-x0-it-the-n.jpg)
![-/MJN · ..*"-,/+ 0 ) 20 5 ' 9 )..*--** 0 ; *..*--,/+ 0 $ ' *.=*0*/+*F GH; I K 5 ' ) M%!!9 9 ) ' ' ) %O!"P 5 ' 9 *.=*0*/+*F GH; UI 5 5 5 ' "%! ' I ]!!M 5 ) %O](https://img.dokumen.tips/doc/110x75/6144c1b534130627ed508d6f/mjn-0-20-5-9-0-0-0f-gh-i-k-5.jpg)
