-
Notifications
You must be signed in to change notification settings - Fork 9
/
RFC5048.TXT
2131 lines (1361 loc) · 78.3 KB
/
RFC5048.TXT
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
Network Working Group M. Chadalapaka, Ed.
Request for Comments: 5048 Hewlett-Packard Co.
Updates: 3720 October 2007
Category: Standards Track
Internet Small Computer System Interface (iSCSI)
Corrections and Clarifications
Status of This Memo
This document specifies an Internet standards track protocol for the
Internet community, and requests discussion and suggestions for
improvements. Please refer to the current edition of the "Internet
Official Protocol Standards" (STD 1) for the standardization state
and status of this protocol. Distribution of this memo is unlimited.
Abstract
The Internet Small Computer System Interface (iSCSI) is a SCSI
transport protocol and maps the SCSI architecture and command sets
onto TCP/IP. RFC 3720 defines the iSCSI protocol. This document
compiles the clarifications to the original protocol definition in
RFC 3720 to serve as a companion document for the iSCSI implementers.
This document updates RFC 3720 and the text in this document
supersedes the text in RFC 3720 when the two differ.
Table of Contents
1. Introduction ....................................................3
2. Definitions, Acronyms, and Document Summary .....................3
2.1. Definitions ................................................3
2.2. Acronyms ...................................................4
2.3. Clarifications, Changes, and New Semantics .................5
3. iSCSI Semantics for SCSI Tasks ..................................7
3.1. Residual Handling ..........................................7
3.1.1. Overview ............................................7
3.1.2. SCSI REPORT LUNS and Residual Overflow ..............7
3.2. R2T Ordering ...............................................9
3.3. Model Assumptions for Response Ordering ....................9
3.3.1. Model Description ..................................10
3.3.2. iSCSI Semantics with the Interface Model ...........10
3.3.3. Current List of Fenced Response Use Cases ..........11
4. Task Management ................................................12
4.1. Requests Affecting Multiple Tasks .........................12
4.1.1. Scope of Affected Tasks ............................12
4.1.2. Clarified Multi-Task Abort Semantics ...............13
4.1.3. Updated Multi-Task Abort Semantics .................14
Chadalapaka Standards Track [Page 1]
RFC 5048 iSCSI Corrections and Clarifications October 2007
4.1.4. Affected Tasks Shared across RFC 3720 and
FastAbort Sessions .................................16
4.1.5. Implementation Considerations ......................17
4.1.6. Rationale behind the New Semantics .................17
5. Discovery Semantics ............................................19
5.1. Error Recovery for Discovery Sessions .....................19
5.2. Reinstatement Semantics of Discovery Sessions .............19
5.2.1. Unnamed Discovery Sessions .........................20
5.2.2. Named Discovery Sessions ...........................20
5.3. Target PDUs during Discovery ..............................20
6. Negotiation and Others .........................................21
6.1. TPGT Values ...............................................21
6.2. SessionType Negotiation ...................................21
6.3. Understanding NotUnderstood ...............................21
6.4. Outstanding Negotiation Exchanges .........................22
7. iSCSI Error Handling and Recovery ..............................22
7.1. ITT .......................................................22
7.2. Format Errors .............................................22
7.3. Digest Errors .............................................22
7.4. Message Error Checking ....................................23
8. iSCSI PDUs .....................................................23
8.1. Asynchronous Message ......................................23
8.2. Reject ....................................................24
9. Login/Text Operational Text Keys ...............................24
9.1. TaskReporting .............................................24
10. Security Considerations .......................................25
11. IANA Considerations ...........................................26
11.1. iSCSI-Related IANA Registries ............................26
11.2. iSCSI Opcodes ............................................26
11.3. iSCSI Login/Text Keys ....................................28
11.4. iSCSI Asynchronous Events ................................30
11.5. iSCSI Task Management Function Codes .....................31
11.6. iSCSI Login Response Status Codes ........................32
11.7. iSCSI Reject Reason Codes ................................34
11.8. iSER Opcodes .............................................35
12. References ....................................................36
12.1. Normative References .....................................36
12.2. Informative References ...................................36
Acknowledgements ..................................................37
Chadalapaka Standards Track [Page 2]
RFC 5048 iSCSI Corrections and Clarifications October 2007
1. Introduction
Several iSCSI implementations have been built since [RFC3720] was
published and the iSCSI community is now richer by the resulting
implementation expertise. The goal of this document is to leverage
this expertise both to offer clarifications to the [RFC3720]
semantics and to address defects in [RFC3720] as appropriate. This
document intends to offer critical guidance to implementers with
regard to non-obvious iSCSI implementation aspects so as to improve
interoperability and accelerate iSCSI adoption. This document,
however, does not purport to be an all-encompassing iSCSI how-to
guide for implementers, nor a complete revision of [RFC3720].
Instead, this document is intended as a companion document to
[RFC3720] for the iSCSI implementers.
iSCSI implementers are required to reference [RFC3722] and [RFC3723]
in addition to [RFC3720] for mandatory requirements. In addition,
[RFC3721] also contains useful information for iSCSI implementers.
The text in this document, however, updates and supersedes the text
in [RFC3720] whenever there is such a question.
2. Definitions, Acronyms, and Document Summary
2.1. Definitions
I/O Buffer
A buffer that is used in a SCSI Read or Write operation so SCSI
data may be sent from or received into that buffer. For a read or
write data transfer to take place for a task, an I/O Buffer is
required on the initiator and at least one is required on the
target.
SCSI-Presented Data Transfer Length (SPDTL)
SPDTL is the aggregate data length of the data that the SCSI layer
logically "presents" to the iSCSI layer for a Data-In or Data-Out
transfer in the context of a SCSI task. For a bidirectional task,
there are two SPDTL values -- one for Data-In and one for Data-
Out. Note that the notion of "presenting" includes immediate data
per the data transfer model in [SAM2], and excludes overlapping
data transfers, if any, requested by the SCSI layer.
Third-party
A term used in this document to denote nexus objects (I_T or
I_T_L) and iSCSI sessions that reap the side effects of actions
that take place in the context of a separate iSCSI session, while
being third parties to the action that caused the side effects.
One example of a third-party session is an iSCSI session hosting
Chadalapaka Standards Track [Page 3]
RFC 5048 iSCSI Corrections and Clarifications October 2007
an I_T_L nexus to an LU that is reset with an LU Reset TMF via a
separate I_T nexus.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in [RFC2119].
2.2. Acronyms
Acronym Definition
-------------------------------------------------------------
EDTL Expected Data Transfer Length
IANA Internet Assigned Numbers Authority
IETF Internet Engineering Task Force
I/O Input - Output
IP Internet Protocol
iSCSI Internet SCSI
iSER iSCSI Extensions for RDMA
ITT Initiator Task Tag
LO Leading Only
LU Logical Unit
LUN Logical Unit Number
PDU Protocol Data Unit
RDMA Remote Direct Memory Access
R2T Ready To Transfer
R2TSN Ready To Transfer Sequence Number
RFC Request For Comments
SAM SCSI Architecture Model
SCSI Small Computer Systems Interface
Chadalapaka Standards Track [Page 4]
RFC 5048 iSCSI Corrections and Clarifications October 2007
SN Sequence Number
SNACK Selective Negative Acknowledgment - also
Sequence Number Acknowledgement for data
TCP Transmission Control Protocol
TMF Task Management Function
TTT Target Transfer Tag
UA Unit Attention
2.3. Clarifications, Changes, and New Semantics
This document specifies certain changes to [RFC3720] semantics as
well as defines new iSCSI semantics. In addition, this document also
clarifies the [RFC3720] semantics. This section summarizes the
contents of the document, categorizing each section into one or more
of a clarification, change, or new semantic.
Section 3.1.1: Clarification on iSCSI residuals computation
general principles
Section 3.1.2: Clarification on iSCSI residuals computation with
an example
Section 3.2: Clarification on R2T ordering requirements
Section 3.3: New Semantics for Response Ordering in multi-
connection iSCSI sessions
Section 4.1.2: Clarifications, changes, and new semantics on
multi-task abort semantics that all implementations must comply
with
Section 4.1.3: Changes and new semantics (FastAbort semantics) on
multi-task abort semantics that implementations should use for
faster error recovery
Section 4.1.3.1: Changes in iSCSI clearing effects semantics
resulting from new FastAbort semantics
Section 4.1.4: New Semantics on third-party session interactions
with the new FastAbort semantics
Chadalapaka Standards Track [Page 5]
RFC 5048 iSCSI Corrections and Clarifications October 2007
Section 4.1.5: Clarification on implementation considerations
related to outstanding data transfers in order to realize correct
iSCSI protocol behavior
Section 4.1.6: Clarification on the intent behind FastAbort
semantics (not clarifications to [RFC3720] semantics)
Section 5.1: Clarification on error recovery semantics as
applicable to Discovery sessions
Section 5.2.1: Clarification and new semantics on applying the
Initiator Session Identifier (ISID) RULE ([RFC3720]) to Unnamed
Discovery Sessions
Section 5.2.2: Clarification on applying the ISID RULE to Named
Discovery Sessions
Section 5.3: Clarification on allowed PDU types and target Logout
notification behavior on a Discovery session
Section 6.1: Clarification on the legality of the Target Portal
Group Tag (TPGT) value of zero
Section 6.2: Clarification on the negotiating order of SessionType
with respect to other keys
Section 6.3: Clarification on the NotUnderstood negotiation
response on declarative keys and the implied semantics
Section 6.4: Clarification on the number of legal outstanding
negotiation PDUs (Text or Login-related)
Section 7.1: Clarification on usage of the ITT value of 0xffffffff
Section 7.2: Clarification on what constitutes format errors for
the purpose of error recovery defined in [RFC3720]
Section 7.3: Change in error recovery semantics for the case of
discarding unsolicited PDUs
Section 7.4: Clarification on the intended level of error checking
on inbound PDUs
Section 8.1: New semantics for a new AsyncEvent code
Section 8.2: Change of legal status for Reject reason code 0x0b;
it is now deprecated
Chadalapaka Standards Track [Page 6]
RFC 5048 iSCSI Corrections and Clarifications October 2007
Section 9.1: New semantics for a new text key TaskReporting
3. iSCSI Semantics for SCSI Tasks
3.1. Residual Handling
Section 10.4.1 of [RFC3720] defines the notion of "residuals" and
specifies how the residual information should be encoded into the
SCSI Response PDU in the Counts and Flags fields. Section 3.1.1
clarifies the intent of [RFC3720] and explains the general
principles. Section 3.1.2 describes the residual handling in the
REPORT LUNS scenario.
3.1.1. Overview
SCSI-Presented Data Transfer Length (SPDTL) is the term this document
uses (see Section 1.1 for definition) to represent the aggregate data
length that the target SCSI layer attempts to transfer using the
local iSCSI layer for a task. Expected Data Transfer Length (EDTL)
is the iSCSI term that represents the length of data that the iSCSI
layer expects to transfer for a task. EDTL is specified in the SCSI
Command PDU.
When SPDTL = EDTL for a task, the target iSCSI layer completes the
task with no residuals. Whenever SPDTL differs from EDTL for a task,
that task is said to have a residual.
If SPDTL > EDTL for a task, iSCSI Overflow MUST be signaled in the
SCSI Response PDU as specified in [RFC3720]. The Residual Count MUST
be set to the numerical value of (SPDTL - EDTL).
If SPDTL < EDTL for a task, iSCSI Underflow MUST be signaled in the
SCSI Response PDU as specified in [RFC3720]. The Residual Count MUST
be set to the numerical value of (EDTL - SPDTL).
Note that the Overflow and Underflow scenarios are independent of
Data-In and Data-Out. Either scenario is logically possible in
either direction of data transfer.
3.1.2. SCSI REPORT LUNS and Residual Overflow
This section discusses the residual overflow issues citing the
example of the SCSI REPORT LUNS command. Note however that there are
several SCSI commands (e.g., INQUIRY) with ALLOCATION LENGTH fields
following the same underlying rules. The semantics in the rest of
the section apply to all such SCSI commands.
Chadalapaka Standards Track [Page 7]
RFC 5048 iSCSI Corrections and Clarifications October 2007
The specification of the SCSI REPORT LUNS command requires that the
SCSI target limit the amount of data transferred to a maximum size
(ALLOCATION LENGTH) provided by the initiator in the REPORT LUNS CDB.
If the Expected Data Transfer Length (EDTL) in the iSCSI header of
the SCSI Command PDU for a REPORT LUNS command is set to at least as
large as that ALLOCATION LENGTH, the SCSI layer truncation prevents
an iSCSI Residual Overflow from occurring. A SCSI initiator can
detect that such truncation has occurred via other information at the
SCSI layer. The rest of the section elaborates this required
behavior.
iSCSI uses the (O) bit (bit 5) in the Flags field of the SCSI
Response and the last SCSI Data-In PDUs to indicate that an iSCSI
target was unable to transfer all of the SCSI data for a command to
the initiator because the amount of data to be transferred exceeded
the EDTL in the corresponding SCSI Command PDU (see Section 10.4.1 of
[RFC3720]).
The SCSI REPORT LUNS command requests a target SCSI layer to return a
logical unit inventory (LUN list) to the initiator SCSI layer (see
Section 6.21 of SPC-3 [SPC3]). The size of this LUN list may not be
known to the initiator SCSI layer when it issues the REPORT LUNS
command; to avoid transferring more LUN list data than the initiator
is prepared for, the REPORT LUNS CDB contains an ALLOCATION LENGTH
field to specify the maximum amount of data to be transferred to the
initiator for this command. If the initiator SCSI layer has under-
estimated the number of logical units at the target, it is possible
that the complete logical unit inventory does not fit in the
specified ALLOCATION LENGTH. In this situation, Section 4.3.3.6 in
[SPC3] requires that the target SCSI layer "shall terminate transfers
to the Data-In Buffer" when the number of bytes specified by the
ALLOCATION LENGTH field have been transferred.
Therefore, in response to a REPORT LUNS command, the SCSI layer at
the target presents at most ALLOCATION LENGTH bytes of data (logical
unit inventory) to iSCSI for transfer to the initiator. For a REPORT
LUNS command, if the iSCSI EDTL is at least as large as the
ALLOCATION LENGTH, the SCSI truncation ensures that the EDTL will
accommodate all of the data to be transferred. If all of the logical
unit inventory data presented to the iSCSI layer -- i.e., the data
remaining after any SCSI truncation -- is transferred to the
initiator by the iSCSI layer, an iSCSI Residual Overflow has not
occurred and the iSCSI (O) bit MUST NOT be set in the SCSI Response
or final SCSI Data-Out PDU. This is not a new requirement but is
already required by the combination of [RFC3720] with the
specification of the REPORT LUNS command in [SPC3]. However, if the
iSCSI EDTL is larger than the ALLOCATION LENGTH in this scenario,
note that the iSCSI Underflow MUST be signaled in the SCSI Response
Chadalapaka Standards Track [Page 8]
RFC 5048 iSCSI Corrections and Clarifications October 2007
PDU. An iSCSI Underflow MUST also be signaled when the iSCSI EDTL is
equal to the ALLOCATION LENGTH but the logical unit inventory data
presented to the iSCSI layer is smaller than the ALLOCATION LENGTH.
The LUN LIST LENGTH field in the logical unit inventory (the first
field in the inventory) is not affected by truncation of the
inventory to fit in ALLOCATION LENGTH; this enables a SCSI initiator
to determine that the received inventory is incomplete by noticing
that the LUN LIST LENGTH in the inventory is larger than the
ALLOCATION LENGTH that was sent in the REPORT LUNS CDB. A common
initiator behavior in this situation is to re-issue the REPORT LUNS
command with a larger ALLOCATION LENGTH.
3.2. R2T Ordering
Section 10.8 in [RFC3720] says the following:
The target may send several R2T PDUs. It, therefore, can have a
number of pending data transfers. The number of outstanding R2T
PDUs is limited by the value of the negotiated key
MaxOutstandingR2T. Within a connection, outstanding R2Ts MUST be
fulfilled by the initiator in the order in which they were
received.
The quoted [RFC3720] text was unclear on the scope of applicability
-- either per task, or across all tasks on a connection -- and may be
interpreted as either. This section is intended to clarify that the
scope of applicability of the quoted text is a task. No R2T ordering
relationship -- either in generation at the target or in fulfilling
at the initiator -- across tasks is implied. That is, outstanding
R2Ts within a task MUST be fulfilled by the initiator in the order in
which they were received on a connection.
3.3. Model Assumptions for Response Ordering
Whenever an iSCSI session is composed of multiple connections, the
Response PDUs (task responses or TMF responses) originating in the
target SCSI layer are distributed onto the multiple connections by
the target iSCSI layer according to iSCSI connection allegiance
rules. This process generally may not preserve the ordering of the
responses by the time they are delivered to the initiator SCSI layer.
Since ordering is not expected across SCSI responses anyway, this
approach works fine in the general case. However, to address the
special cases where some ordering is desired by the SCSI layer, the
following "Response Fence" semantics are defined with respect to
handling SCSI response messages as they are handed off from the SCSI
protocol layer to the iSCSI layer.
Chadalapaka Standards Track [Page 9]
RFC 5048 iSCSI Corrections and Clarifications October 2007
3.3.1. Model Description
The target SCSI protocol layer hands off the SCSI response messages
to the target iSCSI layer by invoking the "Send Command Complete"
protocol data service ([SAM2], clause 5.4.2) and "Task Management
Function Executed" ([SAM2], clause 6.9) service. On receiving the
SCSI response message, the iSCSI layer exhibits the Response Fence
behavior for certain SCSI response messages (Section 3.3.3 describes
the specific instances where the semantics must be realized).
Whenever the Response Fence behavior is required for a SCSI response
message, the target iSCSI layer MUST ensure that the following
conditions are met in delivering the response message to the
initiator iSCSI layer:
(1) Response with Response Fence MUST be delivered chronologically
after all the "preceding" responses on the I_T_L nexus, if the
preceding responses are delivered at all, to the initiator iSCSI
layer.
(2) Response with Response Fence MUST be delivered chronologically
prior to all the "following" responses on the I_T_L nexus.
The "preceding" and "following" notions refer to the order of handoff
of a response message from the target SCSI protocol layer to the
target iSCSI layer.
3.3.2. iSCSI Semantics with the Interface Model
Whenever the TaskReporting key (Section 9.1) is negotiated to
ResponseFence or FastAbort for an iSCSI session and the Response
Fence behavior is required for a SCSI response message, the target
iSCSI layer MUST perform the actions described in this section for
that session.
a) If it is a single-connection session, no special processing is
required. The standard SCSI Response PDU build and dispatch
process happens.
b) If it is a multi-connection session, the target iSCSI layer
takes note of the last-sent and unacknowledged StatSN on each
of the connections in the iSCSI session, and waits for an
acknowledgement (NOP-In PDUs MAY be used to solicit
acknowledgements as needed in order to accelerate this process)
of each such StatSN to clear the fence. The SCSI response
requiring Response Fence behavior MUST NOT be sent to the
initiator before acknowledgements are received for each of the
unacknowledged StatSNs.
Chadalapaka Standards Track [Page 10]
RFC 5048 iSCSI Corrections and Clarifications October 2007
c) The target iSCSI layer must wait for an acknowledgement of the
SCSI Response PDU that carried the SCSI response requiring the
Response Fence behavior. The fence MUST be considered cleared
only after receiving the acknowledgement.
d) All further status processing for the LU is resumed only after
clearing the fence. If any new responses for the I_T_L nexus
are received from the SCSI layer before the fence is cleared,
those Response PDUs MUST be held and queued at the iSCSI layer
until the fence is cleared.
3.3.3. Current List of Fenced Response Use Cases
This section lists the fenced response use cases that iSCSI
implementations MUST comply with. However, this is not an exhaustive
enumeration. It is expected that as SCSI protocol specifications
evolve, the specifications will specify when response fencing is
required on a case-by-case basis.
Whenever the TaskReporting key (Section 9.1) is negotiated to
ResponseFence or FastAbort for an iSCSI session, the target iSCSI
layer MUST assume that the Response Fence is required for the
following SCSI completion messages:
1. The first completion message carrying the UA after the multi-
task abort on issuing and third-party sessions. See Section
4.1.1 for related TMF discussion.
2. The TMF Response carrying the multi-task TMF Response on the
issuing session.
3. The completion message indicating ACA establishment on the
issuing session.
4. The first completion message carrying the ACA ACTIVE status
after ACA establishment on issuing and third-party sessions.
5. The TMF Response carrying the Clear ACA response on the issuing
session.
6. The response to a PERSISTENT RESERVE OUT/PREEMPT AND ABORT
command.
Note: Due to the absence of ACA-related fencing requirements in
[RFC3720], initiator implementations SHOULD NOT use ACA on multi-
connection iSCSI sessions to targets complying only with [RFC3720].
Initiators that want to employ ACA on multi-connection iSCSI sessions
Chadalapaka Standards Track [Page 11]
RFC 5048 iSCSI Corrections and Clarifications October 2007
SHOULD first assess response-fencing behavior via negotiating for
ResponseFence or FastAbort values for the TaskReporting (Section 9.1)
key.
4. Task Management
4.1. Requests Affecting Multiple Tasks
This section clarifies and updates the original text in Section
10.6.2 of [RFC3720]. The clarified semantics (Section 4.1.2) are a
superset of the protocol behavior required in the original text and
all iSCSI implementations MUST support the new behavior. The updated
semantics (Section 4.1.3) on the other hand are mandatory only when
the new key TaskReporting (Section 9.1) is negotiated to "FastAbort".
4.1.1. Scope of Affected Tasks
This section defines the notion of "affected tasks" in multi-task
abort scenarios. Scope definitions in this section apply to both the
clarified protocol behavior (Section 4.1.2) and the updated protocol
behavior (Section 4.1.3).
ABORT TASK SET: All outstanding tasks for the I_T_L nexus
identified by the LUN field in the ABORT TASK SET TMF Request
PDU.
CLEAR TASK SET: All outstanding tasks in the task set for the LU
identified by the LUN field in the CLEAR TASK SET TMF Request
PDU. See [SPC3] for the definition of a "task set".
LOGICAL UNIT RESET: All outstanding tasks from all initiators for
the LU identified by the LUN field in the LOGICAL UNIT RESET
Request PDU.
TARGET WARM RESET/TARGET COLD RESET: All outstanding tasks from
all initiators across all LUs to which the TMF-issuing session
has access on the SCSI target device hosting the iSCSI session.
Usage: An "ABORT TASK SET TMF Request PDU" in the preceding text is
an iSCSI TMF Request PDU with the "Function" field set to "ABORT TASK
SET" as defined in [RFC3720]. Similar usage is employed for other
scope descriptions.
Chadalapaka Standards Track [Page 12]
RFC 5048 iSCSI Corrections and Clarifications October 2007
4.1.2. Clarified Multi-Task Abort Semantics
All iSCSI implementations MUST support the protocol behavior defined
in this section as the default behavior. The execution of ABORT TASK
SET, CLEAR TASK SET, LOGICAL UNIT RESET, TARGET WARM RESET, and
TARGET COLD RESET TMF Requests consists of the following sequence of
actions in the specified order on the specified party.
The initiator iSCSI layer:
a. MUST continue to respond to each TTT received for the affected
tasks.
b. SHOULD process any responses received for affected tasks in the
normal fashion. This is acceptable because the responses are
guaranteed to have been sent prior to the TMF response.
c. SHOULD receive the TMF Response concluding all the tasks in the
set of affected tasks unless the initiator has done something
(e.g., LU reset, connection drop) that may prevent the TMF
Response from being sent or received. The initiator MUST thus
conclude all affected tasks as part of this step in either
case, and MUST discard any TMF Response received after the
affected tasks are concluded.
The target iSCSI layer:
a. MUST wait for responses on currently valid target-transfer tags
of the affected tasks from the issuing initiator. MAY wait for
responses on currently valid target-transfer tags of the
affected tasks from third-party initiators.
b. MUST wait (concurrent with the wait in Step a) for all commands
of the affected tasks to be received based on the CmdSN
ordering. SHOULD NOT wait for new commands on third-party
affected sessions -- only the instantiated tasks have to be
considered for the purpose of determining the affected tasks.
In the case of target-scoped requests (i.e., TARGET WARM RESET
and TARGET COLD RESET), all of the commands that are not yet
received on the issuing session in the command stream however
can be considered to have been received with no command waiting
period -- i.e., the entire CmdSN space up to the CmdSN of the
task management function can be "plugged".
c. MUST propagate the TMF request to and receive the response from
the target SCSI layer.
Chadalapaka Standards Track [Page 13]
RFC 5048 iSCSI Corrections and Clarifications October 2007
d. MUST provide the Response Fence behavior for the TMF Response
on the issuing session as specified in Section 3.3.2.
e. MUST provide the Response Fence behavior on the first post-TMF
Response on third-party sessions as specified in Section 3.3.2.
If some tasks originate from non-iSCSI I_T_L nexuses, then the
means by which the target ensures that all affected tasks have
returned their status to the initiator are defined by the
specific non-iSCSI transport protocol(s).
Technically, the TMF servicing is complete in Step d. Data transfers
corresponding to terminated tasks may however still be in progress on
third-party iSCSI sessions even at the end of Step e. The TMF
Response MUST NOT be sent by the target iSCSI layer before the end of
Step d, and MAY be sent at the end of Step d despite these
outstanding data transfers until after Step e.
4.1.3. Updated Multi-Task Abort Semantics
Protocol behavior defined in this section MUST be implemented by all
iSCSI implementations complying with this document. Protocol
behavior defined in this section MUST be exhibited by iSCSI
implementations on an iSCSI session when they negotiate the
TaskReporting (Section 9.1) key to "FastAbort" on that session. The
execution of ABORT TASK SET, CLEAR TASK SET, LOGICAL UNIT RESET,
TARGET WARM RESET, and TARGET COLD RESET TMF Requests consists of the
following sequence of actions in the specified order on the specified
party.
The initiator iSCSI layer:
a. MUST NOT send any more Data-Out PDUs for affected tasks on the
issuing connection of the issuing iSCSI session once the TMF is
sent to the target.
b. SHOULD process any responses received for affected tasks in the
normal fashion. This is acceptable because the responses are
guaranteed to have been sent prior to the TMF response.
c. MUST respond to each Async Message PDU with AsyncEvent=5 as
defined in Section 8.1.
d. MUST treat the TMF response as terminating all affected tasks
for which responses have not been received, and MUST discard
any responses for affected tasks received after the TMF
response is passed to the SCSI layer (although the semantics
Chadalapaka Standards Track [Page 14]
RFC 5048 iSCSI Corrections and Clarifications October 2007
defined in this section ensure that such an out-of-order
scenario will never happen with a compliant target
implementation).
The target iSCSI layer:
a. MUST wait for all commands of the affected tasks to be received
based on the CmdSN ordering on the issuing session. SHOULD NOT
wait for new commands on third-party affected sessions -- only
the instantiated tasks have to be considered for the purpose of
determining the affected tasks. In the case of target-scoped
requests (i.e., TARGET WARM RESET and TARGET COLD RESET), all
the commands that are not yet received on the issuing session
in the command stream can be considered to have been received
with no command waiting period -- i.e., the entire CmdSN space
up to the CmdSN of the task management function can be
"plugged".
b. MUST propagate the TMF request to and receive the response from
the target SCSI layer.
c. MUST leave all active "affected TTTs" (i.e., active TTTs
associated with affected tasks) valid.
d. MUST send an Asynchronous Message PDU with AsyncEvent=5
(Section 8.1) on:
i) each connection of each third-party session to which at
least one affected task is allegiant if
TaskReporting=FastAbort is operational on that third-party
session, and
ii) each connection except the issuing connection of the
issuing session that has at least one allegiant affected
task.
If there are multiple affected LUs (say, due to a target reset),
then one Async Message PDU MUST be sent for each such LU on each
connection that has at least one allegiant affected task. The LUN
field in the Asynchronous Message PDU MUST be set to match the LUN
for each such LU.
e. MUST address the Response Fence flag on the TMF Response on the
issuing session as defined in Section 3.3.2.
f. MUST address the Response Fence flag on the first post-TMF
Response on third-party sessions as defined in Section 3.3.2.
If some tasks originate from non-iSCSI I_T_L nexuses, then the
Chadalapaka Standards Track [Page 15]
RFC 5048 iSCSI Corrections and Clarifications October 2007
means by which the target ensures that all affected tasks have
returned their status to the initiator are defined by the
specific non-iSCSI transport protocol(s).
g. MUST free up the affected TTTs (and STags, if applicable) and
the corresponding buffers, if any, once it receives each
associated NOP-Out acknowledgement that the initiator generated
in response to each Async Message.
Technically, the TMF servicing is complete in Step e. Data transfers
corresponding to terminated tasks may however still be in progress
even at the end of Step f. A TMF Response MUST NOT be sent by the
target iSCSI layer before the end of Step e, and MAY be sent at the
end of Step e despite these outstanding Data transfers until Step g.
Step g specifies an event to free up any such resources that may have
been reserved to support outstanding data transfers.
4.1.3.1. Clearing Effects Update
Appendix F.1 of [RFC3720] specifies the clearing effects of target
and LU resets on "Incomplete TTTs" as "Y". This meant that a target
warm reset or a target cold reset or an LU reset would clear the
active TTTs upon completion. However, the TaskReporting=FastAbort
(Section 9.1) semantics defined by this section do not guarantee that
the active TTTs are cleared by the end of the reset operations. In
fact, the new semantics are designed to allow clearing the TTTs in a
"lazy" fashion after the TMF Response is delivered. Thus, when
TaskReporting=FastAbort is operational on a session, the clearing
effects of reset operations on "Incomplete TTTs" is "N".
4.1.4. Affected Tasks Shared across RFC 3720 and FastAbort Sessions
If an iSCSI target implementation is capable of supporting
TaskReporting=FastAbort functionality (Section 9.1), it may end up in
a situation where some sessions have TaskReporting=RFC3720
operational (RFC 3720 sessions) while some other sessions have
TaskReporting=FastAbort operational (FastAbort sessions) even while
accessing a shared set of affected tasks (Section 4.1.1).
If the issuing session is an RFC 3720 session, the iSCSI target
implementation is FastAbort-capable, and the third-party affected
session is a FastAbort session, the following behavior SHOULD be
exhibited by the iSCSI target layer:
a. Between Steps c and d of the target behavior in Section 4.1.2,
send an Asynchronous Message PDU with AsyncEvent=5 (Section
8.1) on each connection of each third-party session to which at
least one affected task is allegiant. If there are multiple
Chadalapaka Standards Track [Page 16]
RFC 5048 iSCSI Corrections and Clarifications October 2007
affected LUs, then send one Async Message PDU for each such LU
on each connection that has at least one allegiant affected
task. When sent, the LUN field in the Asynchronous Message PDU
MUST be set to match the LUN for each such LU.
b. After Step e of the target behavior in Section 4.1.2, free up
the affected TTTs (and STags, if applicable) and the
corresponding buffers, if any, once each associated NOP-Out
acknowledgement is received that the third-party initiator
generated in response to each Async Message sent in Step a.
If the issuing session is a FastAbort session, the iSCSI target
implementation is FastAbort-capable, and the third-party affected
session is an RFC 3720 session, the following behavior MUST be
exhibited by the iSCSI target layer: Asynchronous Message PDUs MUST
NOT be sent on the third-party session to prompt the FastAbort
behavior.
If the third-party affected session is a FastAbort session and the
issuing session is a FastAbort session, the initiator in the third-
party role MUST respond to each Async Message PDU with AsyncEvent=5
as defined in Section 8.1. Note that an initiator MAY thus receive
these Async Messages on a third-party affected session even if the
session is a single-connection session.
4.1.5. Implementation Considerations
Both in clarified semantics (Section 4.1.2) and updated semantics
(Section 4.1.3), there may be outstanding data transfers even after
the TMF completion is reported on the issuing session. In the case
of iSCSI/iSER [iSER], these would be tagged data transfers for STags
not owned by any active tasks. Whether or not real buffers support
these data transfers is implementation-dependent. However, the data
transfers logically MUST be silently discarded by the target iSCSI
layer in all cases. A target MAY, on an implementation-defined
internal timeout, also choose to drop the connections on which it did
not receive the expected Data-Out sequences (Section 4.1.2) or NOP-
Out acknowledgements (Section 4.1.3) so as to reclaim the associated
buffer, STag, and TTT resources as appropriate.
4.1.6. Rationale behind the New Semantics
There are fundamentally three basic objectives behind the semantics
specified in Sections 4.1.2 and 4.1.3.
1. Maintaining an ordered command flow I_T nexus abstraction to
the target SCSI layer even with multi-connection sessions.
Chadalapaka Standards Track [Page 17]
RFC 5048 iSCSI Corrections and Clarifications October 2007
o Target iSCSI processing of a TMF request must maintain the
single flow illusion. Target behavior in Step b of Section
4.1.2 and Step a of Section 4.1.3 correspond to this
objective.
2. Maintaining a single ordered response flow I_T nexus
abstraction to the initiator SCSI layer even with multi-
connection sessions when one response (i.e., TMF response)
could imply the status of other unfinished tasks from the
initiator's perspective.
o The target must ensure that the initiator does not see "old"
task responses (that were placed on the wire chronologically
earlier than the TMF Response) after seeing the TMF
response. The target behavior in Step d of Section 4.1.2
and Step e of Section 4.1.3 correspond to this objective.
o Whenever the result of a TMF action is visible across
multiple I_T_L nexuses, [SAM2] requires the SCSI device
server to trigger a UA on each of the other I_T_L nexuses.
Once an initiator is notified of such an UA, the application
client on the receiving initiator is required to clear its
task state (clause 5.5 in [SAM2]) for the affected tasks.
It would thus be inappropriate to deliver a SCSI Response
for a task after the task state is cleared on the initiator,
i.e., after the UA is notified. The UA notification
contained in the first SCSI Response PDU on each affected
Third-party I_T_L nexus after the TMF action thus MUST NOT
pass the affected task responses on any of the iSCSI
sessions accessing the LU. The target behavior in Step e of
Section 4.1.2 and Step f of Section 4.1.3 correspond to this
objective.
3. Draining all active TTTs corresponding to affected tasks in a
deterministic fashion.
o Data-Out PDUs with stale TTTs arriving after the tasks are
terminated can create a buffer management problem even for
traditional iSCSI implementations, and is fatal for the
connection for iSCSI/iSER implementations. Either the
termination of affected tasks should be postponed until the
TTTs are retired (as in Step a of Section 4.1.2), or the