1 ../imapserver/server.go:143
2
3
4
5
6
7Network Working Group A. Melnikov
8Request for Comments: 5182 Isode Ltd.
9Updates: 3501 March 2008
10Category: Standards Track
11
12
13 IMAP Extension for Referencing the Last SEARCH Result 9051:3784 9051:4040 ../imapserver/search.go:215 ../imapserver/server.go:193
14
15Status of This Memo
16
17 This document specifies an Internet standards track protocol for the
18 Internet community, and requests discussion and suggestions for
19 improvements. Please refer to the current edition of the "Internet
20 Official Protocol Standards" (STD 1) for the standardization state
21 and status of this protocol. Distribution of this memo is unlimited.
22
23Abstract
24
25 Many IMAP clients use the result of a SEARCH command as the input to
26 perform another operation, for example, fetching the found messages,
27 deleting them, or copying them to another mailbox.
28
29 This can be achieved using standard IMAP operations described in RFC
30 3501; however, this would be suboptimal. The server will send the
31 list of found messages to the client; after that, the client will
32 have to parse the list, reformat it, and send it back to the server.
33 The client can't pipeline the SEARCH command with the subsequent
34 command, and, as a result, the server might not be able to perform
35 some optimizations.
36
37 This document proposes an IMAP extension that allows a client to tell
38 a server to use the result of a SEARCH (or Unique Identifier (UID)
39 SEARCH) command as an input to any subsequent command.
40
411. Introduction
42
43 Many IMAP clients use the result of a SEARCH command as the input to
44 perform another operation, for example, fetching the found messages,
45 deleting them, or copying them to another mailbox.
46
47 This document proposes an IMAP extension that allows a client to tell
48 a server to use the result of a SEARCH (or UID SEARCH) command as an
49 input to any subsequent command.
50
51 The SEARCH result reference extension defines a new SEARCH result
52 option [IMAPABNF] "SAVE" that tells the server to remember the result
53 of the SEARCH or UID SEARCH command (as well as any command based on
54 SEARCH, e.g., SORT and THREAD [SORT]) and store it in an internal
55
56
57
58Melnikov Standards Track [Page 1]
59
60RFC 5182 Last SEARCH Result Reference March 2008
61
62
63 variable that we will reference as the "search result variable". The
64 client can use the "$" marker to reference the content of this
65 internal variable. The "$" marker can be used instead of message
66 sequence or UID sequence in order to indicate that the server should
67 substitute it with the list of messages from the search result
68 variable. Thus, the client can use the result of the latest
69 remembered SEARCH command as a parameter to another command. The
70 search result marker has several advantages:
71
72 * it avoids wasted bandwidth and associated delay;
73
74 * it allows the client to pipeline a SEARCH [IMAP4] command with a
75 subsequent FETCH/STORE/COPY/SEARCH [IMAP4] or UID EXPUNGE
76 [UIDPLUS] command;
77
78 * the client doesn't need to spend time reformatting the result of
79 a SEARCH command into a message set used in the subsequent
80 command;
81
82 * it allows the server to perform optimizations. For example, if
83 the server can execute several pipelined commands in parallel
84 (or out of order), presence of the search result marker can
85 allow the server to decide which commands may or may not be
86 executed out of order.
87
88 In absence of any other SEARCH result option, the SAVE result option
89 also suppresses any SEARCH response that would have been otherwise
90 returned by the SEARCH command.
91
921.1. Conventions Used in This Document
93
94 In examples, "C:" indicates lines sent by a client that is connected
95 to a server. "S:" indicates lines sent by the server to the client.
96
97 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
98 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
99 document are to be interpreted as described in [KEYWORDS].
100
101 Explanatory comments in examples start with // and are not part of
102 the protocol.
103
104
105
106
107
108
109
110
111
112
113
114Melnikov Standards Track [Page 2]
115
116RFC 5182 Last SEARCH Result Reference March 2008
117
118
1192. Overview
120
1212.1. Normative Description of the SEARCHRES Extension
122
123 The SEARCH result reference extension described in this document is
124 present in any IMAP4 server implementation that returns "SEARCHRES"
125 as one of the supported capabilities in the CAPABILITY command
126 response. Any such server MUST also implement the [ESEARCH]
127 extension.
128
129 Upon successful completion of a SELECT or an EXAMINE command (after
130 the tagged OK response), the current search result variable is reset
131 to the empty sequence.
132
133 A successful SEARCH command with the SAVE result option sets the
134 value of the search result variable to the list of messages found in
135 the SEARCH command. For example, if no messages were found, the
136 search result variable will contain the empty list.
137
138 Any of the following SEARCH commands MUST NOT change the search
139 result variable:
140
141 o a SEARCH command that caused the server to return the BAD tagged
142 response,
143
144 o a SEARCH command with no SAVE result option that caused the
145 server to return NO tagged response,
146
147 o a successful SEARCH command with no SAVE result option.
148
149 A SEARCH command with the SAVE result option that caused the server
150 to return the NO tagged response sets the value of the search result
151 variable to the empty sequence.
152
153 When a message listed in the search result variable is EXPUNGEd, it
154 is automatically removed from the list. Implementors are reminded
155 that if the server stores the list as a list of message numbers, it
156 MUST automatically adjust them when notifying the client about
157 expunged messages, as described in Section 7.4.1 of [IMAP4].
158
159 If the server decides to send a new UIDVALIDITY value while the
160 mailbox is opened, this causes resetting of the search variable to
161 the empty list.
162
163
164
165
166
167
168
169
170Melnikov Standards Track [Page 3]
171
172RFC 5182 Last SEARCH Result Reference March 2008
173
174
175 Note that even if the "$" marker contains the empty list of messages,
176 it must be treated by all commands accepting message sets as
177 parameters as a valid, but non-matching list of messages. For
178 example, the "FETCH $" command would return a tagged OK response and
179 no FETCH responses. See also the Example 5 below.
180
181 Note that even if the "$" marker contains the empty list of messages,
182 it must be treated as a valid but non-matching list of messages, by
183 all commands that accept message sets as parameters.
184
185 Implementation note: server implementors should note that "$" can
186 reference IMAP message sequences or UID sequences, depending on the
187 context where it is used. For example, the "$" marker can be set as
188 a result of a SEARCH (SAVE) command and used as a parameter to a UID
189 FETCH command (which accepts a UID sequence, not a message sequence),
190 or the "$" marker can be set as a result of a UID SEARCH (SAVE)
191 command and used as a parameter to a FETCH command (which accepts a
192 message sequence, not a UID sequence).
193
1942.2. Examples
195
196 1) The following example demonstrates how the client can use the
197 result of a SEARCH command to FETCH headers of interesting
198 messages:
199
200 Example 1:
201 C: A282 SEARCH RETURN (SAVE) FLAGGED SINCE 1-Feb-1994
202 NOT FROM "Smith"
203 S: A282 OK SEARCH completed, result saved
204 C: A283 FETCH $ (UID INTERNALDATE FLAGS RFC822.HEADER)
205 S: * 2 FETCH (UID 14 ...
206 S: * 84 FETCH (UID 100 ...
207 S: * 882 FETCH (UID 1115 ...
208 S: A283 OK completed
209
210 The client can also pipeline the two commands:
211
212 Example 2:
213 C: A282 SEARCH RETURN (SAVE) FLAGGED SINCE 1-Feb-1994
214 NOT FROM "Smith"
215 C: A283 FETCH $ (UID INTERNALDATE FLAGS RFC822.HEADER)
216 S: A282 OK SEARCH completed
217 S: * 2 FETCH (UID 14 ...
218 S: * 84 FETCH (UID 100 ...
219 S: * 882 FETCH (UID 1115 ...
220 S: A283 OK completed
221
222
223
224
225
226Melnikov Standards Track [Page 4]
227
228RFC 5182 Last SEARCH Result Reference March 2008
229
230
231 2) The following example demonstrates that the result of one SEARCH
232 command can be used as input to another SEARCH command:
233
234 Example 3:
235 C: A300 SEARCH RETURN (SAVE) SINCE 1-Jan-2004
236 NOT FROM "Smith"
237 S: A300 OK SEARCH completed
238 C: A301 UID SEARCH UID $ SMALLER 4096
239 S: * SEARCH 17 900 901
240 S: A301 OK completed
241
242 Note that the second command in Example 3 can be replaced with:
243 C: A301 UID SEARCH $ SMALLER 4096
244 and the result of the command would be the same.
245
246 3) The following example shows that the "$"
247 marker can be combined with other message numbers using the OR
248 SEARCH criterion.
249
250 Example 4:
251 C: P282 SEARCH RETURN (SAVE) SINCE 1-Feb-1994
252 NOT FROM "Smith"
253 S: P282 OK SEARCH completed
254 C: P283 SEARCH CHARSET UTF-8 (OR $ 1,3000:3021) TEXT {8}
255 C: YYYYYYYY
256 S: * SEARCH 882 1102 3003 3005 3006
257 S: P283 OK completed
258
259 Note: Since this document format is restricted to 7-bit ASCII text,
260 it is not possible to show actual UTF-8 data. The "YYYYYYYY" is a
261 placeholder for what would be 8 octets of 8-bit data in an actual
262 transaction.
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282Melnikov Standards Track [Page 5]
283
284RFC 5182 Last SEARCH Result Reference March 2008
285
286
287 4) The following example demonstrates that a failed SEARCH sets the
288 search result variable to the empty list.
289
290 Example 5:
291 C: B282 SEARCH RETURN (SAVE) SINCE 1-Feb-1994
292 NOT FROM "Smith"
293 S: B282 OK SEARCH completed
294 C: B283 SEARCH CHARSET KOI8-R (OR $ 1,3000:3021) TEXT {4}
295 C: XXXX
296 S: B283 NO [BADCHARSET UTF-8] KOI8-R is not supported
297 //After this command the saved result variable contains
298 //no messages. A client that wants to reissue the B283
299 //SEARCH command with another CHARSET would have to reissue
300 //the B282 command as well. One possible workaround for
301 //this is to include the desired CHARSET parameter
302 //in the earliest SEARCH RETURN (SAVE) command in a
303 //sequence of related SEARCH commands.
304 //A better approach might be to always use CHARSET UTF-8
305 //instead.
306
307 Note: Since this document format is restricted to 7-bit ASCII text,
308 it is not possible to show actual KOI8-R data. The "XXXX" is a
309 placeholder for what would be 4 octets of 8-bit data in an actual
310 transaction.
311
312 5) The following example demonstrates that it is not an error to use
313 the "$" marker when it contains no messages.
314
315 Example 6:
316 C: E282 SEARCH RETURN (SAVE) SINCE 28-Oct-2006
317 NOT FROM "Eric"
318 C: E283 COPY $ "Other Messages"
319 //The "$" contains no messages
320 S: E282 OK SEARCH completed
321 S: E283 OK COPY completed, nothing copied
322
3232.3. Multiple Commands in Progress
324
325 Use of a SEARCH RETURN (SAVE) command followed by a command using the
326 "$" marker creates direct dependency between the two commands. As
327 directed by Section 5.5 of [IMAP4], a server MUST execute the two
328 commands in the order they were received. (A server capable of
329 out-of-order execution can in some cases execute the two commands in
330 parallel, for example, if a SEARCH RETURN (SAVE) is followed by
331 "SEARCH $", the search criteria from the first command can be
332 directly substituted into the second command.)
333
334
335
336
337
338Melnikov Standards Track [Page 6]
339
340RFC 5182 Last SEARCH Result Reference March 2008
341
342
343 A client supporting this extension MAY pipeline a SEARCH RETURN
344 (SAVE) command with one or more command using the "$" marker, as long
345 as this doesn't create an ambiguity, as described in Section 5.5 of
346 [IMAP4].
347
348 Example 7:
349 C: F282 SEARCH RETURN (SAVE) KEYWORD $Junk
350 C: F283 COPY $ "Junk"
351 C: F284 STORE $ +FLAGS.Silent (\Deleted)
352 S: F282 OK SEARCH completed
353 S: F283 OK COPY completed
354 S: F284 OK STORE completed
355
356 Example 8:
357 C: G282 SEARCH RETURN (SAVE) KEYWORD $Junk
358 C: G283 SEARCH RETURN (ALL) SINCE 28-Oct-2006
359 FROM "Eric"
360 //The server can execute the two SEARCH commands
361 //in any order, as they don't have any dependency.
362 //Note that the second command is making use of
363 //the [ESEARCH] extension.
364 S: * ESEARCH (TAG "G283") ALL 3:15,27,29:103
365 S: G283 OK SEARCH completed
366 S: G282 OK SEARCH completed
367
368 The following example demonstrates that the result of the second
369 SEARCH always overrides the result of the first.
370
371 Example 9:
372 C: H282 SEARCH RETURN (SAVE) KEYWORD $Junk
373 C: H283 SEARCH RETURN (SAVE) SINCE 28-Oct-2006
374 FROM "Eric"
375 S: H282 OK SEARCH completed
376 S: H283 OK SEARCH completed
377
3782.4. Interaction with ESEARCH Extension
379
380 Servers that implement the extension defined in this document MUST
381 implement [ESEARCH] and conform to additional requirements listed in
382 this section.
383
384 The SAVE result option doesn't change whether the server would return
385 items corresponding to MIN, MAX, ALL, or COUNT [ESEARCH] result
386 options.
387
388
389
390
391
392
393
394Melnikov Standards Track [Page 7]
395
396RFC 5182 Last SEARCH Result Reference March 2008
397
398
399 When the SAVE result option is combined with the MIN or MAX [ESEARCH]
400 result option, and none of the other ESEARCH result options are
401 present, the corresponding MIN/MAX is returned (if the search result
402 is not empty), but the "$" marker would contain a single message as
403 returned in the MIN/MAX return item.
404
405 If the SAVE result option is combined with both MIN and MAX result
406 options, and none of the other ESEARCH result options are present,
407 the "$" marker would contain one or two messages as returned in the
408 MIN/MAX return items.
409
410 If the SAVE result option is combined with the ALL and/or COUNT
411 result option(s), the "$" marker would always contain all messages
412 found by the SEARCH or UID SEARCH command. (Note that the last rule
413 might affect ESEARCH implementations that optimize how the COUNT
414 result is constructed.)
415
416 The following table summarizes the additional requirement on ESEARCH
417 server implementations described in this section.
418
419 +----------------+-------------------+
420 | Combination of | "$" marker value |
421 | Result option | |
422 +----------------+-------------------+
423 | SAVE MIN | MIN |
424 +----------------+-------------------+
425 | SAVE MAX | MAX |
426 +----------------+-------------------+
427 | SAVE MIN MAX | MIN & MAX |
428 +----------------+-------------------+
429 | SAVE * [m] | all found messages|
430 +----------------+-------------------+
431
432 where '*' means "ALL" and/or "COUNT"
433 '[m]' means optional "MIN" and/or "MAX"
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450Melnikov Standards Track [Page 8]
451
452RFC 5182 Last SEARCH Result Reference March 2008
453
454
455 The following example demonstrates behavioral difference for
456 different combinations of ESEARCH result options. Explanatory
457 comments start with // and are not part of the protocol:
458
459 Example 10:
460 C: C282 SEARCH RETURN (ALL) SINCE 12-Feb-2006
461 NOT FROM "Smith"
462 S: * ESEARCH (TAG "C283") ALL 2,10:15,21
463 //$ value hasn't changed
464 S: C282 OK SEARCH completed
465
466 C: C283 SEARCH RETURN (ALL SAVE) SINCE 12-Feb-2006
467 NOT FROM "Smith"
468 S: * ESEARCH (TAG "C283") ALL 2,10:15,21
469 //$ value is 2,10:15,21
470 S: C283 OK SEARCH completed
471
472 C: C284 SEARCH RETURN (SAVE MIN) SINCE 12-Feb-2006
473 NOT FROM "Smith"
474 S: * ESEARCH (TAG "C284") MIN 2
475 //$ value is 2
476 S: C284 OK SEARCH completed
477
478 C: C285 SEARCH RETURN (MAX SAVE MIN) SINCE
479 12-Feb-2006 NOT FROM "Smith"
480 S: * ESEARCH (TAG "C285") MIN 2 MAX 21
481 //$ value is 2,21
482 S: C285 OK SEARCH completed
483
484 C: C286 SEARCH RETURN (MAX SAVE MIN COUNT)
485 SINCE 12-Feb-2006 NOT FROM "Smith"
486 S: * ESEARCH (TAG "C286") MIN 2 MAX 21 COUNT 8
487 //$ value is 2,10:15,21
488 S: C286 OK SEARCH completed
489
490 C: C286 SEARCH RETURN (ALL SAVE MIN) SINCE
491 12-Feb-2006 NOT FROM "Smith"
492 S: * ESEARCH (TAG "C286") MIN 2 ALL 2,10:15,21
493 //$ value is 2,10:15,21
494 S: C286 OK SEARCH completed
495
496
497
498
499
500
501
502
503
504
505
506Melnikov Standards Track [Page 9]
507
508RFC 5182 Last SEARCH Result Reference March 2008
509
510
5112.5. Refusing to Save Search Results
512
513 In some cases, the server MAY refuse to save a SEARCH (SAVE) result,
514 for example, if an internal limit on the number of saved results is
515 reached.
516
517 In this case, the server MUST return a tagged NO response containing
518 the NOTSAVED response code and set the search result variable to the
519 empty sequence, as described in Section 2.1.
520
5213. Formal Syntax
522
523 The following syntax specification uses the Augmented Backus-Naur
524 Form (ABNF) notation as specified in [ABNF]. Non-terminals
525 referenced but not defined below are as defined in [IMAP4] or
526 [IMAPABNF].
527
528 Except as noted otherwise, all alphabetic characters are
529 case-insensitive. The use of upper- or lower-case characters to
530 define token strings is for editorial clarity only. Implementations
531 MUST accept these strings in a case-insensitive fashion.
532
533 capability =/ "SEARCHRES"
534 ;; capability is defined in [IMAP4]
535
536 sequence-set =/ seq-last-command
537 ;; extends sequence-set to allow for
538 ;; "result of the last command" indicator.
539
540 seq-last-command = "$"
541
542 search-return-opt = "SAVE"
543 ;; conforms to generic search-return-opt
544 ;; syntax defined in [IMAPABNF]
545
546 resp-text-code =/ "NOTSAVED"
547 ;; <resp-text-code> from [IMAP4]
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562Melnikov Standards Track [Page 10]
563
564RFC 5182 Last SEARCH Result Reference March 2008
565
566
5674. Security Considerations
568
569 This extension requires the server to keep additional state, that may
570 be used to simplify Denial of Service attacks. In order to minimize
571 damage from such attacks, server implementations MAY limit the number
572 of saved searches they allow across all connections at any given time
573 and return the tagged NO response containing the NOTSAVED response
574 code (see Section 2.5) to a SEARCH RETURN (SAVE) command when this
575 limit is exceeded.
576
577 Apart from that, it is believed that this extension doesn't raise any
578 additional security concerns not already discussed in [IMAP4].
579
5805. IANA Considerations
581
582 This document defines the "SEARCHRES" IMAP capability. IANA has
583 added it to the IMAP4 Capabilities Registry, which is currently
584 located at:
585
586 http://www.iana.org/assignments/imap4-capabilities
587
5886. Acknowledgments
589
590 The author would like to thank Mark Crispin, Cyrus Daboo, and Curtis
591 King for remembering that this document had to be written, as well as
592 for comments and corrections received.
593
594 The author would also like to thank Dave Cridland, Mark Crispin,
595 Chris Newman, Dan Karp, and Spencer Dawkins for comments and
596 corrections received.
597
598 Valuable comments, both in agreement and in dissent, were received
599 from Arnt Gulbrandsen.
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618Melnikov Standards Track [Page 11]
619
620RFC 5182 Last SEARCH Result Reference March 2008
621
622
6237. References
624
6257.1. Normative References
626
627 [KEYWORDS] Bradner, S., "Key words for use in RFCs to Indicate
628 Requirement Levels", BCP 14, RFC 2119, March 1997.
629
630 [ABNF] Crocker, D., Ed., and P. Overell, "Augmented BNF for
631 Syntax Specifications: ABNF", STD 68, RFC 5234, January
632 2008.
633
634 [IMAP4] Crispin, M., "INTERNET MESSAGE ACCESS PROTOCOL - VERSION
635 4rev1", RFC 3501, March 2003.
636
637 [IMAPABNF] Melnikov, A. and C. Daboo, "Collected Extensions to IMAP4
638 ABNF", RFC 4466, April 2006.
639
640 [ESEARCH] Melnikov, A. and D. Cridland, "IMAP4 Extension to SEARCH
641 Command for Controlling What Kind of Information Is
642 Returned", RFC 4731, November 2006.
643
6447.2. Informative References
645
646 [UIDPLUS] Crispin, M., "Internet Message Access Protocol (IMAP) -
647 UIDPLUS extension", RFC 4315, December 2005.
648
649 [SORT] Crispin, M. and K. Murchison, "INTERNET MESSAGE ACCESS
650 PROTOCOL - SORT AND THREAD EXTENSIONS", Work in Progress,
651 Septemeber 2007.
652
653Author's Address
654
655 Alexey Melnikov
656 Isode Ltd.
657 5 Castle Business Village,
658 36 Station Road,
659 Hampton, Middlesex,
660 TW12 2BX, United Kingdom
661
662 EMail: Alexey.Melnikov@isode.com
663
664
665
666
667
668
669
670
671
672
673
674Melnikov Standards Track [Page 12]
675
676RFC 5182 Last SEARCH Result Reference March 2008
677
678
679Full Copyright Statement
680
681 Copyright (C) The IETF Trust (2008).
682
683 This document is subject to the rights, licenses and restrictions
684 contained in BCP 78, and except as set forth therein, the authors
685 retain all their rights.
686
687 This document and the information contained herein are provided on an
688 "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
689 OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND
690 THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS
691 OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
692 THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
693 WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
694
695Intellectual Property
696
697 The IETF takes no position regarding the validity or scope of any
698 Intellectual Property Rights or other rights that might be claimed to
699 pertain to the implementation or use of the technology described in
700 this document or the extent to which any license under such rights
701 might or might not be available; nor does it represent that it has
702 made any independent effort to identify any such rights. Information
703 on the procedures with respect to rights in RFC documents can be
704 found in BCP 78 and BCP 79.
705
706 Copies of IPR disclosures made to the IETF Secretariat and any
707 assurances of licenses to be made available, or the result of an
708 attempt made to obtain a general license or permission for the use of
709 such proprietary rights by implementers or users of this
710 specification can be obtained from the IETF on-line IPR repository at
711 http://www.ietf.org/ipr.
712
713 The IETF invites any interested party to bring to its attention any
714 copyrights, patents or patent applications, or other proprietary
715 rights that may cover technology that may be required to implement
716 this standard. Please address the information to the IETF at
717 ietf-ipr@ietf.org.
718
719
720
721
722
723
724
725
726
727
728
729
730Melnikov Standards Track [Page 13]
731
732