Courier: libs/maildir/README.imapkeywords.html

"Fossies" - the Fresh Open Source Software Archive

Member "courier-1.2.2/libs/maildir/README.imapkeywords.html" (20 Jan 2022, 7810 Bytes) of package /linux/misc/courier-1.2.2.tar.bz2:

As a special service "Fossies" has tried to format the requested source page into HTML format using (guessed) HTML source code syntax highlighting (style: standard) with prefixed line numbers. Alternatively you can here view or download the uninterpreted source code file.

1 <?xml version="1.0" encoding="iso-8859-1"?> 2 <html xmlns="http://www.w3.org/1999/xhtml"> 3 <head> 4 <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" /> 5 <title>Courier-IMAP: IMAP keywords implementation</title> 6 <meta name="generator" content="amaya 8.6, see http://www.w3.org/Amaya/" /> 7 <style type="text/css"> 8 .filename { font-family: courier new,courier,fixed; font-weight: bold; } 9 .imapflag { font-family: courier new,courier,fixed; }</style> 10 </head> 11 12 <body> 13 <h1>Courier-IMAP: IMAP keywords implementation</h1> 14 15 This white paper describes how Courier-IMAP implements IMAP keywords. This 16 document is provided for informational purposes only. 17 18 <h2>Background</h2> 19 20 Courier-IMAP is a maildir-based IMAP server. The reader is presumed to 21 know how maildirs work. 22 23 There are five pre-defined flags that may be set on each message in an 24 IMAP folder: \Seen, \Answered, \Draft, 26 \Deleted, and \Flagged. An IMAP server may also optionally offer 28 the ability to set arbitrary client-defined flags for any message. 29 30 <h2>Implementation Requirements</h2> 31 <ul> 32 <li>Maintain the high-performance, lock-free nature of maildir-based mail 33 stores.</li> 34 <li>The current version of Courier-IMAP offers an option to use light, 35 dot-lock based locking to minimize undesirable side-effects brought by 36 concurrent folder updates by multiple IMAP clients. Keyword usage should 37 not rely on locking being enabled.</li> 38 <li>Reading and saving keywords should be reasonably fast, even with large 39 folders.</li> 40 <li>Obtaining a list of all keywords set for a given message should be a 41 fast operation.</li> 42 <li>Obtaining a list of all messages with a given keyword set should also 43 be a fast operation.</li> 44 <li>Updating keywords should be a reasonably fast operation.</li> 45 <li>Should not have any noticable overhead unless keywords are actually 46 used.</li> 47 </ul> 48 49 <h2>Implementation Details</h2> 50 51 The rest of this document describes the technical keyword implementation. 52 This is a short summary of the implement issues that should be understood 53 when using IMAP keywords with Courier-IMAP. 54 <ul> 55 <li>On systems that impose a fixed upper limit on the maximum number of 56 files in a directory, the number of messages whose keywords may be 57 adjusted within a 15-20 minute window may not exceed 1/3rd of the upper 58 limit. For example, Linux ext2 filesystem directories can hold about 59 30,000 files, maximum. On Linux systems, no more than 10,000 messages (in 60 the same folder, of course) may have their keywords changed within any 61 15-20 minute window.</li> 62 <li>The atomicity is on a per-message basis. All keywords set for a 63 particular message are saved as an atomic unit. A client adjusts the 64 keywords that are set for a particular message by reading the existing 65 set of keywords, and then replacing them with a new set of keywords. This 66 means that when multiple clients update the keyword set of the same 67 message, the last update wins. Changes made by the losing client are 68 lost. Moral of the story: do not allow multiple clients to mess with the 69 same message, at the same time.</li> 70 </ul> 71 72 <h2>Data storage</h2> 73 74 A new subdirectory, courierimapkeywords, is 75 created in the maildir. It stores keyword-related data. 76 77 The file courierimapkeywords/:list contains 78 a "stable, known list" of all keywords sets for all messages. It is, 79 essentially, a list of the base filenames of each message in the cur directory that has keywords, without the ":2," 81 suffix, and any message flags. Messages without any set keywords are not 82 listed in this file. 83 84 Additional files may also exist in this subdirectory, named either .N.file, or file. file is the base filename of a message, while "N" is 87 a numeric value. 88 89 The list of keywords set for all messages is obtained by reading the 90 contents of courierimapkeywords according to 91 the process described below. 92 93 <h2>Updating keywords</h2> 94 95 A keywords set for a message may be updated as follows: 96 <ul> 97 <li>Create a file in tmp, containing the new keywords that are set for the 98 message. To remove all existing keywords, the file should be empty.</li> 99 <li>Rename the file as courierimapkeywords/file, with file matching the message's base filename.</li> 102 </ul> 103 104 <h2>Reading keywords</h2> 105 106 First, a list of all messages present in new 107 and cur is obtained. Then: 108 <ul> 109 <li>Read courierimapkeywords/:list. Ignore 110 non-existent base filenames read from the :list file.</li> 112 <li>Divide the current time, in seconds, by 300. Call the result T.</li> 113 <li>Read the contents of the courierimapkeywords directory. Ignore :list, the remaining files in the directory will 116 be named either "file", or ".N.file" where 117 N is a number. When encountering a file that cannot be found in the 118 current list of messages present in new and 119 cur, stat the file, and remove it if its 120 ctime is at least fifteen minutes old (prevents removal of keywords for a 121 message that's just been added to the folder, and the scan for messages 122 in new and cur just missed it).</li> 123 <li>When encountering ".N.file" after another 124 ".N.file" was encountered earlier, remove 125 the file with the lesser N, unless the larger of the two Ns is greater 126 than or equals to T. Keep track of the largest N seen that's less than T, 127 and the largest N that's seen that's greater than or equals to T. When 128 encountering a "file", add it a list of all 129 "file"s that were encountered, and process 130 this entry as if it were .X.file, where 131 X=T+1.</li> 132 <li>After reading the entire directory, apply the following changes to the 133 keywords read from the :list file: the 134 contents of every file seen; if file was not seen but a .N.file was seen, then the contents of the file 137 with the largest N. If an attempt to open an update file failed with 138 ENOENT, restart everything from step 1.</li> 139 <li>Write the new set of keywords to a temporary file in tmp, then rename it as courierimapkeywords/:list. Afterwords go through 142 the list of all "file"s that were 143 encountered two steps ago, and rename each "file" to ".X.file". 145 This step should be omitted unless at least one nonexistent file was 146 skipped in the old :list file, or the 147 contents of at least one .N.file was 148 updated to :list.</li> 149 <li>If exactly one .N.file was seen, and 150 N<T, remove the lone .N.file.</li> 151 </ul> 152 </body> 153 </html>