"Fossies" - the Fresh Open Source Software Archive

Member "courier-1.2.2/libs/maildir/README.imapkeywords.html" (20 Jan 2022, 7810 Bytes) of package /linux/misc/courier-1.2.2.tar.bz2:

As a special service "Fossies" has tried to format the requested source page into HTML format using (guessed) HTML source code syntax highlighting (style: standard) with prefixed line numbers. Alternatively you can here view or download the uninterpreted source code file.

    1 <?xml version="1.0" encoding="iso-8859-1"?>
    2 <html xmlns="http://www.w3.org/1999/xhtml">
    3 <head>
    4   <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
    5   <title>Courier-IMAP: IMAP keywords implementation</title>
    6   <meta name="generator" content="amaya 8.6, see http://www.w3.org/Amaya/" />
    7   <style type="text/css">
    8 .filename { font-family: courier new,courier,fixed; font-weight: bold; }
    9 .imapflag { font-family: courier new,courier,fixed; }</style>
   10 </head>
   12 <body>
   13 <h1>Courier-IMAP: IMAP keywords implementation</h1>
   15 <p>This white paper describes how Courier-IMAP implements IMAP keywords. This
   16 document is provided for informational purposes only.</p>
   18 <h2>Background</h2>
   20 <p>Courier-IMAP is a maildir-based IMAP server. The reader is presumed to
   21 know how maildirs work.</p>
   23 <p>There are five pre-defined flags that may be set on each message in an
   24 IMAP folder: <span class="imapflag">\Seen</span>, <span
   25 class="imapflag">\Answered</span>, <span class="imapflag">\Draft</span>,
   26 <span class="imapflag">\Deleted</span>, and <span
   27 class="imapflag">\Flagged</span>. An IMAP server may also optionally offer
   28 the ability to set arbitrary client-defined flags for any message.</p>
   30 <h2>Implementation Requirements</h2>
   31 <ul>
   32   <li>Maintain the high-performance, lock-free nature of maildir-based mail
   33     stores.</li>
   34   <li>The current version of Courier-IMAP offers an option to use light,
   35     dot-lock based locking to minimize undesirable side-effects brought by
   36     concurrent folder updates by multiple IMAP clients. Keyword usage should
   37     not rely on locking being enabled.</li>
   38   <li>Reading and saving keywords should be reasonably fast, even with large
   39     folders.</li>
   40   <li>Obtaining a list of all keywords set for a given message should be a
   41     fast operation.</li>
   42   <li>Obtaining a list of all messages with a given keyword set should also
   43     be a fast operation.</li>
   44   <li>Updating keywords should be a reasonably fast operation.</li>
   45   <li>Should not have any noticable overhead unless keywords are actually
   46     used.</li>
   47 </ul>
   49 <h2>Implementation Details</h2>
   51 <p>The rest of this document describes the technical keyword implementation.
   52 This is a short summary of the implement issues that should be understood
   53 when using IMAP keywords with Courier-IMAP.</p>
   54 <ul>
   55   <li>On systems that impose a fixed upper limit on the maximum number of
   56     files in a directory, the number of messages whose keywords may be
   57     adjusted within a 15-20 minute window may not exceed 1/3rd of the upper
   58     limit. For example, Linux ext2 filesystem directories can hold about
   59     30,000 files, maximum. On Linux systems, no more than 10,000 messages (in
   60     the same folder, of course) may have their keywords changed within any
   61     15-20 minute window.</li>
   62   <li>The atomicity is on a per-message basis. All keywords set for a
   63     particular message are saved as an atomic unit. A client adjusts the
   64     keywords that are set for a particular message by reading the existing
   65     set of keywords, and then replacing them with a new set of keywords. This
   66     means that when multiple clients update the keyword set of the same
   67     message, the last update wins. Changes made by the losing client are
   68     lost. Moral of the story: do not allow multiple clients to mess with the
   69     same message, at the same time.</li>
   70 </ul>
   72 <h2>Data storage</h2>
   74 <p>A new subdirectory, <span class="filename">courierimapkeywords</span>, is
   75 created in the maildir. It stores keyword-related data.</p>
   77 <p>The file <span class="filename">courierimapkeywords/:list</span> contains
   78 a "stable, known list" of all keywords sets for all messages. It is,
   79 essentially, a list of the base filenames of each message in the <span
   80 class="filename">cur</span> directory that has keywords, without the ":2,"
   81 suffix, and any message flags. Messages without any set keywords are not
   82 listed in this file.</p>
   84 <p>Additional files may also exist in this subdirectory, named either <span
   85 class="filename">.N.file</span>, or <span class="filename">file</span>. <span
   86 class="filename">file</span> is the base filename of a message, while "N" is
   87 a numeric value.</p>
   89 <p>The list of keywords set for all messages is obtained by reading the
   90 contents of <span class="filename">courierimapkeywords</span> according to
   91 the process described below.</p>
   93 <h2>Updating keywords</h2>
   95 <p>A keywords set for a message may be updated as follows:</p>
   96 <ul>
   97   <li>Create a file in tmp, containing the new keywords that are set for the
   98     message. To remove all existing keywords, the file should be empty.</li>
   99   <li>Rename the file as <span
  100     class="filename">courierimapkeywords/file</span>, with <span
  101     class="filename">file</span> matching the message's base filename.</li>
  102 </ul>
  104 <h2>Reading keywords</h2>
  106 <p>First, a list of all messages present in <span class="filename">new</span>
  107 and <span class="filename">cur</span> is obtained. Then:</p>
  108 <ul>
  109   <li>Read <span class="filename">courierimapkeywords/:list</span>. Ignore
  110     non-existent base filenames read from the <span
  111     class="filename">:list</span> file.</li>
  112   <li>Divide the current time, in seconds, by 300. Call the result T.</li>
  113   <li>Read the contents of the <span
  114     class="filename">courierimapkeywords</span> directory. Ignore <span
  115     class="filename">:list</span>, the remaining files in the directory will
  116     be named either "<span class="filename">file</span>", or ".N.file" where
  117     N is a number. When encountering a file that cannot be found in the
  118     current list of messages present in <span class="filename">new</span> and
  119     <span class="filename">cur</span>, stat the file, and remove it if its
  120     ctime is at least fifteen minutes old (prevents removal of keywords for a
  121     message that's just been added to the folder, and the scan for messages
  122     in new and cur just missed it).</li>
  123   <li>When encountering "<span class="filename">.N.file</span>" after another
  124     "<span class="filename">.N.file</span>" was encountered earlier, remove
  125     the file with the lesser N, unless the larger of the two Ns is greater
  126     than or equals to T. Keep track of the largest N seen that's less than T,
  127     and the largest N that's seen that's greater than or equals to T. When
  128     encountering a "<span class="filename">file</span>", add it a list of all
  129     "<span class="filename">file</span>"s that were encountered, and process
  130     this entry as if it were <span class="filename">.X.file</span>, where
  131     X=T+1.</li>
  132   <li>After reading the entire directory, apply the following changes to the
  133     keywords read from the <span class="filename">:list</span> file: the
  134     contents of every <span class="filename">file</span> seen; if <span
  135     class="filename">file</span> was not seen but a <span
  136     class="filename">.N.file</span> was seen, then the contents of the file
  137     with the largest N. If an attempt to open an update file failed with
  138     ENOENT, restart everything from step 1.</li>
  139   <li>Write the new set of keywords to a temporary file in <span
  140     class="filename">tmp</span>, then rename it as <span
  141     class="filename">courierimapkeywords/:list</span>. Afterwords go through
  142     the list of all "<span class="filename">file</span>"s that were
  143     encountered two steps ago, and rename each "<span
  144     class="filename">file</span>" to "<span class="filename">.X.file</span>".
  145     This step should be omitted unless at least one nonexistent file was
  146     skipped in the old <span class="filename">:list</span> file, or the
  147     contents of at least one <span class="filename">.N.file</span> was
  148     updated to <span class="filename">:list</span>.</li>
  149   <li>If exactly one <span class="filename">.N.file</span> was seen, and
  150     N&lt;T, remove the lone <span class="filename">.N.file</span>.</li>
  151 </ul>
  152 </body>
  153 </html>