"Fossies" - the Fresh Open Source Software Archive

Member "amavisd-new-2.11.1/README_FILES/README.protocol" (2 Apr 2011, 17635 Bytes) of package /linux/misc/amavisd-new-2.11.1.tar.bz2:


As a special service "Fossies" has tried to format the requested text file into HTML format (style: standard) with prefixed line numbers. Alternatively you can here view or download the uninterpreted source code file.

    1 AMAVIS POLICY DELEGATION PROTOCOL (AM.PDP)
    2 ==========================================
    3   Author: Mark Martinec <Mark.Martinec@ijs.si>
    4   Revisions:
    5     2003-11-10 created
    6     2005-03-18
    7     2005-06-22
    8     2006-08-18 (added attributes: version_server, insheader and quarantine),
    9     2007-05-17 (allow attribute value: request=requeue)
   10     2008-06-21 (new attribute partition_tag=xx on release requests)
   11     2009-11-16
   12     2010-01-22 (new attribute log_id on response)
   13 
   14 NOTE: at the end of this document there is a description
   15 (by Stephane Lentz) of the old protocol, now called AM.CL.
   16 
   17 
   18 Amavis policy delegation protocol (AM.PDP) is intended to replace the
   19 old amavis client protocol (AM.CL) as spoken between amavisd-new helper
   20 programs (amavis.c or amavis-milter.c) and the amavisd daemon. The server
   21 side is implemented by amavisd-new daemon. A sample AM.PDP client in Perl
   22 is helper-progs/amavis.pl, and a rewrite by Petr Rehor of the helper
   23 program amavis-milter.c to use the new AM.PDP protocol is available
   24 as a separate project, see:
   25   http://sourceforge.net/projects/amavisd-milter/
   26 
   27 
   28 The new amavisd client/server protocol is based on the
   29 'Postfix policy delegation protocol', described in the Postfix
   30 document file README_FILES/SMTPD_POLICY_README, which can be used to
   31 delegate policy decisions to an external server that runs outside Postfix.
   32 It is conceptually similar to sendmail/milter mechanism, but based
   33 on a documented text protocol which is easier to test and debug,
   34 instead of using an undocumented binary protocol such as sendmail/milter.
   35 
   36 There are a few departures from the original Postfix policy delegation
   37 protocol:
   38 
   39 - Postfix policy delegation protocol terminates lines with LF,
   40   AM.PDP protocol terminates lines with CR LF;
   41   ( This was changed to facilitate debugging via telnet, and to make it
   42     more similar to related protocols, e.g. SMTP, HTTP, FTP, ... )
   43 
   44 - Postfix policy delegation protocol restricts the reply to one field only,
   45   AM.PDP allows arbitrary number of fields passed from server back to client;
   46   ( This allows additional functionality like modifying or removing
   47     recipient addresses, adding or editing mail header fields, etc. )
   48 
   49 - Postfix policy delegation protocol expects the same attribute name
   50   to be used only once; AM.PDP allows repeated appearances of the attribute,
   51   which makes passing of lists easier (such as multiple recipient addresses);
   52 
   53 - attribute value may consist of more than one field. Fields are delimited
   54   by exactly one non-encoded space. Spaces within a field must be encoded
   55   like any other restricted character (see below).
   56 
   57 The protocol may be spoken over a Unix STREAM socket, or over an
   58 inet tcp socket.
   59 
   60 The client request is a sequence of name=value attributes, each terminated
   61 by CR LF. The sequence is terminated by an empty line (only CR LF).
   62 The server reply has the same structure. After client receives server
   63 response sequence terminated by an empty line, it may close the session,
   64 or issue another request on the same session. During normal operation
   65 the server should not close the socket until it has been closed by the client.
   66 Only under special circumstances is the server allowed to close the session,
   67 e.g. in response to a timeout or fatal error condition.
   68 
   69 The order of attributes does not matter, except for the 'request'
   70 attribute which must appear first in the client request.
   71 The policy client as well as the policy server should ignore
   72 any attributes that it does not care about.
   73 
   74 An attribute name must not contain non-encoded characters: "=", "%",
   75 space, null, CR or LF. An attribute value must not contain
   76 non-encoded characters: "%", space, null, CR, or LF.
   77 
   78 All restricted characters must be hex-encoded to a sequence of three
   79 characters: a percent sign, followed by two hex digits, e.g. %2A.
   80 Hexadecimal digits 0..9,a..f or 0..9,A..F are allowed.
   81 
   82 Any other character MAY be encoded as well. Although not mandated,
   83 it is prudent to encode all non-printable characters.
   84 
   85 The line length is not limited by this protocol. Both the server and the
   86 client must be prepared to handle arbitrary line lengths without breaking
   87 the protocol or rising security issues. Both are allowed to internally
   88 truncate unreasonably long lines to a sensible length, and issue a warning.
   89 
   90 Neither the client nor the server must make any assumptions that certain
   91 characters will not be used in the attribute name or values. E.g. a presence
   92 of encoded null or newline or other special character in the attribute name
   93 or value must be safely and appropriately handled. If such a character
   94 does not comply with the expected syntax, the case should be handled
   95 to the best of client or server understanding and capability,
   96 e.g. character ignored, attribute ignored, and/or a warning logged.
   97 
   98 The following example Perl expression may be used for encoding/decoding:
   99 
  100   to decode attribute name and attribute values:
  101     s/%([0-9a-fA-F]{2})/pack("C",hex($1))/eg
  102 
  103   to encode attribute name:
  104     s/([^0-9a-zA-Z_-])/sprintf("%%%02x",ord($1))/eg;
  105 
  106   to encode attribute values (each space-separated field individually):
  107     s/([^\041-\044\046-\176])/sprintf("%%%02x",ord($1))/eg;
  108 
  109 
  110 Attributes in the client request are:
  111 -------------------------------------
  112 
  113 request=AM.PDP
  114   is a required first attribute, its value must be AM.PDP
  115 
  116 sender=<foo@example.com>
  117   specifies the envelope sender address (reverse-path).
  118   The attribute should appear exactly once. The attribute value syntax
  119   is specified in rfc5321 as 'Reverse-path' (i.e. smtp-quoted form,
  120   enclosed in <>);  a null reverse path is specified as <>.
  121 
  122 recipient=<user1@example.net>
  123   specifies the envelope recipient address. The attribute appears once
  124   for each recipient address, the order of addresses must be preserved
  125   and might be significant for some setups or functions.
  126   The attribute value syntax is specified in rfc5321 as 'Forward-path'.
  127 
  128 tempdir=/var/amavis/amavis-milter-MWZmu9Di
  129   Specifies a temporary work directory to be used for mail unpacking,
  130   typically also containing the original mail file - see attribute
  131   'mail_file' below. This attribute should be present exactly once.
  132   The server is allowed to use the specified directory to create additional
  133   temporary files if it chooses so. As a security precaution, currently
  134   amavisd restricts the temporary directory path, which must be a
  135   subdirectory under $TEMPBASE or $MYHOME.
  136 
  137 tempdir_removed_by=client
  138   Specifies the client will be responsible for removing the temporary
  139   directory. The server must not remove the file email.txt nor the directory,
  140   but it may remove temporary files and subdirectories it has created.
  141 
  142 tempdir_removed_by=server
  143   Specifies the server is responsible to remove the temporary directory
  144   if/when it deems appropriate. This is a default in the absence of this
  145   attribute (for compatibility with traditional amavis clients).
  146 
  147 mail_file=/var/amavis/amavis-milter-MWZmu9Di/email.txt
  148   Specifies a file name (full file path) of a file containing the original
  149   mail with header and body. This attribute should be present at most once.
  150   In its absence the file name defaults to <tempdir>/email.txt.
  151 
  152 delivery_care_of=client
  153   Specifies that server should NOT actively forward the mail to recipients,
  154   but should only report its opinion in its reply, and let the client
  155   act on it. This is a default in the absence of this attribute.
  156 
  157 delivery_care_of=server
  158   Specifies that server is responsible to actively forward the mail
  159   to recipients if it deems the message appropriate for forwarding.
  160   This attribute value indicates that the client has no capability
  161   or intention to forward mail by itself.
  162 
  163 queue_id=qid
  164   optional informational attribute: MTA queue id;
  165 
  166 protocol_name=ESMTP
  167   optional informational attribute: the name of the protocol used by the
  168   original SMTP client to deliver the mail to the client (or to its
  169   associated MTA). Common values are ESMTP, SMTP, LMTP.
  170 
  171 helo_name=b.example.com
  172   optional informational attribute: the value of the HELO or EHLO or LHLO
  173   command specified to our MTA by the original SMTP client;
  174 
  175 client_address=10.2.3.4
  176   optional informational attribute: the IP address of the original SMTP
  177   client;
  178 
  179 client_name=mail.example.com
  180   optional informational attribute: the DNS name of the original SMTP client
  181   as obtained by DNS reverse mapping of the original SMTP client;
  182 
  183 policy_bank=TLS,ORIGINATING,MYNETS
  184   value is a comma-separated list of policy bank names. Names of
  185   nonexistent banks are silently ignored, so are leading and trailing spaces
  186   and TABs around each name. The order of policy bank loading generally
  187   follows the order in which information about a message were obtained:
  188     - interface or socket -based policy banks (when MTA connects to amavisd);
  189     - MYNETS (when client's IP address becomes known);
  190     - the list as specified in the policy_bank attribute of AM.PDP;
  191     - MYUSERS (when sender e-mail address becomes known);
  192 
  193 Attributes in the server response are:
  194 --------------------------------------
  195 
  196 The current set of attributes maps almost exactly to the capabilities
  197 of libmilter, which should facilitate initial implementation.
  198 See sendmail libmilter documentation as well.
  199 
  200 version_server=2
  201   Since amavisd-new-2.4.3 the 'version_server=2' is inserted, older
  202   versions did not provide this attribute (assumed version was 1).
  203   With version 2 the following changes to the protocol were made:
  204   - version_server=2 is provided by the server as the first attribute;
  205   - delheader and chgheader now stand in a response before insheader
  206     and addheader, assuming that milter MTA will execute these in
  207     the same order;
  208   - new attribute insheader, see below;
  209   - new attribute quarantine, see below.
  210 
  211 log_id=xxx
  212   Provided since amavisd-new-2.7.0, tells a log_id under which the
  213   amavisd daemon logged the request and related debugging events;
  214 
  215 delrcpt=<user1@example.net>
  216   The specified recipient should be removed from the list of
  217   recipients of this mail. The specified address must exactly match
  218   the recipient addresses as specified in the client request,
  219   i.e. a smtp-quoted form enclosed in <> should be specified;
  220 
  221 addrcpt=<user1@example.net>
  222   The specified recipient should be added to the list of recipients
  223   of this mail. Paired with 'delrcpt' the pair indicates an existing
  224   recipient address to be replaced by a modified address, e.g. to
  225   add an address extension.
  226 
  227 delheader=index hdr_head
  228   Specifies an existing mail header field should be removed.
  229   Index is a decimal integer indicating which of the header fields
  230   with the same head should be affected, count starts with 1.
  231   'hdr_head' does not include a colon!
  232 
  233 chgheader=index hdr_head hdr_body
  234   Specifies an existing mail header field should have its body replaced by
  235   a new hdr_body. Index is a decimal integer indicating which of the header
  236   fields with the same head should be affected, count starts with 1.
  237   'hdr_head' does not include a colon!
  238 
  239 insheader=index hdr_head hdr_body
  240   Similar to addheader, but specifies a mail header field to be inserted to
  241   the mail header at a given position, index 0 implies top of the header.
  242   Amavisd-new passes a value of $prepend_header_fields_hdridx as the index
  243   argument, which is configurable and defaults to 1 for compatibility with
  244   dkim-milter and dk-milter signing milters. Alternative useful value is 0,
  245   which may be used in absence of other milters inserting their header fields
  246   at index 1. Header fields to be prepended are listed in reverse order,
  247   the last one listed in AM.PDP is to appear at the top of a resulting mail
  248   header. Inserting a header field at an arbitrary position is a later
  249   addition to sendmail milter protocol
  250   (introduced with sendmail 8.13.0 2004-06-20)
  251 
  252 addheader=hdr_head hdr_body
  253   Specifies a mail header field to be appended to the mail header.
  254   Note the use of exactly two value fields, separated by exactly one space.
  255   As described above, spaces in each field must be hex-encoded.
  256   'hdr_head' does not include the colon!
  257 
  258 replacebody=new_body
  259   Not implemented - for future consideration.
  260 
  261 quarantine=reason
  262   place message on hold or to a quarantine maintained by MTA, and supply
  263   a reason text (e.g. client may call smfi_quarantine milter routine;
  264   btw, smfi_quarantine was introduced with sendmail 8.13);
  265   For future use - it is currently (2.4.3 or earlier) never used.
  266 
  267 return_value=val
  268   where val can be any of: continue, accept, reject, discard, tempfail
  269   This attribute should be present exactly once, and indicates to the
  270   client what should be done with the mail and how the original SMTP
  271   session should be treated.
  272 
  273 setreply=rcode xcode message
  274   SMTP response code and text suggested by server to the client
  275   to be used in response to the original SMTP client;
  276   There are exactly three value fields, separated by a single space.
  277 
  278 exit_code=n
  279   Similar in semantics to return_value attribute, but uses
  280   sysexits.h -based exit values for the purpose. Useful with old
  281   amavis clients, but should be ignored by new clients,
  282   which should base its decision on return_value instead.
  283 
  284 
  285 
  286 An example
  287 ==========
  288 
  289 Indentation and text in parenthesis in the following example
  290 is not part of the actual session.
  291 
  292 
  293 $ telnet 127.0.0.1 9998
  294   Trying 127.0.0.1...
  295   Connected to localhost.example.com.
  296   Escape character is '^]'.
  297 
  298 (client->server request)
  299   request=AM.PDP
  300   sender=me@example.com
  301   recipient=user1@example.net
  302   recipient=user2@example.net
  303   protocol_name=ESMTP
  304   client_address=10.2.3.4
  305   tempdir=/var/amavis/amavis-milter-MWZmu9Di
  306 
  307 (server->client response)
  308   setreply=250 2.5.0 Ok,%20id=MWZmu9Di,%20continue%20delivery
  309   return_value=continue
  310   exit_code=0
  311 
  312 (or:)
  313   setreply=250 2.7.1 Ok,%20discarded,%20UBE,%20id=mYOljdn2
  314   return_value=discard
  315   exit_code=99
  316 
  317 (or:)
  318   setreply=550 5.7.1 Message%20content%20rejected,%20UBE,%20id=S7uS4qvA
  319   return_value=reject
  320   exit_code=69
  321 
  322 (or:)
  323   setreply=451 4.5.0 Error%20in%20processing,%20id=...
  324   return_value=tempfail
  325   exit_code=75
  326 
  327 
  328 
  329 ===============================================================================
  330 Releasing a message from a quarantine:
  331 
  332 request=release     (or request=requeue, available since 2.5.1)
  333 mail_id=xxxxxxxxxxxx
  334 secret_id=xxxxxxxxxxxx         (authorizes a release)
  335 partition_tag=xx               (optional, but recommended if info is available)
  336 quar_type=x                    F/Z/B/Q/M (file/zipfile/bsmtp/sql/mailbox)
  337 mail_file=...       (optional: overrides automatics; $QUARANTINEDIR prepended)
  338 requested_by=<releaser@example.com> (optional: lands in Resent-From:)
  339 sender=<foo@example.com>            (optional: replaces envelope sender)
  340 recipient=<bar1@example.net>        (optional: replaces envelope recips)
  341 recipient=<bar2@example.net>
  342 recipient=...
  343 
  344 In reply, for each recipient a SMTP status response is returned
  345 (similar to LMTP)
  346 
  347 quar_type defaults to Q if spam_quarantine_method is sql:, otherwise to F.
  348 
  349 ===============================================================================
  350 AMAVIS SIMPLE CLIENT/SERVER PROTOCOL (traditional)
  351   description by Stephane Lentz
  352   2003-11-06
  353 
  354 amavisd is the daemon part of AMAVIS in charge of scanning SMTP messages.
  355 It receives messages from other applications (clients) using either
  356 SMTP or a simple protocol which is detailed in this document.
  357 The protocol being used depends on the MTA and architecture chosen.
  358 The "simple protocol" is most often used with sendmail in a MILTER
  359 set-up (the client program in such a case is amavis-client).
  360 
  361 AMAVISD receives messages from clients through a UNIX socket :
  362 The UNIX socket used is by default /var/amavis/amavisd.sock .
  363 It is defined :
  364 - at the amavisd server level in amavisd.conf as $unix_socketname
  365 - at the client level when using MILTER (amavis-milter available in
  366 helper-progs) as a configure option :  --with-sockname swith
  367 
  368 The protocol used between the client and server is simple (basic & limited).
  369 There is no possibility for the server to ask the client to remove/add/change
  370 headers. The server can only say if the message was detected as CLEAN, as
  371 UNSAFE and to be rejected/discarded) or not analysed successfully due to
  372 some errors.
  373 
  374 PROTOCOL IN DETAILS :
  375 
  376 The client connects to the AMAVISD server's socket.
  377 IF successful then for each incoming message :
  378 - the client computes and create a new temporary directory ($tempdir)
  379   to store the new incoming message.
  380 - the incoming message is stored as $tempdir/email.txt
  381 - the client sends the directory name to the SERVER
  382 - the server sends \1 to the client if the directory is ok
  383 - the client sends the envelope sender recipient address
  384 - the server sends \1 to the client if ok
  385 - the client sends the envelope recipient addresses one by one to the server:
  386     the client trims the address if it is longer than the
  387       maximum length possible
  388     the client sends this address to the server
  389     the server sends \1 to the client if ok
  390 - the client sends some request to analyze the message to the SERVER.
  391     The character used is  EOT (end of transmission) :\3
  392 - the server processes the mail stored in the directory ($tempdir/email.txt)
  393 - the server sends a STATUS number to the CLIENT. This number returned is
  394   either :
  395     EX_OK (2)           : message CLEAN
  396     EX_UNAVAILABLE (69) : message UNSAFE to be rejected at the SMTP LEVEL
  397                           (550 reject)
  398     99                  : message UNSAFE to be silently (250 code) discarded
  399                           at the SMTP LEVEL
  400     EX_TEMPFAIL (75)    : message not processed successfully (error in
  401                           communication, or server error, ...)