"Fossies" - the Fresh Open Source Software Archive

Member "sanitizer-changelog.html" (20 Jan 2006, 31297 Bytes) of package /linux/privat/old/procmail-sanitizer.tar.gz:

Caution: In this restricted "Fossies" environment the current HTML page may not be correctly presentated and may have some non-functional links. You can here alternatively try to browse the pure source code or just view or download the uninterpreted raw source code. If the rendering is insufficient you may try to find and view the page on the project site itself.


Enhancing E-Mail Security With Procmail

History of changes

Back to the home page

{development, no projected release date - comments solicited} TO DO:
Poison/Strip by MIME type. Repair malformed MIME boundary strings (e.g. begin with "A--" instead of "--"). Add option for inserting a header on problems. Add option for modifying Subject: on problems. Don't add a second end-comment when killing STYLE tags. Option to simplify MIME boundary strings. Allow specification of a program for the quarantine destination. Sanitize improperly-wrapped RFC822 headers. Allow customization of the "DEFANGED" text. Mangle CID (Content-ID:) headers to disable IFRAME and related exploits. Set envelope from header on admin and recipient notices too. Scan mac-office documents (detect MIME type not just filename). Filter out odd characters from MIME boundary strings? Simplify notification options - right now they're complicated and redundant. Log if the sendmail call fails. Default to newer sendmail command line syntax. Option to control MTA exitcode on quarantine failure (SECURITY_SILENT_QUARANTINE_FAILURE). Move HTML defanging into the main perl script. Option to suppress attachment mangling on poisoned messages. Scan tarballs the same as ZIP files. Scan first line of attachments for magic to determine whether to perform detailed (ZIP, JPEG) scanning. SPLIT OUT THE PERL SCRIPT!

01/20/2006 (1.151) Add poisoning by default of MIME attachment type APPLICATION/X-MSDOWNLOAD, disabled with $SECURITY_TRUST_MS_DOWNLOAD.

01/10/2006 (1.150) Fix the unrar executable check so that it does not test for unrar if RAR scanning is already explicitly disabled by $DISABLE_RAR_SCAN. Fix recipient notification so that it will now work on relays - see procmail-on-gateway.txt for details.

12/31/2005 (1.149) Scan RAR files the same as ZIP files; added $DISABLE_RAR_SCAN, $RARRED_WARNING and $RAR_MAGIC_WARNING. Added WMF to the default $MANGLE_EXTENSIONS list, per the serious bug in Windows' WMF library. Added $SECURITY_POISON_WMF, $WMF_WARNING and magic detection of WMF image files; requires /usr/bin/od. Added MIME Content-* headers to the excessive length test list. Added some new ZIP file magic values (Windows native zip client?).

12/19/2004 (1.148) Fix bug related to scanning multiple levels of MIME attachments. Fix an infinite-loop bug on MIME boundary end detection. Clean up temporary files on BASE64-decode failure. Fix multiple-? manipulation in filespecs. Scan multiple image extensions for JPEG BO attack - filename may be inaccurate.

10/02/2004 (1.147) Fix bug in scanning ZIPs and documents whose names are only an extension (e.g. just ".zip"). Add basic JPEG scanning for Windows BO exploit.

09/22/2004 (1.146) Fix minor bug that confused detection of hostile base64 encoding and missing ZIP magic.

09/20/2004 (1.145) Fix bug in null-boundary sanitizing. Fix hang bug in decoding Office/ZIP attachments to attached messages (RFC822 attachments with attachments). Fix bug in scanning non-base64-encoded document/zip attachments. Detect ZIPs by MIME type, not just filename.

07/28/2004 (1.144) Fix subject line on recipient notification if message was discarded (Thanks to Joe Steele). Defang webbugs in table elements. Defang additional HTML tags. Add $SPOOFED_SENDER handling option for reply control. Minor bugfix in ZIP file detection and scanning. Trap poorly-formed BASE64-encoded ZIP attachments (short lines). Fix bug in BASE64-encoded zipfile decoding.

04/10/2004 (1.143) Add a zip "decrypt" pass to de-obfuscate filenames (bugtraq announcement pending). Defang FORM tags (see bugtraq posting http://www.securityfocus.com/archive/1/359139). Defang webbug images in tables.

03/13/2004 (1.142) Bugfix release: zip filename scanner too greedy, corrected.

03/01/2004 (1.141) Add scan of ZIP archive attachment index for suspicious files, quarantine message if found, and options to set ZIP archive policy (ZIPPED_EXECUTABLES, DISABLE_ZIP_SCAN, ZIPPED_WARNING, ZIP_MAGIC_WARNING); this also makes the standard POISONED_EXECUTABLES and STRIPPED_EXECUTABLES lists work for ".zip" attachment filenames. Reduce false-positives in Windows Magic scanner. Partial support for CPAN Perl modules instead of external programs for attachment scanning (USE_CPAN, PVT_CPAN). Moved the Macro Poison warning text out of the script (MACRO_WARNING).

02/11/2004 (1.140) Fix DISCARD and NOTIFY tie-in - can now DISCARD without notifying. Make Smart Sender Notification Suppression a bit smarter - see SECURITY_TRUSTED_MTAS. Clean up .tmp files on mimencode failure.

09/07/2003 (1.139) Sanitize bare CR in message headers (Outlook bug). Sanitize multiple null addresses (sendmail exploit). Improve the UUE exclusion of the HTML defanger. Permit spaces after MIME type in MIME headers. Override csh use, as it is sanitizer-hostile. Add Microsoft Office Suite VBE buffer overflow attacks to macro scanner.

01/26/2003 (1.138) Fix some minor security bugs - thanks to Daniel Riley. Allow tuning of score for MS Office INCLUDETEXT and INCLUDEPICTURE in macro scanner - see $SECURITY_OFFICE_EMBED_SCORE. Fixed a MIME mangling that I missed in 137.

12/22/2002 (1.137) Fix code in filename shortening that breaks pre-5.004_05 perl. Mangle MIME types to APPLICATION/DEFANGED rather than TEXT/PLAIN, as some mailers perform text-related file modifications when saving such an attachment - this corrupts binary files.

10/20/2002 (1.136) Added CPL (Control Panel applet) and WSZ (WinAmp skin - scriptable) to default list of executable extensions. Handle extension-only filenames properly. Don't corrupt HTML-encoded multibyte characters. Collapse runs of spaces in filenames before length-limiting. Don't lose original extension(s) during length-limiting. Add a kill-all-EXEs option ($SECURITY_POISON_WINEXE) to check base64 body for WIN exe magic. Mangle MIME type to TEXT/PLAIN instead of APPLICATION/OCTET-STREAM to (hopefully) prevent magic scanning and execution. Detect MSWord INCLUDETEXT and INCLUDEPICTURE as an attack in macro scanner. Special case for sender detection in messages from AOL. Smarten up sender postmaster notification a bit.

05/26/2002 (1.135) Smarten $SECURITY_NOTIFY_SENDER up to reduce spoofing by forged headers; disable this by setting $SECURITY_DISABLE_SMART_REPLY to any value; side-effect is the sender address is now taken from the Return-Path: header instead of the From: header. Add original message headers to sender notification message. Allow override of FROM address on notifications; set $SECURITY_LOCAL_POSTMASTER to the address to use, e.g. "abuse@myrootdomain.com". Set envelope FROM address so bounced notifications go to admin rather than user; this is done in the default $MTA_FLAGS_HDRS so if you override that you'll want to make sure you use the appropriate flags in your custom command line. Option to notify abuse@ in addition to postmaster@ at sender domain; set $SECURITY_NOTIFY_SENDER_ABUSE to any value to enable. Refine active-HTML defanging a bit in response to a bugtraq post. Improve detection of obscured HTML tags. Option to specify quarantine lockfile; set $SECURITY_QUARANTINE_LOCKFILE to a full path-and-filename writable by all users (e.g. "/var/tmp/quarantine.lock"). Option to log poisoned Message-IDs to a file; set $SECURITY_MSGID_LOG to a full path-and-filename writable by all users (e.g. "/var/tmp/msgid.log"). Properly enquote unquoted attachment filenames that have embedded semicolons. Minor cosmetic changes to log messages. Fix the "Extraneous deliver-head flag ignored" booboo.

04/21/2002 (1.134) Customize the MTA command line, to allow for newer sendmail command line options and non-sendmail MTAs: $MTA_FLAGS_CMDLN and $MTA_FLAGS_HDRS. Mangle MIME types in deferred headers if appropriate. Improve encoded-filename handling. Set Errors-To: header. Put the version number in the $NOTIFY message. Fix no-LOGFILE-breaks-UUE-sanitization bug. Defang quotes-in-extension Outlook attack. Add WMA and WMV to mangled executable extensions, per bugtraq. Fix trailing periods in addition to trailing whitespace - Windows drops trailing periods from filenames without warning. Work around memory allocation error in procmail v3.22. Add the OnContextMenu and OnDragStart events to HTML defanger. Improved recipient address parsing for logs and bounce messages. Minor procmail efficiency enhancements.

01/05/2002 (1.133) Fixed bug in handling of some recursive multipart messages; this has serious security implications, you should upgrade right away. Fixed stripping of attachment-only MIME messages. Added stripping of UUE attachments. Added support for multiline status reports (for example, if multiple file attachments are processed). Made some cosmetic improvements in report messages. Recoded some procmail and perl statements for minor efficiency gains. Now truncate stripped and poisoned filespecs at space to allow for comments in the poisoned- and stripped-filenames lists - if you are poisoning or stripping filespecs containing spaces, MAKE SURE you use \s instead of a literal space!

12/05/2001 (1.132) Fixed a couple of bugs in MIME boundary string parsing and handling; this seriously compromised sanitization of recursive multipart messages. Added detection of UUE attachments to the HTML decoder to avoid corruption of data in UUencoded attachments. Added "DISCARD" response; for now this will only work for local-rules traps. Added "NONOTIFY" handling; for now this will only work for local-rules traps. Changed the URLs to point at the Sanitizer Intro page instead of the home page.

11/22/2001 (1.131) Fixed the script so that it now actually respects the setting of $SECURITY_TRUST_HTML. Added support for the Perl regular expression (?...) construct in the poisoned files list, so that more flexible poisoning lists may be constructed - see man perlex for details. Fixed a bug that caused the sanitizer to misinterpret multi-line RFC822 Content-Type headers, leading to attachments not being sanitized in some cases. Added a hack to recognize filenames in Content-Description comment headers, where MS Outlook helpfully looks for a filename if one isn't specified in the Content-Type or Content-Disposition headers; if you don't want Content-Description to be modified, define $SECURITY_DISABLE_OUTLOOK_HACKS to be any value. Recognize multipart attachment specification where the MIME boundary string is not in quotes. Added $SECURITY_NONOTIFY_LONGSUBJECT to suppress long-subject-header notifications. Remove trailing spaces from attachment filenames. Remove trailing spaces if truncating long attachment filename with embedded spaces. Defer echo of Content-Type and Content-Transfer-Encoding headers to remove the need for default filename generation and to make inserted warnings display properly. Fix misparsing of the attachment following an empty attachment. Cosmetic fix in one warning message. Added $STRIPPED_EXECUTABLES to strip attachments by name in a manner similar to $POISONED_EXECUTABLES - stripping an attachment does not poison the entire message.

09/08/2001 (1.130) Moved the embedded "attachment mangled" and "TNEF stripped" texts to environment variables to improve customizability and reduce the size of the Sanitizer perl script; see $POISONED_WARNING and $TNEF_WARNING. Added $SECURITY_DEFANG_SIGNED to allow defanging of signed messages if you're willing to accept that they will fail the signature check. Added $SECURITY_TRUST_HTML to disable HTML defanging. Moved encoded-character decoding to earlier in the HTML defanging process, so that an obscured tag like "<SCR&#73;PT>" will be properly defanged. Added defanging of the <LINK> tag. Added support for mangling and poisoning files with Microsoft Class-ID extensions. Added a check for "already quarantined", so that if your local-rules script has marked a message to be quarantined the main sanitizer perl script will be skipped - this saves time processing the message. Various changes in coding to reduce the size of the sanitizer Perl script - it should now work successfully on AIX and other OSes with relatively small command-line size limits. Added attempt to defang background images in case they are used as webbugs. Added a version with the macro scanning code removed to save space and time if it's not being used.

04/14/2001 (1.129) Detect and truncate Subject: headers longer then 250 characters, to protect Outlook Express users. Add VCF and NWS to the default MANGLE_EXTENSIONS list. Only defang HTML in message body, to avoid defanging email addresses like <meta.smith@example.org>. Change macro scanner to allow detailed reporting of what it finds; if you add SCORE_DETAILS=YES to your sanitizer configuration, the sanitizer will now tell you why it is considering a document to be poisoned - thanks to Brian D. Hanna for the original version of this. Modified macro score logging to include the recipient name (only meaningful if the sanitizer is running on the same system as the user mailboxes) - thanks to Peter Burkholder for his patch. Changed default filename to "default.txt" to try to force Windows to treat it safely. Fixed the REPORT bug from 1.128. Changed the canned reply text a bit to make it more clear that security policy can involve more than just a virus scanner.

02/24/2001 (1.128) More tweaking of the macro scanner to reduce the likelihood of false positives. Added some modifications to prevent accidental quarantine of a clean message that happens to contain X-Content-Security: text, for example in a forwarded body part.

02/03/2001 (1.127) Added the <LAYER> tag to HTML defanging; this is primarily of interest to people running webmail programs. The sanitizer now recurses into multipart attachments in addition to RFC822 attachments; the only hole left now is defanging BASE64-encoded HTML attachments. If a file attachment does not have a filename specified, a default filename will be provided; this should prevent some social-engineering attacks on Outlook users. Modified the Office Macro scanner a bit; some code used in default-template infector macros was being ignored, and some false positives based on document contents (vs. macro code) were being generated.

01/11/2001 (1.126) Added the <BGSOUND> tag to web-bug defanging. Moved the quarantine and notification routines out of the encrypted-message skip block so that custom rules can still poison encrypted messages. Minor wording change in the MS-TNEF notification text.

12/26/2000 (1.125) Closed the gaping hole: the failure to scan attachments that are themselves RFC822 messages with MIME attachments; the sanitizer will now recurse into attached messages (several layers deep if necessary) and sanitize MIME headers in all. Unfortunately the RFC822 excessively-long-header checks are still only performed on the outermost headers. Added SECURITY_TRUST_STYLE_TAGS as an option. Catch encoded periods in filenames so that they can't be used to prevent filename mangling or poisoning.

12/01/2000 Improved handling of &# and % escapes; they could have been "fixed" within the body of base64 attachments, thus corrupting the attachment. Added ".asx", ".wms", ".wmz" and ".wmd" to the default mangle list; the mangling and poisoning model is going to change very soon. Added stripping of MS-TNEF attachments created by Microsoft Outlook Rich-Text format; to strip MS-TNEF attachments, define SECURITY_STRIP_MSTNEF to be any value. See http://support.microsoft.com/support/kb/articles/Q241/5/38.ASP and http://www.microsoft.com/TechNet/exchange/2505ch10.asp for more information. Rewrote the document macro scanner to be more efficient; now it only makes one pass through the attachment, where before it was making two passes. Changed filename length limit to 128 characters from 64; 64 characters is not enough for proper handling of long filenames with encoded international characters.
Remodeled the website - there's too much there to fit on one intelligible page.

11/11/2000 Improved <STYLE> defanging to keep the style settings from being visible in the message body when viewed in an HTML mail client. Added defanging of MIME values that have been explicitly set to null (e.g. encoding="") - this is a major DoS attack against Microsoft Exchange. Added SECURITY_NOTIFY_RECIPIENT option. Added ".pps" (PowerPoint slide show) to the default mangle list and scanned documents.

10/07/2000 Improved long attachment filename truncation where the attachment filename encoded for international character support. Added SECURITY_NOTIFY_SENDER_POSTMASTER option. Added DEBUG_VERBOSE option; set it to anything to turn verbose debugging back on inside the sanitizer (DROPPRIVS turns VERBOSE off).

09/25/2000 Added ".ocx" to the default mangle list - if you are maintaining custom mangle lists, you should update them. You probably also want to add *.OCX to your poisoned-attachments list. Check for a null MIME boundary string and supply one if necessary; this is a major DoS attack against Microsoft Exchange (see bugtraq). Added support for $DEFANG_WEBBUGS - see above for details.

09/19/2000 Added .DLL, .MDA and .MDW to the default mangle list - if you are maintaining custom mangle lists, you should update them. You probably also want to add *.DLL to your poisoned-attachments list. Modified the macro scanner slightly to reduce the chance of false positives on Excel spreadsheets. Added From:, Status:, X-Status: and X-Keywords: to the excessively-long headers check since UW IMAP is vulnerable to overflows in these. Increased the Excessively Long Header length to 512 characters to further reduce false positives.
The sanitizer home page is moving to http://www.impsec.org/email-tools/procmail-security.html

08/08/2000 D'OH! Left in a debugging trap. Update to 1.117 and prune your logfiles. Sorry.

08/04/2000 Okay, don't trigger the Excessively Long Header trap until the header exceeds 250 characters. Added asd to the default MANGLE_EXTENSIONS. If you are overriding the default list you should add it to your custom list. Fixed a problem where it was possible for the sanitizer to overlook every other attachment in a series of document attachments, or in general any attachment following a document attachment. Added clearing of the MIME content type if the attachment filename gets mangled, to prevent the mail program from figuring out what program to run even though the filename is mangled. For the same reason, drop x-mac-* clauses that Eudora uses to indicate the file type and restore the filename extension.

07/26/2000 Bugfix in NOTIFY SENDER.

07/23/2000 Added checks for certain excessively long standard headers, to address the MS Outlook header buffer-overflow bug; previously only MIME-related headers were length-limited, and only in MIME messages. Disabled sanitizing of encrypted/signed messages; changing the body of such a message breaks the signature, so there's no good way to sanitize it. Moved DROPPRIVS=YES into the sanitizer itself to avoid configuration errors - this may break gateway use, watch it closely. Enabled scanning of PowerPoint files, which weren't being scanned due to an oversight (D'oh!). Improved handling of RFC822 comments embedded in unquoted attachment filenames. Improved handling of filenames containing international characters. Added a debugging mode - if you want to see the poisoned filespecs it is comparing attachment names to, define $DEBUG to be anything. Improved loop-prevention in notification messages; if you want to secure your system against someone forging the X-Loop: headers in an attempt to suppress attack notification messages, define $SECRET to be a short string of random text.
Given the severity of the Outlook BO bug, you probably want to install the updated sanitizer right away.

05/18/2000 (Announcement here delayed, sorry) Okay, it's happened. A working demonstration attack that uses a combination of active-scripted HTML and a scriptable attachment (in the form of a Microsoft Compiled Help file) to automatically save and execute an arbitrary program remotely via email without the user having to double-click on an attachment has been posted to Bugtraq. This means that, for example, someone could email you a copy of Back Orifice that would install itself on your computer the moment you simply previewed the message in your mail client.
Make sure that chm appears in your MANGLE_EXTENSIONS list and that *.chm is in your poisoned executables list. You should also visit this page that describes tightening down Outlook's security settings.

05/22/2000 Added some new executable extensions to MANGLE_EXTENSIONS. See above for the new default. Fixed a bug that prevented macro scanning if document attachments were in MANGLE_EXTENSIONS. Dynamically set LINEBUF so that we're no longer vulnerable to extremely long To: headers.

05/14/2000 Fixed a bug in notification. Added error logging on failure to open poisoned spec file.

05/13/2000 Made sender notification optional. Added ability to specify executable extensions list in configuration file. No more script updates for new executables! Site-customized executable mangling!

05/12/2000 Improved sender notification. Added quarantine reliability assurance (i.e. bounce if quarantine fails).

05/10/2000 Added ".vbe" to the executable extensions list. You should add "*.vbe" to your poisoned executables list. Fixed a problem where a message that was *only* a poisoned executable (e.g. no text body at all) wouldn't be quarantined.

05/06/2000 Added ".wsf" and ".wsh" to the executable extensions list. Fixed another DoS bug in header fixups. Fixed a missing executable extension in the UUE checker. Added notification of the message sender on hits.

03/26/2000 Added ".eml" to the executable extensions list. Dynamic configuration of this soon...

02/01/2000 Improved handling of quotes in tag arguments.

01/22/2000 Sanitizer now deals with attempted obscuration of tag options with &# and % escapes.

01/14/2000 Fixed another DoS bug in certain quoted strings, and generally improved quoted string and wrapped-header handling.

01/09/2000 Added ".chm" to the executable extensions list. Compiled Help files are just as vulnerable as uncompiled Help files. Added defanging of javascript and other scripting languages embedded in IMG SRC= and other tags. This is actively being exploited on Hotmail. Thanks to Georgi Guninski. Fixed another DoS bug in international character set quoted-illegible filenames.

12/11/1999 Added ".hlp" to the executable extensions list. Somehow it's possible to script Microsoft help files.

11/09/1999 Fixed another DoS bug, this one tickled by MIME filenames containing certain Perl regular expression characters - for example, filename="file (1).exe" would cause an infinite loop.
News flash: Microsoft Outlook and Outlook Express are now subject to Active HTML trojan horse attacks. Make sure your email clients have scripting disabled.

10/31/1999 Yet another Microsoft executable filename extension attack. See this advisory for details about .SHS trojans.

10/12/1999 Added Perl .PL and PowerPoint .POT extensions to executable attachments processing. (This will soon be easily site-customizable.)
(oops!) Fixed DoS bug for MIME headers with multiple "name=" clauses. Apparently Z-Mail 3.2.1 is generating somewhat iffy MIME headers...

09/14/1999 Added defanging of <STYLE> tags and clauses because they can be used to hide scripting code.

08/28/1999 Added wildcard support to the poisoned-filename list. The syntax is a cross between shell-filespec syntax and Perl RE syntax. Some examples:


08/18/1999 Added trap for the ODBC remote shell exploit. Note that this may not catch all possible variants of this exploit. See the bugtraq vulnerabilities database for more details. (NB: this also renders the "no signature files" comment below a little less true. Oh, well.)

08/10/1999 Sigh. Yet more Microsoft macro file types that can be exploited via email. Added support for poisoning .HTA and .VBS attachments.

06/12/1999 Improved javascript defanger - events such as OnLoad= are now directly recognized. Defanging them individually is better than tossing <TITLE> and <BODY> tags in toto, and also protects links (which wasn't happening before).

05/24/1999 Added trapping of <TITLE> tags to secure against Netscape's execution of javascript in the wrong security context.

04/02/1999 Scanning of Word documents and Excel worksheets for possibly hostile macros seems to work acceptably well, so I'm going to release it. Note that this does not scan for variant-specific strings, so there aren't any "signature files" involved, but this means that it is possible to get false positives on complex macros. You may want to profile for a while before turning on macro-based attachment poisoning.

03/31/1999 I have added some scanning of Word documents and Excel worksheets for possibly hostile macros. If you'd like to beta test this, please contact me.

03/30/1999 Well, it's just beginning, folks. Somebody's ported Melissa to Excel (search for "Papa Virus" if you want details). I've added "path.xls" to the list of suggested trojans, but that is only a stopgap.

03/29/1999 It is possible for Word infected with a different virus to save .DOC-format files when told to "Save as" .RTF, so it's now possible to specify .RTF extensions in the poisoned list.
Also, fixed a bug in VERBOSE notification. D'oh!

03/27/1999 Thanks to Melissa you can now add .DOC, .DOT, .XLS and .XLW filenames to the poisoned-executables list.

03/20/1999 Another bugtraq announcement of a Eudora buffer-overflow bug in attachment filename handling, affecting versions 4.1 and 4.2 beta. If you're running the sanitizer you should be safe.

03/06/1999 Added the ability to trap on specific executable attachment names and mangle the attachment formatting so that reformatting is necessary to extract the attachment from the message. This was done in response to the Happy99.exe worm, but filenames to trap are specified in a configuration file, making extension of this simple.

02/08/1999 Added sanitization of double backquotes in MIME headers to prevent remote attacks against Metamail via the UW Pine MUA. See the the initial report to bugtraq and the Pine development team's response for more details.

Logs not kept during this period. Various improvements of HTML defanging and development of MIME sanitization, culminating in the initial release of the general-purpose sanitizer in August of 1998.

02/01/1998 Development of the Sanitizer begins, as a simple tool to deactivate active HTML <SCRIPT> tags as part of my general spam-filter kit, hence the name "html-trap.procmail".
Little did I know what I was getting myself into.

I can be contacted at <jhardin@impsec.org> - you could also visit my home page.

Created with vi   Best viewed with Any Browser

$Id: sanitizer-changelog.html,v 1.55 2006-01-20 07:40:08-08 jhardin Exp jhardin $
Contents Copyright (C) 2006 by John D. Hardin - All Rights Reserved.