ASSP change log

rebuildspamdb.pl  2.5.1
- email notify on completion
- Griplist upload and download will now be done in rebuildspamdb.
- Droplist download 
- Griplist scripts and data are now on SourceForge.
- Will not delete files at cleanup, but defer deletion for 30 days to give resend requests a chance.
- Will delete files in "discarded folder" older than 30 days
- Will delete files in error folders according to maxerrordays.


List of the perl modules to be installed :

New in 1.5.1:
IO::Socket::SSL - native SSL support
IO::Socket::INET6  - 
Email::Send - resend mail & block report

New in 1.4.3:
Email::MIME::Modifier version 1.442  - attachment detection 
Mail::SPF  version 2.005
Net::CIDR::Lite version 0.20  - hyphenated IP address range 
Net::IP::Match::Regexp version 1.01  - CIDR notation for IP range 
Net::SenderBase version 1.01  - countrycode checks 

Before:
Net::LDAP version 0.33 
Net::DNS version 0.63 
Email::Valid version 0.179 
Mail::SPF::Query version 1.999001 
Mail::SPF  version 2.005
Mail::SRS version 0.31  - Sender Rewriting Scheme 
Compress::Zlib version 2.015  - HTTP compression 
Digest::MD5 version 2.36  - delaying can use MD5 keys for hashes
Digest::SHA1 version 2.11  
File::ReadBackwards version 1.04  - searching of log files enabled
Time::HiRes version 1.9715  - CPU usage statistics 
Sys::Syslog version 0.27  - Unix centralized logging possible
Net::Syslog version 0.03  - network Syslog logging possible
Tie::RDBM  - mysql usage 
Net::IP::Match::Regexp version 1.01  - CIDR notation for IP range 
Net::CIDR::Lite version 0.20  - hyphenated IP address range 
Net::SenderBase version 1.01  - countrycode checks 
LWP::Simple version 1.41  - griplist 
Email::MIME::Modifier version 1.442  - attachment detection 
Net::SMTP version 2.31  - Verify Recipients 

1.5.1.2

section rebuildspamdb:
Notification Email To (RebuildNotify)
Email address(es) to which you want ASSP to send a notification email after the rebuild task is finished. The file rebuildrun.txt is included in this notification. Separate multiple entries by "|".

section logging:
Notification Email To (Notify)
Email address(es) to which you want ASSP to send a notification email, if a matching log entry ( NotifyRe , NoNotifyRe ) is found. Separate multiple entries by "|".

Do Notify, if log entry matches* (NotifyRe)
Regular Expression to identify loglines for which a notification message should be send.
usefull entries are:
adminupdate: - for config changes
admininfo: - for admin informations
option list file: - for option file reload
error: - for any error
restart - to detect a ASSP restart
Admin connection - for GUI logon

Do NOT Notify, if log entry matches* ( NoNotifyRe)
Regular Expression to identify loglines for which no notification message should be send.


Fields marked with an additional asterisk (**) accept a second weight value separated by => from the regular expression. For example: spammer=>1.45 . The multiplication result of the weight and the penaltybox valence value will be used for scoring.

Which Link Should be included (BlockResendLink)
If HTML is enabled in inclResendLink, two links (one on the left and one on the right site) will be included in the report email by default. Depending on the used email clients it could be possible, that one of the two links will not work for you. Try out what link is working and disable the other one, if you want.

My Helo (myHelo)
How ASSP will identify itself when connecting to the target MTA. 
transparent - the Helo of the sender will be used 
use myName - use myName
use FQDN - fully qualified domain name of the host assp is running on

Use File System Virus Scanner (DoFileScan)
If activated, the message is written to a file inside the 'FileScanDir' with an extension of 'maillogExt'. After that ASSP will call 'FileScanCMD' to detect if the temporary file is infected or not. The temporary created file(s) will be removed.
The viruses will be stored in a special folder if the SpamVirusLog is set to 'quarantine' and the filepath to the viruslog is set.
 
File Scan Directory (FileScanDir)
Define the full path to the directory where the messages are temporary stored for the file system virus scanner. This could be any directory inside your file system. The running ASSP process must have full permission to this directory and the files inside!
 
File Scan Command (FileScanCMD) 
ASSP will call this system command and expects a returned string from this command. This returned string is checked against 'FileScanBad' and/or 'FileScanGood' to detect if the message is OK or not! If the file does not exists after the command call, the message is consider infected. ASSP expects, that the file scan is finished when the command returns!
The literal 'FILENAME' will be replaced by the full qualified file name of the temporary file.

The literal 'FILESCANDIR' will be replaced with the value of FileScanDir.
All outputs of this command to STDERR are automatic redirected to STDOUT.
FileScan will not run, if FileScanCMD is not specified.
If you have your online/autoprotect file scanner configured to delete infected files inside the 'FileScanDir', define 'NORUN' in this field! In this case FileScanGood and FileScanBad are ignored. If there is a need to wait some time for the autoprotect scanner, write 'NORUN-dddd', where dddd are the milliseconds to wait!
Depending on your operating system it may possible that you have to quote (' or ") the command, if it contains whitespaces. The replaced file name will be quoted by ASSP if needed.

Native SSL support added!
(new module necessary: IO::Socket::SSL)

- manage & resend spam & notspam from maillog tail
- user blocking report

Request Block Report (EmailBlockReport)
Any mail sent by local/authenticated users to this username will be interpreted
as a request to get a report about blocked emails. Leading digits/numbers in the
mail subject will be interpreted as "report request for the last number of
days". If the number of days is not specified in the mail subject, a default of
5 days will be used to build the report. Only the users defined in EmailBlockTo,
EmailAdmins and EmailAdminReportsTo are able to define a list of email addresses
in the mail body. If such an Admin wants to request a report like it is done
using the BlockReportFile, '=>' must be used in any of the request lines (body)
- please read the section BlockReportFile for more details and syntax.

Queue User Block Report Requests (QueueUserBlockReports)
How to process block report requests for users (not EmailBlockTo, EmailAdmins,
EmailAdminReportsTo).
'run immediately' - the request will be processed immediately (not stored).
'store and run once at midnight' - the request will be stored/queued, runs at
midnight, and will be removed from queue after that
'store and run scheduled' - the request will be stored/queued, runs permanently
scheduled at midnight until it will be removed from queue - a '+' in the subject
is not needed
To add a request to queue the user has to send an email to EmailBlockReport.
Leading digits/numbers in the mail subject will be interpreted as "report
request for the last number of days". If the number of days is not specified in
the mail subject, a default of 5 days will be used to build the report.
If 'run immediately' or 'store and run once at midnight' is selected, but a user
wants to schedule a permanent request, a leading '+' before the digits in
subject is required.
To remove a request from queue the user has to send an email to EmailBlockReport
with a leading '-' in the subject.


File for Blockreportrequest (BlockReportFile)
A file with BlockReport requests. ASSP will generate a block report for every
line in this file (file:files/blockreportlist.txt - file: is required if
defined!) every day at midnight for the last day. The perl modules Net::SMTP and
Email::MIME::Modifier are required to use this feature. A report will be only
created, if there is at least one blocked email found! The syntax is: 
QueryAddress=>ReportRecipient=>ReportDays - there are many possible combinations
of this three parameters. For example:
user@domain and user@domain=>user@domain - will send a report for this user to
this user
*@domain (better use) *@domain=>* - will send a report for every blocked user in
this domain to this user
user@domain=>recipient@any-domain - will send a report for user@domain to
recipient@any-domain
*@domain=>recipient@any-domain - will send a report for every blocked user in
this domain to recipient@any-domain
A third parameter is possible to set, which defines the number of days for which
the report should be created. The default (if empty or not defined) is one day.
This value is used to calculate the 'next run date'. For example:
*@domain=>recipient@any-domain=>2 - creates a report for two days.
*@domain=>*=>14 - creates a report for 14 days.
user@domain=>=>3 or user@domain=>*=>3 - creates a report for three days. The
second parameter is here empty or *!


- user can add/remove himself to redlist, spamlover, noprocessing via
email-interface
- admin(s) can add/remove any address to redlist, spamlover, noprocessing via
email-interface

-DNSBL providers can get a "weight" like bl.spamcop.net=>1.
The value of the weight can be set directly like=>45 or as a divisor of
RBLmaxweight. Low numbers < 6 are divisors . So if RBLmaxweight = 50 (default)
bl.spamcop.net=>50  would be the same as bl.spamcop.net=>1, bl.spamcop.net=>2
would be the same as bl.spamcop.net=>25. 
If the sum of weights surpasses RBLmaxweight, the DNSBL check fails.  If not,
the DNSBL check is scored as "neutral" even with RBLmaxhits reached.

It is possible to use all hits regardless of maxhits.-> Showmaxreplies

For example:
RBLmaxhits=2
RBLmaxweight=50

zen.spamhaus.org=>1
bl.spamcop.net=>1
safe.dnsbl.sorbs.net=>1
combined.njabl.org=>1
dnsbl-1.uceprotect.net=>2
dnsbl-2.uceprotect.net=>2
dnsbl-3.uceprotect.net=>3
ix.dnsbl.manitu.net=>2
psbl.surriel.com=>2
2.apews.org=>3
blackholes.five-ten-sg.com=>10

A "fail" will result of:
2 hits in group 1
1 hit in group 1 and 1 hit in group 2
2 hits in group 2
1 hit in group 1




1.4.3.1 


added in Menu
sorted alphabetical index of GUI fields

added in section LDAP
LDAP Cache

added in section Recipients
VRFY recipients in multiple MTAs

added in section Email-Interface
multiple attached mails in email-interface in spam/ham reports possible
all (not only the first) attachments within MaxBytes/ClamAVBytes will be
detected and processed!

new section Backscatter Detection 
DNS-BackScatter Detection implemented

added in section Penalty Box
Use Invalid Addresses as Traps

added in section Relaying
Support for IP based checks when ASSP is not in front

added in section CC Messages
Do Not Copy Ham Filter* (ccnHamFilter)
Do Not Copy Ham to these addresses. Accepts specific addresses
(user@domain.com), user parts (user) or entire local domains (@domain.com).
Wildcards are supported (fribo*@domain.com).

added in section Whitelisting
Whitelist all RWL Validated Addresses (RWLwhitelisting)
If set, the message will pass also Bayesian Filter and URIBL.

added in section Server Setup
Use OpenDNS NameServers (UseOpenDNS)
 http://www.opendns.com/
If the option is enabled (default) local DNS servers are not used and OpenDNS 
servers are instead used for URIBL, DNSBL etc.. lookups

added in section Sender Validation
Enforce Early Helo Checks (ForceValidateHelo)
If set, ASSP will Validate/Invalidate Format of HELO before DELAYING.
Collecting, Testmode, CopySpam, Spam-Lover is ignored.
Enforce Early Check of Remote Sender with Local Domain Address
(ForceNoValidLocalSender)
If set, ASSP will check Remote Sender with Local Domain Address before DELAYING.
Collecting, Testmode, CopySpam, Spam-Lover is ignored.
Enforce Early Check of Forged Helos (ForceFakedLocalHelo)
If set, ASSP will check Forged Helos before DELAYING. Collecting, Testmode,
CopySpam, Spam-Lover is ignored.

added in section Spam Control
Send 250 OK (send250OK)
Set this checkbox if you want ASSP to reply with '250 OK' instead of SMTP error
code '554 5.7.1'.

added in section Whitelist
Wildcard User for White Domain (wildcardUser)
If you add this user via email-interface(eg: _ALL_@domain.com), the whole domain
will be whitelisted. For example: _ALL_


added in section PenaltyBox
Do Blocking IP's (DoBlockingIP)
If activated, the IP is checked against Block these IP's.
Block these IP's* (denySMTPConnectionsFrom)
Manually maintained list of IP's which should be blocked. IP's in NoPB, noDelay,
acceptAllMail, ispip, whiteListedIPs, noProcessingIPs, whitebox will pass. For
example: file:files/denysmtp.txt. IP ranges are defined for example 182.82.10.
If Net::IP::Match::Regexp is installed CIDR notation is allowed(182.82.10.0/24).
If Net::CIDR::Lite is installed, hyphenated/spaced ranges are allowed
(182.82.10.0-182.82.10.255,182.82.10.0 182.82.10.255). Text without a numbersign
will be a comment to be shown in a match (182.82.10.0/24 AOL).

added in section SMTP Session Limits
Do Deny SMTP Connections from IP's (Strictly) (DoDenyConnect)
If activated, the IP is checked against Deny SMTP Connections from these IP's
Strict.
Deny SMTP Connections from these IP's Strictly* (denySMTPConnectionsFromAlways)
Manually maintained list of IP's which should strictly be denied SMTP access.
Connection will be denied right away, before the body and header is downloaded.

added in section Regex Filters / Spambomb
Use Black Regular Expression to Identify Spam Strictly (DoBlackRe)
Each message is checked against the BlackRe to identify Spams. Technically the
BlackRe Check is part of the Bayesian Check. However it can be separately
activated.
BlackRe - Regular Expression to Identify Spam Strictly* (blackRe)
If an incoming email matches this Perl regular expression it will be considered
spam . For example: penis|virgin|X-Priority


added in section Collection
Use Collect Addresses for Testing Your Environment (DoNotBlockCollect)
ASSP will not block messages from Collect Addresses just because they are
Collect Addresses but handle them normally. That may help you to test and
control your environment.
Do Not Collect Messages from/to these Addresses* (noCollecting)
Accepts specific addresses (user@domain.com), user parts (user) or entire local
domains (@domain.com).


added in section My Server Setup
Web Statitics Port (webStatPort)
The port on which ASSP will listen for http connections to the statistics
interface. You may also supply an IP address to limit connections to a specific
interface.
Examples: 55553, 192.168.0.5:12345

added in PB section
Do Export Penalty BlackBox Extreme (DoExtremeExport)
Exported BlackBox Extreme File (exportExtremeBlack)
IP's in Penalty BlackBox which surpassed the extreme level will be regularly
stored into this file.
Use Exported Penalty BlackBox Extreme for SMTP Denying (exportExtremeFileDeny)
Deny SMTP connections from IP's in Exported Penalty Black Box Extreme File in a
very early stage. This reduces the load on your MTA.
Exported Penalty BlackBox Interval (exportInterval)
Exported Penalty Black Box Extreme File every this hours.
Defaults to 6 hours.

added in section Sender Validation
Do Country Code Lookup (DoSenderBase)

added in several fields
address-lists and wildcards
They accept  specific addresses (user@domain.com), user parts (user) or entire
local domains (@domain.com). Wildcards are supported (fribo*@domain.com).

added in Mail Analyzer
You may put here helo=aaa.bbb.helo or ip=123.123.123.123 to look up the helo/ip
information. Putting a textstring only in will start a lookup in the regular
expression files for the matching regex.



added in SPF section
Override Domains*
Set override to define SPF records for domains that do publish but which you
want to override anyway. Wildcards are supported. For example: abc.com|*.def.com
Internal Name: SPFoverride
Fallback Domains*
Set fallback to define "pretend" SPF records for domains that don't publish them
yet. Wildcards are supported. For example: abc.com|*.def.com
Internal Name: SPFfallback
Local SPF Record
Used in Fallback/Override Domains
The default is v=spf1 a/24 mx/24 ptr -all
Internal Name: SPFlocalRecord


added in Copy Spam & Ham section
Do Not Copy Spam Regex*
Never Copy Spam regardless of collection mode. Put anything here to identify
messages which should not be copied.

added in Greylisting/Delaying section:
Use MD5 for DelayDB
Message-Digest algorithm 5 is a cryptographic hash function and adds some level
of security to the delay database. Must be set to off if you want to list the
database with DelayShowDB/DelayShowDBwhite.
Internal Name: DelayMD5

Show Delay/Greylisting Database
The directory/file with the delay database file. If you change the filename in
section Filepath you must change it here too.
Internal Name: DelayShowDB

Show Delay/Greylisting Safe Database
The directory/file with the safe delay database file. If you change the filename
in section Filepath you must change it here too.
Internal Name: DelayShowDBwhite

added in PenaltyBox section:
Force Extreme Denying for Mode 2
PBextreme will deny connections from IP's whose score meet or exceed the extreme
level/extreme counter - even if PB is only monitoring (mode 2)
Extreme Bad IP History
* Message scoring only,
Internal Name: pbeValencePB
Bad IP History
* Message scoring only
Internal Name: pbValencePB


added in CC Mail section:
Do Not Copy Messages Above This MessageTotal
Messages whose score exceeds this threshold will not be copied. For example: 75


CIDR and Hyphenated IP Range Notation added in IP notation
(In Hyphenated IP Range you may replace the hyphen with a space,
 123.123.123.123 123.123.123.123 is also valid.)
You can freely mix all notations:
123.123.
123.123.0.0/17
123.123.123.123 123.123.123.123
123.123.123.123-123.123.123.123

You can add comments to be seen when matching is logged:

123.123. comment1
123.123.0.0/17 comment2
123.123.123.123 123.123.123.123 comment3
123.123.123.123-123.123.123.123 comment4

This comments are *not* the comments usually used in lists, they can be used
additionally:

123.123. comment1 # a line with a comment1
123.123.0.0/17 comment2



added caching for:
-SPF
-MXA
-PTR
-RWL
-LDAP


added in folder notes:
Config History (confighistory.txt)  
Admin Info (admininfo.txt)
-configdefaults.txt
-config.txt 

Added Options:
-Do Bomb/Script Regular Expressions Checks for ISP/Secondary
-Do URI Blocklist Validation for ISP/Secondary

Added:
All Spam-Haters*
All Emails to Spam-Haters found to be spam are blocked by ASSP rather than
processed in testmode/spamlover. When a Spam-Hater is not the sole recipient of
a message, the message will only be blocked if all recipients are Spam-Haters.
Overwrites Spam-Lover addresses/domains. Accepts specific addresses
(user@domain.com), addresses at local domains (user), or entire local domains
(@domain.com). Wildcards are supported.
For example: jfribo*@thisdomain.com|fribo|@sillyguys.org
Internal Name: spamHaters
Bayesian Spam-Hater*
DNSBL Spam-Hater*

