News and Updates Archive

Back to News & Updates

2012-11-23 New GBUdb Tool

We have been playing with a new utility that some of you may enjoy.

http://www.armresearch.com/message-sniffer/download/GBUDBTool-V0.1.zip

GBUDB Tool allows you to create a list of IP addresses from your GBUdb snapshots (.gbx files). You can select IPs that are "blacker" or "whiter" than a provided probability figure and confidence figure. It outputs one IP per line, optionally with details about the statistics for the IP. This can be useful for feeding-forward blacklists to block at your firewall or for other research purposes.

Run GBUDBTool without any parameters and it will tell you about its command line options.

2012-06-26 Message Sniffer Rule #5,000,000 Coded!

Message Sniffer Rule #5,000,000 was coded by Andy (Worm Thunder) 20120626.1408 SortMonsters Rock!

2012-02-23 Message Sniffer System Upgrades

Here are some of the system upgrades that we have made recently:

Note that over the next few weeks we will be making additional changes to our technical infrastructure. During service windows occurring at times of low-activity there may be short disruptions in SYNC server connections and/or rulebase delivery. We will do our best to avoid these, and those that do occur should go unnoticed.

Your Message Sniffer software installation is designed for high performance and high availability. It will continue to function normally even if we have a disruption during our upgrades, and it will automatically recover from any such disruption without any assistance.

2011-11-01 New Small Business Rates Offered!

We are now offering special pricing for small businesses. We are offering two rates: SMB Rate at $199/instance/year and SOHO Rate at $99/instance/year.

During your 30 day free trial, we will be monitoring your telemetry. We will be monitoring your HAM ratio (average number of good messages per day) that your system processes. Based on the numbers we see, we will notify you via email if you qualify for either of the special rates.

For current customers, you will be notified in your renewal notices if you qualify for these rates.

For questions about these rates, please contact sales at sales@armresearch.com.

2011-09-26 SNF Server/Client for *nix Updated - Important Bug Fixed

Tarball snf-server-3.0.13.tar.gz has been posted to the Downloads page.

This distribution contains some minor bug fixes and code improvements bringing the SNFMulti Engine up to 3.0.17.

IMPORTANT: This distribution also contains a "clean" SNFServer/main.cpp that fixes a random crash bug!

The previous distribution snf-server-3.0.12 contained testing code that would intentionally force a crash (seg fault) under specific load conditions. The testing code would make it appear that SNFServer was crashing at random with crashes being more likely under higher load conditions.

The testing code should not have escaped the lab and was not intended for use in production. We have reviewed adn revised our publishing procedures to ensure this does not happen again. This new distribution snf-server-3.0.13 does not contain the testing code.

This bug was not included in Win* distributions - only snf-server-3.0.11.tar.gz and snf-server-3.0.12.tar.gz included the errant testing code.

2011-04-06 4 Millionth Rule!

We have reached our 4 Millionth Rule! -- Our rule bots now have more than 4 Million heuristics available for activation at any moment. When new spam is spotted that matches an old rule, that rule is reactivated automatically.

The vast majority of our rules have been coded by hand over the years by our amazing Rule-Techs (The SortMonsters). These highly trained professionals work around the clock (24x7x365) and consistently produce the most accurate rules available anywhere. They are really a fantastic team and a great bunch of folks to boot. :-)

2011-01-18 CommuniGate Pro Plugin for MS Windows Updated

We've updated the MS Windows version of our Anti Spam / Anti Malware plugin for CommunigGate Pro.

We have rewritten the documentation and distribution files to make the installation process simpler and clearer. We've also updated the main configuration file with CSS and XSL so that you can view a clear, human friendly version of your snf_engine.xml file simply by opening it in your web browser.

For more information, visit the SNF4CGP page in the Documentation section.

To download the SNF4CGP plugin, visit the Downloads page.

2010-11-13 Rulebase Compiler Retuning Completed

Over the past few days we've finished a major re-tuning of our rulebase compiler system. The improved rulebase compiler bots are just a bit smarter and as a result many systems are receiving their updated rulebase files sooner than ever before. This means capturing more spam early on more systems and as a result more accurate data in GBUdb for new bot-nets. A win for everyone.

2010-06-22 GBUdb.com Website Launched!

We have launched the GBUdb.com website: http://www.gbudb.com.

We have also updated the generator for the truncate.gbudb.net list so that the TXT records include a link to the list descriptor at http://www.gbudb.com/truncate/ and the IP address in [square brackets].

Please tell us what you think.

2010-04-29 Opening truncate.gbudb.net

We have been testing a blacklist based on real-time GBUdb data (generated from Message Sniffer).

We have decided to experiment with opening up the blacklist for a wider audience and so as of now you can use truncate.gbudb.net as an ip4r test.

You should get a result of 127.0.0.1 if the IP is well into the truncate range -- That is: truncate.gbudb.net is designed to be ultra-conservative so that it should be safe to reject connections based on the test in most cases. This also means that it won't block everything -- only the worst of the worst. That said, the folks who have been testing it have reported that it did drop a significant amount of traffic from their systems on average.

UPDATE: RFC 5782 states:

"IPv4-based DNSxLs MUST NOT contain an entry for 127.0.0.1."
and also states:
"The A record contents conventionally have the value 127.0.0.2"
So we will be changing the result code for truncate.gbudb.net to 127.0.0.2 effective immediately.

Please keep us all posted about how it's working for you.

2010-03-30 SNF4SA Upgrade

We have posted two new files to our Downloads page containing an upgrade to our Message Sniffer for Spam Assassin plugin:

Message Sniffer Client/Server for *Nix (Linux, BSD, OSX, etc...)

Message Sniffer For Spam Assassin

The newest version of Message Sniffer for Spam Assassin (SNF4SA) contains minor bug fixes, but most importantly provides support for older implementations of SpamAssassin that do not support dynamic scores from plugins.

When SNF4SA detects a version of SpamAssassin prior to 3.2 it will automatically produce a static score based on reaching the configured threshold. This allows SNF4SA to work automatically in both old and new versions of SpamAssassin to dramatically increase spam filtering performance and accuracy without additional tuning or tweaking.

We implemented this feature because there are some systems out there using older versions of SpamAssassin and the administrators of those systems do not want to upgrade SpamAssassin to the latest version for some reason. Message Sniffer generally runs on these systems without a problem and now so does SNF4SA.

Previously if you were to install SNF4SA on an older version of SpamAssassin it would not work properly and no score would be added when Message Sniffer detected spam. If you have had this experience in the past you should try again with this new version and please let us know.

2010-02-05 Rulebase updates increased by 25%!

After more back-end improvements and some careful analysis we have increased our rulebase update rate by another 25%.

This will mean:

2010-02-04 New Proactive False Positive Prevention Initiatives

Unqualified false positive candidates: Through this review process we are able to remove and modify pattern rules that cause occasional low-level false positives that would otherwise not be reported. This system is already allowing us to recode or remove dozens of rules per day to make them more accurate; and to update our rule coding practices and support systems to further improve our accuracy moving forward.

Real-time rule / IP conflict analysis: This system monitors conflicts between IP reputations and pattern rule matches across the entire fleet of Message Sniffer installations in real-time. Any time a pattern match is in disagreement with a source IP's reputation that information is analyzed and pumped through a sophisticated collection of filters and data-mining tools. The resulting analysis is displayed in real-time in our spam-weather center so that our staff can respond immediately (24x365) if there is any sign of a "bad rule".

2010-01-04 Message Sniffer DLL now used in Declude

The Declude folks have announced version 4.10.42. With this version Declude now integrates Message Sniffer via our DLL.

Benefits:

Here is a link to their announcement as archived on "The Mail Archive".

http://www.mail-archive.com/declude.junkmail@declude.com/msg33094.html

2010-01-01 New Year's Message Sniffer Promotion

For each NEW customer in the month of January 2010, MicroNeil will donate a new sleeping bag to TOP to benefit the homeless in the Washington DC Area!

2009-11-21 Message Sniffer Antispam/Antimalware plugin for CommuniGate Pro Beta Released

Today we're releasing version 0.1.0 (a beta) of our spam filter plugin for CommuniGate Pro (CGP). You can find the distributions on our Downloads page.

We've been testing this for a while in the lab and in our spamtrap processing servers. It's very fast and very stable.

More documentation is on it's way -- however each distribution also contains the documentation typical of CGP plugins.

SNF4CGP (CGPSNF) does everything a typical CommuniGate filter plugin does and a bit more. In addition to providing X- headers that can be used with filter rules, CGPSNF can also be configured to take any of these actions (configurable by result code, of course):

Allow - This is the typical CommuniGate plugin response. CGPSNF will provide X- headers as configured. The X- headers can be used to trigger CGP message processing rules.

Bypass - This action bypasses SNF4CGP -- the message has been scanned and logged, but CGP is not provided with headers and no additional action is taken.

Delete - This action tells CGP to discard the message.

Hold - This action takes the message as it was provided by CGP, injects the SNF headers, and then puts that message in a folder of your choice for later processing. This is a great hook to use if you are a service provider and you want to build sophisticated quarantine and/or policy review processes.

Reject - This action tells CGP to reject the message with the provided reason.

CGPSNF can also be configured to add its log entries to the CGP log for easy review -- even if the log is not stored as a file by SNF (use mode='api'). Also, just like SNFServer, the XCI interface is provided so you can use SNFClient for GBUdb manipulation or "out of band" message scanning. The full SNFServer engine is in place whenever the CGPSNF plugin is active.

As always - there is no need to restart SNF after making changes to the configuration -- so you can change these options on the fly as needed.

If you have any questions please let us know.

2009-09-11 SNFMilter 1.0.3 released -- bug fix

Those of you using SNFMilter should upgrade to the latest.

We have fixed a bug which would cause SNFMilter to exit with a SIGSEGV under some conditions -- Specifically the error would occur when mlfi_connect() was called with a NULL host address.

2009-08-30 Postfix with Milter, Out-of-Sync Issue Fixed

This week Postfix stable release 2.6.5 as well as Postfix legacy release 2.5.9, 2.4.13, and 2.3.19 have been posted. These versions fix the Milter out-of-sync problem. If you are using SNFMilter with postfix, you should consider upgrading to one of these version so that you can enable use of the quarantine method.

2009-08-26 Updates for SNFServer and SNFMilter

We have posted the following new *nix distributions for SNFServer and SNFMilter & Windows SNFServer:

snf-milter-1.0.2.tar.gz
snf-server-3.0.10.tar.gz
SNFServerV3.0.2-E3.0.11.exe

These new versions fix a rare memory leak bug that occurs when corrupt rulebase files are presented to the SNF engine. The SNF engine would read and ultimately reject the bad rulebase file but would not release the memory associated with it.

Most systems never saw this bug because their update mechanism would validate the rulebase (.snf) file before swapping it into place.

As a result most folks don't technically _need_ this update--- but it is best if you update to this latest version when you can schedule it in.

Windows users can download the SNFServerV3.0.2-E3.0.11.exe file,
Stop SMTP (to prevent queuing)
Stop SNFServer
Rename SNFServer.exe to SNFServer.exe.bak
Copy SNFServerV3.0.2-E3.0.11.exe over SNFServer.exe
Start SNFServer
Start SMTP

2009-07-29 SNFMilter Released

Today we've officially released SNFMilter - a version of Message Sniffer that integrates directly with sendmail and postfix servers.

2009-07-29 Updated Client/Server Distribution for Linux, BSD, and & *nix Systems

We've posted a new version of our Client/Server distribution for Linux, BSD, & other *nix systems. You can find snf-server-3.0.9.tar.gz on our Downloads page.

This update contains a fix for a minor bug in the CodeDweller/Networking code: Under some (rare) circumstances SNFServer would exit with SIGPIPE. The new code includes an appropriate use of MSG_NOSIGNAL or SO_NOSIGPIPE depending on the platform used to build the software.

The SIGPIPE bug does not affect Windows systems. However, a new update to the Windows installer is due relatively soon just to keep all of the versions up to date and to update some documentation for some of the integrated platforms.

This update includes improved control scripts that provide for a special debug mode. The debug mode runs SNFServer with a number of debugging options enabled to capture detailed information about how SNFServer is running. Most folks will never need this ;-)

Other improvements to the source code have also been included.

2009-05-12 SNF4SA - Message Sniffer Anti-Spam Plugin for SpamAssassin Released

We have just released a MUCH improved plugin for SpamAssassin. Our new plugin makes full use of the SpamAssassin Plugin API to provide features like:

The SNF4SA plugin is included in the latest *nix distribution of SNF on our Downloads page.

Also we have packaged the SNF4SA plugin separately for those of you running SpamAssassin on Windows machines -- or if you already have SNF up and running and just want to switch to the latest SpamAssassin plugin.

For more information visit our SNF4SA page.

We look forward to your feedback!

2009-03-18 Server Side Solutions is now offering an eWall + Message Sniffer package

Server Side Solutions, creator of eWall, is now offering an eWall + Message Sniffer package, as well as Message Sniffer subscriptions and renewals. Please visit the SNF section of their website for more information.

2009-03-17 Updated Windows Installer

Our Windows installer (available on the Downloads page) now directly supports the following platforms:

It also provides the option to install SNFServer and SNFClient without any particular integration in case you want to do something custom or integrate with a platform that is not yet supported by the installer.

A number of items have been removed from the products page now that they are included in the new installer. Also note that the installer is up to date and now uses the CURL based getRulebase.cmd script.

Note: There are many more Windows based platforms that are compatible with SNF. Some of those install the SNF engine as part of their own installation process. Others require some manual tweaking. Please use the custom / other option from the installer to help with those.

2009-03-17 WIN* SDK (DLL) Prelimary Release

A number of folks have recently asked us about our SDK. Although we are not yet finished with the SDK package (documentation, examples, etc...) the 32 bit DLL itself has been in production on a number of large systems for several years now.

From now on a snapshot of the SDK package will be available from our Downloads page so that folks can download the package, try it out, and give us feedback on how we can improve it.

Yes, for those who are about to ask, we have a 64 bit DLL also (available upon request). Once we've had some more fun testing it we will add it to the SDK package.

2009-03-17 SNF4ASSP Posted

Last year on the prompting of several SNF users we built an ASSP plugin for SNF. We held back posting it to our site because we wanted to see more testing and feedback before making it public. Now we've posted it.

2009-02-02 Announcing ClamAID - Clam AV installer for Windows

We've noticed that folks often have trouble getting Clam AV (the free open source anti-virus scanner) working correctly on their mail servers, so we've created a free product to help solve that: ClamAID (Clam AV Assisted Install Device).

What ClamAID does is collect all of the bits and pieces that make ClamAV work, configure them, install them, and get them running with your email / filtering platform.

So far ClamAID supports IceWarp, Declude/IMail, and Declude/SmarterMail. We will add support for additional platforms as requested (time permitting).

Please take a look, keep us posted on your progress, and tell your friends about ClamAID if it helps you. If you have any questions or run into problems then please let us know.

2008-10-09 SNF Now directly supported in IMGate!

Message Sniffer is now directly supported in Len Conrad's IMGate. IMGate + SNF allows you to move your spam filtering out in front of your mail server improving scalability, stability, and performance.

Here are some links:

http://www.imgate.net/?page_id=101
http://www.imgate.net/?page_id=111

2008-07-31 Installers Posted (available in the Products section)

We have re-posted our Windows Installer for the new Client/Server version of SNF 3.0. This installer will help you upgrade from previous versions of SNF if you are using any of:

Other combinations are also supported and often detected automatically including raw client/server installations for use on systems we don't know about yet ;-)

This installer is relatively new (but well tested in our lab). Please keep us posted on how it works for you.

If you prefer to get the new SNF and install it yourself:
http://www.armresearch.com/message-sniffer/download/SNFWinClientServer3.0.zip

MDaemon users-- don't forget that we nave an installer for the new MDaemon plugin also!

If you prefer to install the new SNF MDaemon plugin manually:
http://www.armresearch.com/message-sniffer/download/SNFMDPlugin.3.0.zip

Linux/BSD/OSX users-- If you haven't heard, the new *nix distribution of SNF has been updated with improved V3 specific instructions and example control & update scripts. Also - if you had trouble compiling SNF before on your PowerPC or 64bit box the latest version includes big/little endian detection and bug fixes. At this time there are no known problems on all of these platforms. As always: Keep us posted please :-)

http://www.armresearch.com/message-sniffer/download/SNFSourceClientServer.3.0.1.zip

Everyone should upgrade to the newest version as soon as practical. While we will continue to support version 2 for a time, version 2 of SNF is deprecated. Of course, that's not the only reason to upgrade. SNF Version 3.0 has many improvements that reduce leakage, reduce the chance for false positives, reduce administration costs, and reduce system loads.

You can find links to our latest distribution files on our Downloads page. You can find installation guides and upgrade notes in the documentation section.

Some of our site is still under construction (it is large). If you find something under construction that would help you please let us know and we will reschedule that work to get it done more quickly. In the mean time we'll be happy to answer your questions directly.

2008-07-31 2 Millionth Rule!

We have reached our 2 Millionth Rule! -- Our rule bots now have more than 2 Million heuristics available for activation at any moment. When new spam is spotted that matches an old rule, that rule is reactivated automatically.

The vast majority of our rules have been coded by hand over the years by our amazing Rule-Techs (The SortMonsters). These highly trained professionals work around the clock (24x7x365) and consistently produce the most accurate rules available anywhere. They are really a fantastic team and a great bunch of folks to boot. :-)

At present about 122315 rules are typically active at one time.

Our most active rule at the moment was coded some 2062 days ago (has it been that long? Wow!).

Here's to the next 2 million !

2008-7-14 New verson of eWall includes tight integration with SNF!

The newest version of eWall from Server Side Solutions includes direct support for SNF:

Here is a link to the announcement: http://forum.sssolutions.net/showthread.php?p=14524

2008-07-12 Rulebase Delivery System Upgraded

Our rulebase delivery subsystem has been upgraded. The new system supports 10x the previous bandwidth and a minimum of 5x the the number of transactions per second.

2008-07-10 *nix Source Distribution Upgraded to 3.0.1

The *nix source distribution has been updated to include Version 3 specific install instructions and to correct a minor bug.

2008-06-26 It's official. SNF Version 3.0 is Ready!

Back in Q1 we were sure we'd be ready with the new SNF after nearly a year of testing on both large and small systems. What a surprise!

After publishing the first release candidate we went from version 1-5 to version 2-27 at a breathtaking pace!

Thank you to everyone who has tested, poked, prodded, and twisted the new SNF -- not to mention keeping up with all of those updates during the final phase of testing. I can't imagine getting to this point without your patience, trust, attention to detail, and persistence! Bravo!

Without further fanfare: Today the latest release candidate becomes the official production release of Message Sniffer (SNF) Version 3.0.

The changes:

We have been bug free for more than 2 months with several hundred systems using the new engine.

You can download the latest distributions from the Downloads page.

You may also notice that we've published our new web site! There are a few bits of documentation still under construction here and there, but we're well on our way to filling those in along with a stream of continues improvements and additions based on our work with you!

Once again, Thanks to everyone for a fantastic job!

Thanks for all of your support, comments, and efforts! As always we're hear to help. Now, onward to the next upgrade... always work to do ;-)

2008-06-20 ARM Research Labs Launches New Website!

2008-06-10 Final RC before Version 3 (fingers crossed)

The latest SNF distributions have just been posted:

http://kb.armresearch.com/index.php?title=Message_Sniffer.GettingStarted.Distributions#NEW_SNF_V2-9_Wide_Beta

This release is a performance update, no new bugs in many weeks now.

Here is a snip from the change log:

20080524 - Version V2-9rc2.25.7

PS: ****** We expect to begin wide testing of two new pieces of software soon: Windows Installers for the MDaemon plugin and Command Line versions of the new SNF. Stay tuned!

2008-04-25 New version: Engine 24, MDPlugin 6

This release is an upgrade more than a bug fix. Replace your SNFServer.exe or snfmdplugin.dll as appropriate.

No changes have been made to the configuration file.

This version improves memory management in the SNF Engine for improved performance, improves the header injection mechanism for improved reliability, and improves logging for IP scans done with the MDaemon plugin.

As usual you can get the latest distributions here:

http://kb.armresearch.com/index.php?title=Message_Sniffer.GettingStarted.Distributions#NEW_SNF_V2-9_Wide_Beta

Here is an excerpt from the change log (this time from the MDaemon plugin change log since it contains all changes from the last version):

20080424 - Version V2-9rc6.24.6

2008-04-16 New Version: Engine 23 - fix for network bug on some win* systems.

This update fixes a bug that effects some Win* systems.

Please replace your SNFServer or snfmdplugin.dll and your SNFClient.

You can always get the latest distribution here:

http://kb.armresearch.com/index.php?title=Message_Sniffer.GettingStarted.Distributions#NEW_SNF_V2-9_Wide_Beta

Here is a snippet from the change log:

20080416 - Version V2-9rc2.23.6

2008-04-13 New Version SYNC bug fix! SNFEngine 22

It seems that in our last update we introduced a bug that effects SYNC operations for some customers - particularly those with longer network transit times to our servers.

The bug could cause SYNC sessions to fail either consistently or intermittently depending on the transit time. If SYNC sessions consistently fail then the new UpdateReady feature will not fire. GBUdb collaboration is also diminished with failed SYNC sessions.

A new version has been posted that solves this problem. Please upgrade your SNFServer or snfmdplugin.dll files from the new distributions as soon as possible to avoid missing telemetry, UpdateReady information, and GBUdb collaboration traffic.

We have also included a new build of the SNFClient program since it uses the same networking library. Although it is unlikely this bug would cause a problem with the SNFClient program you should update to the newest build to be sure.

No configuration changes are necessary with this update.

Here is a description of the changes in this newest distribution:

20080413 - Version V2-9rc2.22.6

20080413 - Version V2-9rc2.21.6

2008-04-11 Latest RC release SNFMulti 20, SNFServer 2, SNFClient 6, MDaemon 5

The newest RC release has been posted in the usual location:

http://kb.armresearch.com/index.php?title=Message_Sniffer. GettingStarted.Distributions#NEW_SNF_V2-9_Wide_Beta

There are NO changes to the configuration file. You need only replace SNFServer.exe and SNFClient.exe and/or snfmdplugin.dll (as appropriate to your system). This release resolves all known bugs / tweaks.

Snippets from the change log:

20080411 - Version V2-9rc2.20.6

20080410 - Version V2-9rc2.19.6

20080409 - Version V2-9rc2.18.6

20080407 - Version V2-9rc2.17.6

2008-04-05 New Version Engine: 16, Client 6

The newest distributions for the Command Line (Std Test Package), MDaemon plugin, and Source have been posted. You can find them here as always:
http://kb.armresearch.com/index.php?title=Message_Sniffer.GettingStarted.Distributions

This update is important because it includes a bug fix to the networking library. This update also includes some tweaks intended to improve network performance under heavy traffic conditions.

Please upgrade to the new DLL, SNFClient, and SNFServer. There is no need to change your configuration file ;-)

A snippet from the change log:

20080405 - SNFServer V2-9rc2.16.6

2008-03-27 More progress SNF2-9 SNFMulti engine goes to version 15

Short version:

Here is a new beta/rc release. The changes are internal and should solve a bug that happens on a handfull of systems. You should upgrade so that you're on the latest version. If you're not having trouble you can put off upgrading until some later time (but not too long please).

Please find the newest release here:

http://kb.armresearch.com/index.php?title=Message_Sniffer.GettingStarted.Distributions#NEW_SNF_V2-9_Wide_Beta

The long version:

This release goes further to eliminate the "hanging" bug on those few systems that see it. One case should be solved completely by this revision and possibly all cases (we shall see).

This release also eliminates a minor bug (not worth a revision) that was in the previous release. It seems I failed to remove a line of code that forces the Debug mode in SNFServer before pushing out the last release -- so SNFServer in the previous release would be in debug mode no matter what -- thus creating extra monitor data on the screen (if not run as a service or piped to /dev/null).

The big change in this release is in the snfNETmgr module that handles SYNC operations (GBUdb & Telemetry). The previous version used blocking IO and a separate thread (TCPWatchdog) to kill off connections that lasted too long. The new version uses non-blocking IO and has been refactored to consolidate some of it's communications routines.

In one case the "hanging" bug presented as a loss of telemetry without errors or exceptions. It appeared from the debug data that the snfNETmgr thread had gotten stuck in an IO call and that even though the TCPWatchdog thread had killed the connection the function call never returned.

The theory supporting this change is that after some number of these TCPWatchdog events the TCP stack might become unstable on some systems and cause this kind of behavior. The new non-blocking methodology eliminates this possibility.

It is possible, if the above theory is true in any way, that this change will solve the other "hanging" cases also -- In those cases the snfXCImgr thread appears to get stuck while attempting to accept() another client. If the TCPWatchdog methodology used before did cause instability in some way to cause this, then this presentation of the "hanging" bug should also disappear.

If the new revision doesn't solve the XCI related "hanging" bug then the addition of very detailed status tracking in the snfXCImgr module should help us see more clearly where to look.

Excerpts from the change log:

20080326 - SNFServer V2-9rc2.15.4

20080325 - SNFServer V2-9rc2.14.4

20080325 - SNFServer V2-9rc2.13.4

2008-03-25 New 2-9rc Versions Posted Client - v2, Server - v4, MDaemon DLL v4, Engine v12

Three new releases today:

SNFv2-9rc2.12.4.StdTestPackage.zip
SNFv2-9rc4.12.4.MDaemon.zip
SNFv2-9rc2.12.4.source.zip

You can find them here as usual:
http://kb.armresearch.com/index.php?title=Message_Sniffer.GettingStarted.Distributions#NEW_SNF_V2-9_Wide_Beta

Some items of note:

There is an obscure, intermittent bug in SNFServer that has been reported on a handfull of systems. The vast majority of systems run normally (including our lab systems) for hundreds of days -- only stopping when we tell them to.

The bug manifests as either:

SNFServer stops listening for requests.
OR
SNFServer stops sending telemetry.

In both of the above cases there are no errors in the logs, no core dumps, no unhandled exceptions, no corruption of any kind--

Only two cases show any kind of pattern so far:

If you come across this scenario please let us know all of the data you can about the situation and then please run your SNFServer in debug mode (see next item) to help us track down this critter.

SNFServer now has a debug mode. If "debug" or "Debug" are found in the path to the SNFServer.exe then debug mode is turned. Most commonly to run SNFServer in debug mode rename it to SNFDebugServer.exe.

When in debug mode SNFServer will make a thread status report to the console once per second along with the usual activity information. The idea is to pipe all of this information to a log file so that when the above bug occurs we can record the status of all of the active threads at that time, before, and after.

For example:

/SNF/SNFServer.exe /SNF/snf_engine.xml > debuglog

---- now some good news ---

There is a new feature in SNFServer. When there is a new rulebase file available SNFServer can call a user-defined script to retrieve the new rulebase file. We've also provided that script and set up the default settings to call it ;-)

The script name is getRulebase.cmd on Win* systems and simply getRulebase on *nix systems. Please read the readme files and check your configuration files to make sure that the script is setup properly for your system. Wget and Gzip utilities are included in all of the above distributions for your convenience.

If the script fails to replace the rulebase file then it will be retried after 3 minutes (default). Retries will continue until the script is successful.

-- The feature can be turned off.

-- SNFServer still produces an UpdateReady.txt file so if you want to continue using that methodolgy nothing will break -- though you should turn off the in your config file.

-- If you write your own script or want to launch the script some other way (such as calling cmd or start with special options) then you can do that -- but be careful! The update-script engine runs in it's own thread and makes a system() call when triggered. If your script fails to return then the update-script thread will be stuck waiting for it to return. Remember: what you put into the call= attribute will be passed to system() when the feature is triggered. The best way to do anything special there is to write a script that does what you want and have the update-script mechanism call that script. It's probably not a good idea to put a lot of special switches and "other craziness" in the call= attribute --- If you need them, put them in your script and keep the call= simple :-)

2008-03-20 MDaemon Plugin SNFv2-9rc4.11.4 Posted

I have just posted the latest beta (release candidate) MDaemon plugin.

You can find the latest betas here:

http://kb.armresearch.com/index.php?title=Message_Sniffer.GettingStarted.Distributions#NEW_SNF_V2-9_Wide_Beta

This distribution includes an automated update utility that is triggered from the SNFServer engine running in the plugin. When a newer rulebase file is available an UpdateAvailable.txt file is created in the SNF directory. The getRulebase.cmd script can be scheduled to run once per minute. When the UpdateAvailable.txt file is present the script will download, validate, and install the latest rulebase file. Before using the getRulebase.cmd script be sure to edit the top of it to establish the correct working directory, license ID, and authentication string.

Engine improvements and updates to the SNFClient utility are also included...

A few excerpts from the change log:

20080319 - Version SNF2-9rc4.11

20080318 - SNF2-9rc1.11.exe Consolidated several mods/fixes

2008-03-07 Version 2-9rc1.8.2 Release Candidate (Std Test Package) Released

This is the first release candidate for what will become version 3 this quarter!

You can find the latest updates here as they arrive:

http://kb.armresearch.com/index.php?title=Message_Sniffer. GettingStarted.Distributions#NEW_SNF_V2-9_Wide_Beta

Over the next few days we will be updating the MDaemon DLL with the new engine and a new feature or two. Then we will update the source distribution for *nix & OEM systems. Then we will be launching two SDKs -- one is a .SO for *nix systems and the other is a DLL for Win* systems. Along the way we will be launching a new web site with documentation for the new version. Then later this year (Q2 - Q3 perhaps) we'll be launching DNS based IP reputation services.

For now -- back to this moment in time and the new SNFServer and SNFClient release. There are extensive updates to both the client and server programs. Be sure to go through the readme files if you are upgrading.

Also - if you are upgrading you will want to update your snf_engine.xml file to cover the new features. (GHASP! What if I forget to do that?!!) -- If you don't get to it right away then your existing snf_engine.xml file will work fine... but do get the update process on your to-do list so you can take advantage of the new features and improved default settings.

Here is a chunk of the change log to show you what is new since version 2-9b1.5.1:

20080306 - SNF2-9rc1.8.exe (FIRST RELEASE CANDIDATE for VERSION 3!)

20080207 - SNF2-9b1.7.exe

2008-03-05 MX Uptime adds Fully Integrated SNF!

The newest version of Message Sniffer is now an integral component of MX Uptime's plugin for MailEnable. Anyone wishing to use SNF only needs to enter their license ID and authentication string, then check the box :-) Screenshot for integration.

2007-10-17 Message Sniffer Version 2-9b1.5 Wide Beta

This version is considered stable for production environments. The next release will include some minor feature additions and improved default settings (thus our long wait while we monitor installed systems and refine our data). If there are no problems with the next release then we will freeze all features and create the official production release in Q1.

2007-10-05 Message Sniffer Version 2-9b1.1 Wide Beta

At your earliest convenience, please follow the following link to read about the newest version of Message Sniffer which has just been released for wide beta testing.

http://kb.armresearch.com/index.php?title= Message_Sniffer.GettingStarted.Distributions#NEW_SNF_V2-9_Wide_Beta

The command line client/server version is available now. It is a drop-in replacement for folks who have been running the current command line version (2-3.5) with a persistent instance on Winx platforms. The version in the posted distribution file requires a P3 or better.

MDaemon and *nix (source) distributions will be coming shortly.

This new engine has been in testing on a number of production systems from the very big to the very small for quite some time. There are no known bugs at this time. None the less, please be careful :-) and read carefully!

A GREAT BIG THANK-YOU goes out to the folks who have helped us alpha test and refine this version over the previous months and weeks through scores of alpha iterations! We really appreciate the help.

Over the next few days/weeks we will be adding documentation and answering questions to help folks explore and make the most use of the new features. We will also be looking for any last minute tweaks that might be needed; and we will be building a list of any additional features and/or refinements that come to light so we can get them into the production release, or at the very least the .1 that will follow.

As always, your comments, questions, and feedback will help guide our efforts. The value of the discussions we share both privately and on this list cannot be overstated.

Thanks for your patience, trust, and participation!

2007-06-26 Rulebase Compiler Upgrade

We have just completed an upgrade to the rulebase compiler software. The new version is 20-50% more efficient - as a result, updates will be produced a bit more quickly and consistently.

There is no need to make any changes on your systems.

2007-02-05 SurgeMail adds a feature to call Message Sniffer.

2007-01-05 Rulebase Update Rate Increased by 16.6%.

Now that the new delivery server is in place and functioning properly, we have re-tuned the rulebase compilers to deliver updated rulebase files 16.6% more quickly on average.

This means that you will receive updated rules more frequently throughout the day and as a result you should also see less leakage and quicker responses to new mutations of spam.

2007-01-05 FTP Access to Rulebases Being Deprecated

Note that FTP downloads of SNF rulebases is deprecated. If you are using FTP to download your rulebase files you should switch to using http w/ gzip as soon as practical.

FTP access to SNF rulebase files will continue for a time but support may be removed without notice in the future. It's a safe bet that FTP access for SNF rulebase files will remain functional through the end of this month however.

2007-01-03 Upgrading SNF rulebase delivery servers

Over the next few days we will be upgrading the SNF rulebase delivery servers. If all goes well - nobody will notice except that downloads will become faster and (likely) more frequent.

On the off- chance that this might effect you or that something unpredicted might happen we am making this announcement :-)

Expect to see the IP change for http://www.sortmonster.net. If you have closed your firewall to outgoing traffic then this may effect you - you will need to make a new "hole". Please also note that the authentication realm has changed on our delivery servers. The old realm was "SortMonster". The new realm is "SNF".

It is possible that you may miss one or more updates during the transition. We will do what we can to minimize this possibility.

2006-10-23 Version 2-3.5 Release -- Faster Engine

The plan was to hold off until the next major release, however in light of recent increases in spam traffic we are pushing out a new version with our faster engine included. All other upgrades are will wait for the major release ;-)

The scanning engine upgrade results in a 2x speed increase that hopefully will help with the higher volumes we are seeing now. Version 2-3.5 also rolls up 2-3.2i1 which included the timing and file locking upgrades. Version 2-3.5 can be found in our wiki, in the Distributions area.

2006-06-19 Rulebase Pacing Updated

We have just reduced our rulebase update pacing from 150 minutes to 120 minutes. This means rulebase updates will now arrive 20% faster.

If you are using a scheduled task to retrieve your updates, please adjust your timing appropriately (about every 60 minutes should be reasonable provided your script checks for an updated file before performing the download).

If you are triggering your updates based on the arrival of our update notification messages then you need not take any additional action - the change will be automatic.

2006-06-07 WeightGate Available

This program is distributed AS-IS, with no warranty of any kind. You are welcome to use this program on your own systems or those that you directly support. Please do not redistribute this program except as noted above, however feel free to recommend this program to others if you wish and direct them to our wiki where they can download it for themselves. Thanks!

This program is most commonly used to control the activation of external test programs from within Declude based on the weight that has been calculated thus far for a given message.

For more information and to get WeightGate, please visit the Tools page (in the Technical Details section) in the wiki.

2006-05-12 Compressed Log Files Now Accepted!

We are now able to accept compressed log files. Compressed log files can either be in zip or gzip form. For complete guidelines on submitting compressed log files please visit the Log Files Technical Details page in our wiki.

2006-04-26 Update Notification FROM Address Changing

We are changing the rulebase update notification's FROM address to:

updates@armresearch.com

You shouldn't have to take any action on this, but just in case you have any filtering or whitelists set up you should change them.

2006-04-05 SNFRV2R3i1 - ready for testing...

The first in a long line of coming updates has been posted for those brave souls who wish to test or may have use for the changes. You are looking for the file: snfrv2r3i1-EngineOnly.zip You can find the current interim release, Version 2-3.2i1 (Engine Only) on the following page:

http://kb.armresearch.com/index.php?title= Message_Sniffer.GettingStarted.Distributions

Be aware - this distribution only contains the SNF executable for Winx systems and source code for BSD, Linux, & other GNU (g++) capable *nix boxen.

BTW: The source now contains a handy make file for a change ;-) Also, we are now using all gnu compilers for testing and development. We previously used Code Warrior for Winx and g++ for *nix. We now use minGW (Code::Blocks) on Winx and g++ on Linux (RHES3) for testing and development.

This release addresses two key areas that are related:

* The timing functions have been replaced using a new cross-platform Timing Module. If you are curious or interested in cross-platform development in C++ you can find more info on that module here:

http://www.microneil.com/OpenLibrary/index.html

The Timing Module simplifies a number of critical timing features in SNF and made it simple to correct some unusual timing and control conditions that would occur on some systems under very specific circumstances -- these were odd, difficult to reproduce bugs which by all indications have been solved now. That is to say, those that I have been able to reproduce have been repaired and tested -- those that I had strong theories about have also been addressed and are very likely solved -- I will know more after your reports ;-)

* During the refit I also did some additional testing and tuning to improve SNF's command-line scanner performance under heavy loads, in transition (dynamic loads) and during live configuration changes (switching from persistent mode to peer-server mode and back), and on systems with multiple processors and higher speed processors (it still works great on slower boxen too). Comparative testing in the lab shows some noticeable improvements in throughput and resilience - YMMV, I look forward to your reports.

There is NO NEED to upgrade to this version at this time unless you are looking for a tiny bit more speed or solving one of the previous timing and/or control bugs (reload, rotate & stop commands for example, or the "Adjusted Persistence Race Condition" on some bsd or linux boxes -- these are now fixed and tested as far as we can test them).

The other reason you might try the new version is if you would like to help us (and others who are cautious of early adoption) by testing the latest and greatest.

Folks using the MDaemon plugin are not effected by these updates since they apply almost exclusively to command line coordination code -- the plugin has no such code ;-) Folks using other plugins, DLLs, SNFMulti or other custom configurations are also not effected by these updates.

Please keep us posted on your results.

2006-03-10 New RuleBot F002 Online

This rulebot captures and creates geocities web links from the "chatty" campaigns. This is largely a time saver for us humans... we will focus our attention more on abstracts for these campaigns now that F002 will be capturing the raw links. Rules from F002 will produce a 60 result code (Ungrouped).

2006-03-06 New Rulebase Compliers Online

Work has been completed to upgrade the rulebase compiler bots.They are now significantly more efficient. As a result, you will be seeing updates more frequently. Previous lag was between 40-120 minutes. Current lag (sustained) is < 5 minutes. More timely updates should equate to lower spam leakage for new spam.

2006-03-06 New Rulebot F001 is Online

Rulebot F001 creates IP rules for sources that consistently failmany tests while also reaching the cleanest of our spamtraps. The rules will appear in group 63. Expect an increase in your rulebase size while F001 catches up with current spamtrap data.

2006-02-15 Updated Expired Rulebase Cleanup Code

New code has been added to the server that delivers rulebase files. The code removes any rulebase file where the license is disabled. This was a task tha was done manually, but is now automated.

If you get a 404 when you attempt to download your rulebase file then it is very likely you need to renew. If you want to check first, feel free to send us a note at support@armresearch.com.

2005-12-21 Sniffer Engine Updates

Increased Updates per Day: Standard rulebase delivery pacing has been changed from 200 to 150. This means that, on average, rulebase files will be recompiled every 2.5 hours or so. This timing will be variable based on system loads etc, but it is a significant improvement. We have sped up our rulebase delivery process by 267%!! (from 3.6 updates/day to 9.6 updates/day).

Improved IP Rule Coding: A new piece of optimization code was added to drop any Received IP rule that has 0 rule strength and is more than 30 days old. This will help to reduce false positives caused by IP rules that "hang on" after the infection/problem with the source is fixed. It also reduces the compiler workload a bit by reducing the core rulebase size.

2005-11-02 Rule Strength Analysis Upgrades

The Rule Strength Analysis upgrade makes the rule strength calculation more sensitive to the recent activity of any given rule. This will also cause rule fitness decisions to be more competitive so that the most effective rules will be more strongly selected over time.

This will improve SNFs performance in two ways:

1. Rulebase files will be smaller and will require less bandwidth to download and to load during operation. There will also be a measurable increase in scanning speed (though this is already measured in small numbers of milliseconds on most systems).

2. The smaller, more efficient files can be compiled and delivered more quickly which will allow us to increase the rate at which we deliver updates.

2005-08-11 Message Sniffer and Assert! Used to Halt New Bagle Variant

Assert! and Message Sniffer rules were quickly updated upon news that a Bagle variant outbreak had reached very high numbers according to AppRiver, a leading anti-spam service provider.  Within hours customers were protected from the rapidly spreading variant, contained in compressed .RAR and .ZIP files.  Though Message Sniffer primarily focuses on anti-spam content filtering, the engine can also help prevent email-borne virus outbreaks.

2005-08-01 ARM Research Releases "Assert! Message Sniffer for SMTP and Exchange"

Assert! version 1.1 encapsulates the raw power of the Message Sniffer engine with an easy, intuitive interface for Exchange or the IIS SMTP Service. Assert! is a powerful anti-spam tool that does not require a bloated feature set or period of tuning to be effective. Assert! includes a one-year subscription to the Message Sniffer spam database, which is automatically updated multiple times daily for pinpoint accuracy.  

2005-07-01 AppRiver LLC and MicroNeil Research Corporation form ARM Research Labs (ARM).

With the goal of exploring ideas and raw data as a means for producing internet-based technology products, a leading anti-spam service provider AppRiver LLC and software research innovator Microneil Research Corporation have joined efforts as ARM Research Labs LLC.  ARM is dedicated to strengthening the world of computing online innovations in areas such as application development, security services and other web-based operations.

Back to News & Updates

Copyright © 2007 - 2020 ARM Research Labs, LLC