The Blindness Paradox (and how to get out of it)

As previously stated, messages from IPs in the other ranges continue to be scanned by SNF's pattern matching engine. Messages in the truncate range are not scanned however. This can create what is known as the blindness paradox.

The blindness paradox says that a spam filtering system may become so good at filtering out spam that it can no-longer see what spam looks like.

In order to prevent this, the truncate mode also has a "peek" setting that allows some fraction of truncated messages to be scanned in the normal way. This allows the pattern matching engine to "see" what kinds of messages are coming from the IP source and retrain the GBUdb - albeit at a slower rate than normal.

If an IP source in the truncate range suddenly becomes a source of good messages then the combination of re-training through the "peek" mechanism and regular GBUdb "condensation" will eventually force the IP back into the ordinary black range where all of its messages will be evaluated by the SNF pattern matching engine.

If the system administrator notices the change before the GBUdb then they can always use the SNFClient utility (or an SNF_XCI transaction) to immediately update their system.

Related Topics