[Gross] perpetual match

[Jesse Thompson]

Eino Tuominen wrote:
> Jesse Thompson wrote:
>>
>> This seems to have solved the false positive issue for those campus MTAs
>> that were always at the top of the list.  Although, the match rate
>> didn't seem to change (still at 4-6%,) which I was hoping would go lower.
> 
> I just checked our statistics, we have seen 6.1% matches since September
> 13th. I haven't yet analyzed, if we see something similar. I rember that
> our match percentage was lower - could it be that spammers are starting
> to queue? I try to find time to see those logs more thoroughly.

Our match rate hasn't grown over time as much as you'd think.  For
whatever reason, the spammers generally aren't retrying (other than
maybe firing off 2 messages simultaneously, which is prevented by the
grey_delay=10 setting.)


> Now that SJSM offers Milter interface, it would be possible to demand
> that every message individually should be tried second time. That is, to
> calculate the hash from the whole message, not just the triplet. Of
> course that means more network bandwith would be wasted.

That would just encourage them to retry, meanwhile wasting your
resources.  The ones that are retrying are probably properly retrying
the same message anyway.

I think the way to go is to raise the grey_delay to something higher as
a first step.  Then, if the spammers do retry, it gives the vendor/dnsbl
spam traps time to catch up so that it makes your content scanning more
effective.  Plus, it would help if we build the sophistication into
Gross to perm-block connections when the IP is listed on multiple (or
weighted) blacklists.

Is there a limit to how high you can raise the grey_delay setting?

If I assume correctly, your implementation of this is just to keep the
entry in memory for the grey_delay period of time before adding it to
the bloom filter.  I assume that it wouldn't be a good idea to raise the
setting too high, right?  As an alternative implementation, would it be
possible to configure gross to query only the N-x oldest bloom filters?
 (N is the number of filters, x is the number of filters you want
ignored)  This way you achieve a large-value grey_delay behavior without
the overhead of keeping the pending entries in memory.

If this sounds like something that others would be interested in, I will
take a stab at implementing it.

Jesse
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3340 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.utu.fi/pipermail/gross/attachments/20071112/2cc7d849/attachment.bin>