14 Dec
2006
14 Dec
'06
8:58 p.m.
The growing problem with SPAM, etc..
It was noted in one of the trade publications I looked at that a very effective
check on SPAM was to confirm that the stream opened up to you (on your port 25)
actually did a "conversation" with the remote player. It seems that the
'bots
just open up port 25 and send (spew), and ignore the responses. Kinda like
'typeahead'. If you can sense this, it quashes some stuff (so I've been
told).
Another tact might be to see if the sender HAS a port 25 on their machine. I
suspect that the 'bot machines don't.
Just a couple of ideas.
Unfortunately fixes like these require a deep diving into the SMTP mailer at
the code level.
ob. on topic.. Sendmail is over 10 years old (maybe that is the problem!)
--
Tom Watson
tsw at johana.com
____________________________________________________________________________________
Have a burning question?
Go to www.Answers.yahoo.com and get answers from real people who know.
15 Dec
15 Dec
9:31 a.m.
On Thu, 14 Dec 2006, Tom Watson wrote:
Unfortunately fixes like these require a deep diving
into the SMTP mailer at
the code level.
ob. on topic.. Sendmail is over 10 years old (maybe that is the problem!)
I prefer Exim because of Sendmail's bitrot.
--
David Griffith
dgriffi at cs.csubak.edu
A: Because it fouls the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing in e-mail?
10:23 a.m.
Tom Watson schrieb:
The growing problem with SPAM, etc..
It was noted in one of the trade publications I looked at that a very effective
check on SPAM was to confirm that the stream opened up to you (on your port 25)
actually did a "conversation" with the remote player. It seems that the
'bots
just open up port 25 and send (spew), and ignore the responses. Kinda like
'typeahead'. If you can sense this, it quashes some stuff (so I've been
told).
Another tact might be to see if the sender HAS a port 25 on their machine. I
suspect that the 'bot machines don't.
Just a couple of ideas.
Unfortunately fixes like these require a deep diving into the SMTP mailer at
the code level.
ob. on topic.. Sendmail is over 10 years old (maybe that is the problem!)
If you have sendmail, have a look at "milter-sender" (google for it)
which intercepts the mail delivery conversation and checks for RFC
compliance. No need to hack sendmail.cf or the source.
Bots, as you correctly noted, do not care about correct SMTP dialogue.
They will be filtered out by milter-sender. Also, it can be configured
to also verify the incoming addresses at the remote MX (which is likely
not the bot, thus it will discard such unauthorized sendings as well). I
use it at our central mail gateway with good success, as the vast
majority of valid mail severs meanwhile are RFC compliant.
Holger
5:33 p.m.
In article <458277BD.4020406 at iais.fraunhofer.de>,
Holger Veit <holger.veit at iais.fraunhofer.de> writes:
Bots, as you correctly noted, do not care about
correct SMTP dialogue.
Not yet. As soon as this starts to cramp their style, they will.
--
"The Direct3D Graphics Pipeline" -- DirectX 9 draft available for download
<http://www.xmission.com/~legalize/book/download/index.html>
Legalize Adulthood! <http://blogs.xmission.com/legalize/>
7:01 p.m.
Also, it can be configured
to also verify the incoming addresses at the remote MX (which is likely
not the bot, thus it will discard such unauthorized sendings as well). I
use it at our central mail gateway with good success, as the vast
majority of valid mail severs meanwhile are RFC compliant.
I've been also being strict about forged hosts, i.e., when you do a reverse
lookup on the IP, and then forward lookup the reverse PTR, the IP should be
there. This has helped a lot, and doesn't impact the common situation where
a host (usually a home static IP) has a legitimate domain, but its PTR is
that of the network upstream (as long as that PTR resolves).
--
--------------------------------- personal: http://www.armory.com/~spectre/ ---
Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckaiser at floodgap.com
-- Proponents of other opinions will be merrily beaten to a bloody pulp. ------
3:06 p.m.
It was noted in one of the trade publications I looked
at that a very
effective check on SPAM was to confirm that the stream opened up to
you (on your port 25) actually did a "conversation" with the remote
player.
I wouldn't call this "very effective". It helps, but not all that
much - unless you combine it with banner delay or DATA delay, at which
point I think "very effective" does become accurate.
Another [tack] might be to see if the sender HAS a
port 25 on their
machine. I suspect that the 'bot machines don't.
Right - but neither do the outgoing mailhosts of large senders, large
enough to have divorced incoming functionality from outgoing
functionality.
Unfortunately fixes like these require a deep diving
into the SMTP
mailer at the code level.
Yes and no.
If you want to implement them in your MTA proper, yes - though some of
the more modern MTAs already have such support present. But you don't
*have* to do it there.
I wrote a "shim" program that accepts incoming SMTP connection and
passes them on to the real MTA when it gets a valid RCPT. It was
designed to turn away bogus RCPTs very very cheaply (and it works
spectacularly well for that[%]), but it also does banner delay, DATA
delay, and early talker rejection. (Because it connects through to a
real MTA in real time, it avoids the worst of the hairy issues that
plague MTAs, like queueing and retries.)
I wrote this for work, but they've given me approval to release it (I
asked preemptively); I can put it up for FTP if there's any interest.
[%] We once had a host from Korea or Taiwan or some such place open up
some two hundred parallel connections and start a dumb-as-rocks
dictionary attack. We turned away some 10-20 thousand bogus RCPTs
in about seven seconds - and never noticed; we had no idea it
happened at all until I stumbled across it when reading logs for
other reasons. Even knowing exactly when it was, I couldn't find
the blip in the load average data we collect.
/~\ The ASCII der Mouse
\ / Ribbon Campaign
X Against HTML mouse at rodents.montreal.qc.ca
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
3:40 p.m.
It was noted in one of the trade publications I looked
at that a very
effective check on SPAM was to confirm that the stream opened up to
you (on your port 25) actually did a "conversation" with the remote
player.
Something I've been experimenting with on one of my SMTP servers is
"greylisting", as implemented in a sendmail milter:
http://hcpnet.free.fr/milter-greylist/
On my system, it blocks over 95% of SPAM delivery attempts. Most of
the SPAMbots I've seen attempting to deliver a message don't wait around
for a reply and simply disconnect when they get the temporary reject code.
Those that do often attempt to deliver the next message with a different
"from" address, which starts a new delay timer. I've also enabled it's
DNSRBL feature, with the SORBS list. Anything that is listed in SORBS
gets a longer delay. I can't say how well it scales on a busy system, but
it's a valid concept.
I passed this on to Jay in a private message.
Mike Loewen mloewen at cpumagic.scol.pa.us
Old Technology http://ripsaw.cac.psu.edu/~mloewen/Oldtech/
4:27 p.m.
Mike Loewen wrote:
[...]
It was noted
in one of the trade publications I looked at that a very
effective check on SPAM was to confirm that the stream opened up to
you (on your port 25) actually did a "conversation" with the remote
player.
Something I've been experimenting with on one of my SMTP servers is
"greylisting", as implemented in a sendmail milter: On my system, it blocks over 95% of SPAM delivery
attempts. Most of
the SPAMbots I've seen attempting to deliver a message don't wait around
for a reply and simply disconnect when they get the temporary reject
code. [...]
The problem, as mentioned in a few places online, is that it increases
the burden of sending a (legitimate) email as it spends more time on the
outgoing mail server being repeatedly sent -- I'm not aware of any
system where the rejection message is parsed and then used to insert the
correct retry delay. For the big boys this would massively increase the
amount of work their servers would have to do.
I experimented with greylisting on my servers which handle email for
clients. It worked *extremely* well, but I still had to turn it off
after about 3 days. We found that customers where failing to get email
from people at big ISPs because their systems would not
retry the mails
(or didn't do enough retry attempts to pass the 1hr barrier.)
Whilst
many people might feel that dropping all emails from AOL[1] is a good
thing my customers didn't seem to agree ...
It might be possible to use in conjunction with other tools, or by
skipping it for the set of ISPs that can't/won't accept being
greylisted, but it really shouldn't be used at the primary filter level.
Alistair
[1] AOL wasn't the only one, but it was the most complained about one.
7:50 p.m.
It was thus said that the Great Alistair MacDonald once stated:
I experimented with greylisting on my servers which handle email for
clients. It worked *extremely* well, but I still had to turn it off
after about 3 days. We found that customers where failing to get email
from people at big ISPs because their systems would not retry the mails
(or didn't do enough retry attempts to pass the 1hr barrier.) Whilst
many people might feel that dropping all emails from AOL[1] is a good
thing my customers didn't seem to agree ...
I've noticed that about 40% of my spam (I run my own personal email
server) comes in through my backup MX host (which has no user information
and accepts *all* mail to my domain). Unrelated, a few months ago we got
DoSed due to an interesting anti-spam technique [2] whereby the MX record
for a domain we were trying to send bounce messages to was set to 127.0.0.1
(and therefore, the mail got requeued back into our system).
I figured that if it was DoSing us, it might be used to DoS the spammers,
so I tried it. [3] It worked okay, but I might be relunctant to put it into
production use.
I also thought that maybe using an MD5 hash on the body of a message would
work, but that's way too easy for the spammers to get around.
One other thing you might want to do (or enforce, if you are a webhosting
company or ISP) is not allow catchalls. We're getting away from that as it
causes us to get blacklisted by AOL and BellSouth (we're in Florida) on a
regular basis.
-spc (Perhaps the best thing would be to kill SMTP ... )
[1] Not my footnote.
[2] http://boston.conman.org/2006/09/05.1
[3] http://boston.conman.org/2006/09/05.3
http://boston.conman.org/2006/09/07.2
8:08 p.m.
Lol. You know that he's on this list, don't you?
http://boston.conman.org/2006/12/13.1
--
--------------------------------- personal: http://www.armory.com/~spectre/ ---
Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckaiser at floodgap.com
-- Good-bye. I am leaving because I am bored. -- George Saunders' dying words -
8:10 p.m.
Oh, my gawd. Never mind. It's *your* blog!
Well, I feel stupid now.
--
--------------------------------- personal: http://www.armory.com/~spectre/ ---
Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckaiser at floodgap.com
-- If I wanted your opinion, I'd have beaten it out of you. -------------------
8:29 p.m.
----- Original Message -----
From: "Tothwolf" <tothwolf at concentric.net>
To: "General Discussion: On-Topic and Off-Topic Posts"
<cctalk at classiccmp.org>
Sent: Friday, December 15, 2006 3:15 PM
Subject: Re: SPAM (on list) was:Re: classiccmp list (sort of) help requested
On Fri, 15 Dec 2006, Sean Conner wrote:
I think most spammers are located overseas, so if you did hunt one getting a
stuffed head through customs might be a little tricky.
-spc (Perhaps the best thing would be to kill
SMTP ... )
I dunno...I think that allowing hunting/culling of spammers would be far
more effective.
-Toth 
4:33 p.m.
It was written...
Something I've been experimenting with on one
of my SMTP servers is
"greylisting", as implemented in a sendmail milter:
Unfortunately most of our customers get really upset when an email doesn't
arrive almost instantaneously. But moreover, believe it or not I've found a
fair number of servers that take the temporary rejection as a permanent one
and never try again. Customers won't accept this. Yes, I know it's easy to
say in an ivory tower mentality "well, that's their fault", but in a
business setting we don't have the luxury of the ivory tower.
Der Mouse mentioned his Shim program... We already use something similar. It
is a poor mans LDAP, where it allows the front ends to check any backend
(based on mailertable) to see if it's a valid account at the RCPT stage.
This blocks all of the spam coming in that is for unknown users in valid
domains (which is around 80% of all the spam we get). It does so without
wasting cpu cycles to scan or lookup anything on the bl lists. It also
maintains it's own cache of positive and negative results so it doesn't have
to keep bugging the backend (you can set the timeout for pos & neg results
separately). It also gets rid of some of the overhead of building up and
tearing down the tcp connection to the back end host by attempting to
utilize already open sessions.
The product is milter-ahead, put out by snert software (snertsoft.com I
believe). It costs about $99 for a site license. They have some other milter
products which do what milter-ahead does and much more, but they are of
course much more expensive. As far as bang-for-the-buck, to have my front
ends know about valid backend mailboxes without Lightweight(not a
chance)DAP, and reject well over 80% of the spam with no processing - I was
quite happy with the purchase.
Jay West
17 Dec
17 Dec
8:39 a.m.
But moreover, believe it or not I've found a fair
number of servers
that take the temporary rejection as a permanent one and never try
again. Customers won't accept this.
Customers have no choice. Greylisting is not the only thing that can
produce a temporary rejection...unless you're running a very
exceptional MTA!
Yes, I know it's easy to say in an ivory tower
mentality "well,
that's their fault", but in a business setting we don't have the
luxury of the ivory tower.
I actually think you don't have the luxury of not, in the long term.
It's decisions like that that are breaking email, worse than any
spammer overload. Tolerating nonconformance only encourages more
nonconformance. (I actually had someone use the term "broken spam
filters" for the filters I have in place that reject gross labeling
errors, like "text/plain; charset=ISO-8859-1" on mail containing octets
which are not 8859-1 printables. Just amazing.)
/~\ The ASCII der Mouse
\ / Ribbon Campaign
X Against HTML mouse at rodents.montreal.qc.ca
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
11:51 p.m.
der Mouse wrote:
Yes - i find it hard to believe that an smtp mta would not 'store and
forward'. At the very least it it would be non-compliant with RFC's and
worse it would be quickly deamed "very broken".
people have multiple mx's for a reason - servers go down.
But moreover,
believe it or not I've found a fair number of servers
that take the temporary rejection as a permanent one and never try
again. Customers won't accept this.
Customers have no choice. Greylisting is not the only thing that can
produce a temporary rejection...unless you're running a very
exceptional MTA! > Yes, I know it's easy to say in an ivory tower
mentality "well,
> that's their fault", but in a business setting we don't have the
> luxury of the ivory tower.
I understand, and agree, but at some point (about now) the volume of spam
gets to be 100x the volume email and something has to be done.
I just turned on greylisting (two different versions, one for each mx)
on a domain and the volume of spam went down 90%. I am willing to pay a
1 hour tax for that relief. The time I was wasting was costing me
money. Plus, I whitelist domains I do business with.
(at some point it becomes an economic issue where the lost time and
messages ends up costing you money. not very ivory tower to me.)
-brad
18 Dec
18 Dec
12:06 a.m.
Brad Parker wrote:
I just turned on greylisting (two different versions,
one for each mx)
on a domain and the volume of spam went down 90%. I am willing to pay a
1 hour tax for that relief. The time I was wasting was costing me
money. Plus, I whitelist domains I do business with.
If they're just firing out the odd spam, is it even necessary to wait an
hour?
Gordon
1:17 a.m.
On Sun, 17 Dec 2006, Brad Parker wrote:
I just turned on greylisting (two different versions,
one for each mx)
on a domain and the volume of spam went down 90%. I am willing to pay a
1 hour tax for that relief. The time I was wasting was costing me
money. Plus, I whitelist domains I do business with.
You don't have to go with the default 1 hour delay on milter-greylist:
I have mine set for 15 minutes, and it had the same result. On the other
hand, systems listed in the SORBS DNSRBL are given a 2 hour delay. I
whitelist frequently used servers (such as the classicmp server and some
of the yahoo groups servers) so there is no delay for those. You can also
play with the auto-whitelist time, so mail that does make it through the
delay will be automatically whitelisted for whatever time you like.
Mike Loewen mloewen at cpumagic.scol.pa.us
Old Technology http://ripsaw.cac.psu.edu/~mloewen/Oldtech/
15 Dec
15 Dec
3:43 p.m.
I wrote a "shim" program that accepts
incoming SMTP connection and
passes them on to the real MTA when it gets a valid RCPT. It was
designed to turn away bogus RCPTs very very cheaply (and it works
spectacularly well for that[%]), but it also does banner delay, DATA
delay, and early talker rejection. (Because it connects through to a
real MTA in real time, it avoids the worst of the hairy issues that
plague MTAs, like queueing and retries.)
I wrote this for work, but they've given me approval to release it (I
asked preemptively); I can put it up for FTP if there's any interest.
I would, for one, be quite interested in your implementation. I have
greet_pause implemented in sendmail, and some fairly draconian custom
rules, but it needs all the help it can get.
--
--------------------------------- personal: http://www.armory.com/~spectre/ ---
Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckaiser at floodgap.com
-- So what's my point? I don't know, it's fun to talk about. -- Judy
Blackburn
3:51 p.m.
Cameron Kaiser wrote:
<AOL>Me too!</AOL>
Peace... Sridhar
I wrote a
"shim" program that accepts incoming SMTP connection and
passes them on to the real MTA when it gets a valid RCPT. It was
designed to turn away bogus RCPTs very very cheaply (and it works
spectacularly well for that[%]), but it also does banner delay, DATA
delay, and early talker rejection. (Because it connects through to a
real MTA in real time, it avoids the worst of the hairy issues that
plague MTAs, like queueing and retries.)
I wrote this for work, but they've given me approval to release it (I
asked preemptively); I can put it up for FTP if there's any interest.
I would, for one, be quite interested in your implementation. I have
greet_pause implemented in sendmail, and some fairly draconian custom
rules, but it needs all the help it can get.
7:38 p.m.
It was thus said that the Great Sridhar Ayengar once stated:
Cameron Kaiser wrote:
<AOL>Me too!</AOL>
Me three!
-spc (Who works at a webhosting company, and still runs his own
personal email server ... )
I wrote this for work, but they've given me
approval to release it (I
asked preemptively); I can put it up for FTP if there's any interest.
I would, for one, be quite interested in your implementation. I have
greet_pause implemented in sendmail, and some fairly draconian custom
rules, but it needs all the help it can get.
7:43 p.m.
I wrote
> I wrote a "shim" program [...]
> I wrote this for work, but they've given me approval to release it
> (I asked preemptively); I can put it up for FTP if there's any
> interest.
I've seen five, I think it is, emails expressing interest (some on
list, some off).
I'll put it up for FTP. I'll try to get it done over the weekend;
we'll see how that goes. I'll drop a note here when it's ready.
/~\ The ASCII der Mouse
\ / Ribbon Campaign
X Against HTML mouse at rodents.montreal.qc.ca
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
20 Dec
20 Dec
7:17 a.m.
Back around Friday I wrote
> I wrote a "shim" program [...]
> I wrote this for work, but they've given me approval to release it
> (I asked preemptively); I can put it up for FTP if there's any
> interest.
I'll put it up for FTP. I'll try to get it
done over the weekend;
we'll see how that goes. I'll drop a note here when it's ready.
I finally got this done. See ftp.rodents.montreal.qc.ca, specifically,
the /mouse/misc/mail/shim/ directory.
I want to set up a samples/ subdir containing (slightly redacted
versions of) the files we use at work for the "designed to be modified
by the mailadmin" code...but that's going to take longer, because I
have to go through them and anonymize some things.
/~\ The ASCII der Mouse
\ / Ribbon Campaign
X Against HTML mouse at rodents.montreal.qc.ca
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
1:48 p.m.
I'll put
it up for FTP. I'll try to get it done over the weekend;
we'll see how that goes. I'll drop a note here when it's ready.
I finally got this done. See ftp.rodents.montreal.qc.ca, specifically,
the /mouse/misc/mail/shim/ directory.
6362
days inactive
6368
days old
24 comments
15 participants
participants (15)
-
A.MacDonald+classiccmp＠slitesys.demon.co.uk -
brad＠heeltoe.com -
dgriffi＠cs.csubak.edu -
gordon＠gjcp.net -
holger.veit＠iais.fraunhofer.de -
jwest＠classiccmp.org -
legalize＠xmission.com -
mloewen＠cpumagic.scol.pa.us -
mouse＠Rodents.Montreal.QC.CA -
ploopster＠gmail.com -
sdc695＠yahoo.com -
spc＠conman.org -
spectre＠floodgap.com -
teoz＠neo.rr.com -
tothwolf＠concentric.net