|
|
| Author |
Message |
| 25 new of 248 responses total. |
gull
|
|
response 212 of 248:
| 
Dec 12 20:46 UTC 2002 |
Re #211: Thanks. I don't know about other people here, but mostly I just
wanted to know that the problem was being taken seriously and investigated.
:)
|
dpc
|
|
response 213 of 248:
|
Dec 13 14:56 UTC 2002 |
Thanx! The problem is quite annoying. An e-mail was sent to
me from aadl.org (the Library) on Wednesday, Dec. 11 at 5:36 p.m.
and I didn't receive it until today, Dec. 13, at 3:32 a.m.
|
dang
|
|
response 214 of 248:
|
Dec 14 00:46 UTC 2002 |
It's causing havok on the Grex staff list as staffers answer questions
multiple times because the answers cross each other in flight by hours.
|
mdw
|
|
response 215 of 248:
|
Dec 14 11:29 UTC 2002 |
I think I better understand why mail got slow. We apparently have at
least 3 different spammers trying to connect to grex, but failing to do
so. Presumably there's a firewall, possibly at their end, perhaps in
the middle, that drops packets from grex to the spammers. The half-open
connections occupy slots in the input queue, and there are only a
limited number of such connections that are permitted to exist. After
those slots fill up, other connection attempts are silently ignored.
Since the 3 spammers have at least 20 IP addresses between them, they're
doing a pretty good job of keeping those slots full. And, since we
never actually get any connections from them, let alone mail, they don't
appear in our logs at all. This is certainly going to be "interesting"
to fix. So far, a few simple things I tried haven't been useful, but
there are certainly any number of not so simple fixes we can do.
I suppose we should be thankful that we don't get the spam.
|
mcnally
|
|
response 216 of 248:
|
Dec 14 13:43 UTC 2002 |
Are we running any sort of IP filtering SW that can simply block those
IP addresses completely or don't we have the horsepower for that?
|
keesan
|
|
response 217 of 248:
|
Dec 14 16:11 UTC 2002 |
Can you simply write to these spammers, point out to them that the mail is
not getting through to grex anyway but they are disabling our system, and ask
them to 'delist' us (stop sending things in our direction)?
|
dpc
|
|
response 218 of 248:
|
Dec 14 17:19 UTC 2002 |
Thanks for keeping us informed, Marcus. What a problem!
|
cross
|
|
response 219 of 248:
|
Dec 14 21:20 UTC 2002 |
This response has been erased.
|
mcnally
|
|
response 220 of 248:
|
Dec 14 22:58 UTC 2002 |
It's been a long time since I've dealt with something like this but it
seems at first glance that a simple package like tcp_wrappers should be
fine for this, I don't think you need full bpf-style flexibility.
|
dang
|
|
response 221 of 248:
|
Dec 15 01:04 UTC 2002 |
We can completely block their IP ranges, and we routinely put such
blocks in place. However, there appear to be quite a few legit users
who connect from those ranges, so we'd prefer not to block them entirely
if possible. Besides, if Marcus can fix this problem, then this won't
happen in the future. I'd say blocking the range is a last resort.
|
carson
|
|
response 222 of 248:
|
Dec 15 01:05 UTC 2002 |
(I'd say blocking the range is a first, temporary measure, but that's
a discussion for another conference.)
|
keesan
|
|
response 223 of 248:
|
Dec 15 01:13 UTC 2002 |
What is an IP range? Can you block just specific IP numbers rather than a
range, and if legitimate users are also using those IP numbers can the ISP
be asked to cancel just those accounts or is this a free mail service where
they keep signing up for new accounts? Or am I confused?
|
mdw
|
|
response 224 of 248:
|
Dec 15 01:32 UTC 2002 |
Tcp wrappers can't help - they work at the user level; this problem
isn't even visible at the user level, so there's no way to use it to
reject connections. The DSL router has a limited capability of
filtering packets, but we don't administer it, so that's not useful to
us. obsd pf would be quite capable of that, but we're not on openbsd
yet, and an obsd pf firewall of some sort would also work, but that's
quite a bit of work. It may be worth doing this anyways, but it's not
going to happen this weekend. My current plan is to try to write
something that can "sniff" the wire, detect when grex tries to complete
a connect to a set of bad IP ranges, and send packets to grex to cause
the connections to be aborted. I tried to do just this last night, but
I'm afraid lack of sleep was catching up fast with me and I didn't
succeed in getting this to work. I'm going to try to debug this
separately outside of grex where I can better figure out what's
happening, and hope to have better luck making this work with grex
tomorrow.
|
gelinas
|
|
response 225 of 248:
|
Dec 15 03:33 UTC 2002 |
(#217 assumes that spammers *care* that they are inconveniencing other. If
they did, they'd not send the junk in the first place. So why bother asking
them to stop?)
|
mcnally
|
|
response 226 of 248:
|
Dec 15 05:13 UTC 2002 |
No doubt my understanding of the problem is completely deficient but
nothing said so far illustrates to me why this problem can't be dealt
with by using tcp wrappers to block these IP addresses from connecting
to just the SMTP port.
I'm really not understanding what Marcus is saying in the first sentence
of 224. My recollecton concerning TCP wrappers is that the major reason
to run the package is to allow selective blocking of incoming connections
on certain ports. I'll admit it's been a long time, though, maybe I'm
thinking of another package entirely.
|
gull
|
|
response 227 of 248:
|
Dec 15 17:26 UTC 2002 |
Re #225: They might care they're wasting time trying to connect to a machine
they can't get through to.
Re #226: I got the impression these were half-open connections -- a syn
flood, in effect, filling up the state table. In that case TCP wrappers
wouldn't help because it doesn't kick in until the connection is
established. Did I read wrong?
|
krj
|
|
response 228 of 248:
|
Dec 15 19:01 UTC 2002 |
I have another report of lost mail. I sent my sister e-mail on
Wednesday; she tells me by phone today that she received the mail
promptly and replied, but the reply hasn't shown on up on Grex after
about three days.
|
cross
|
|
response 229 of 248:
|
Dec 15 19:11 UTC 2002 |
This response has been erased.
|
gelinas
|
|
response 230 of 248:
|
Dec 16 01:01 UTC 2002 |
(Three days is the usual "first notification" limit, so she may have gotten
her message back by now, or at least a warning that it has not yet been
delivered.)
|
gelinas
|
|
response 231 of 248:
|
Dec 16 01:21 UTC 2002 |
Re #227: It's bulk mail; the individual recipient, or even the individual
machine, isn't important. *IF* the hung connections were inconveniencing
the senders, they *might* be interested in fixing it. But if they were
being inconvenienced, they'd have noticed, no? They haven't fixed it,
so they (must not | probably don't) care.
|
mdw
|
|
response 232 of 248:
|
Dec 16 02:32 UTC 2002 |
My guess is that the bulk mailers were in the "process" of being
disconnected; they still had the physical IP connection and could send
packets, but were using addresses that were no longer "routable", so it
was no longer possible to send packets back to them.
The tcp wrappers come with a library and it's possible to compile other
programs (such as sshd) to use the same basic blocking code. It
wouldn't be useful for sendmail hwever as there are way too many mail
gateways that, when a connection fails for any reason, immediately retry
without even waiting. So, in fact, you actually want to accept the
connection, wait until they try to send mail, and *then* deliver a 5xx
error code. That *should* clear the queue entry out at their end and
avoid the retry. Grex's sendmail isn't great at doing this, but it does
have support and we use it routinely against people we recognize as
spammers. Last week, we rejected 1904 pieces of mail using this
facility. The 5xx error also contains a URL, and 11 people read the
file referenced by the URL.
|
dang
|
|
response 233 of 248:
|
Dec 16 02:36 UTC 2002 |
SYN floods require kernel level protection. All modern kernels have
such protections built in. Unfortuantly, Grex is not yet running a
modern kernel. Hence our current problem.
|
mcnally
|
|
response 234 of 248:
|
Dec 16 04:36 UTC 2002 |
Ahh.. Ok, now I get it.
|
jazz
|
|
response 235 of 248:
|
Dec 16 14:22 UTC 2002 |
Conventional kernel-level SYN flood protection wouldn't help you here,
anyways, since it isn't a "flood" in the sense that there aren't many
connection attempts being made.
|
davel
|
|
response 236 of 248:
|
Dec 16 20:56 UTC 2002 |
For whatever it's worth: I send (from off site) several messages a day to
a couple of people on Grex. From the system where I send them, I see them
hanging in the queue for long periods. Usually the error message given is
Deferred: Connection timed out with grex.cyberspace.org.
but sometimes it is
Deferred: Interrupted system call
Given what's been said, I assume that it times out because there are too many
of these half-open connections. But I have to wonder if the "Interrupted
system call" error doesn't occur because I somehow get a half-open connection.
At any rate, I thought I'd mention this message.
|
