|
Grex > Oldcoop > #376: The problems with Grex, e-mail and spam | |
|
| Author |
Message |
| 25 new of 480 responses total. |
keesan
|
|
response 168 of 480:
| 
Dec 9 17:29 UTC 2006 |
Rane, you don't need to fiddle with or maintain a very simple spam filter
based only on spamassassin, if you don't mind maybe 20% of the spams slipping
through it,. Set to three stars, it gives me no false positives (I don't lose
any real mail), set to two stars it gets an occasional mail from friends who
you could put on a whitelist. If most of your wanted mail comes from just
a few people, you could whitelist them and have their mail go to a separate
folder to be read first, along with mail from grex. I can set this up for
you if you like, and then you just copy it to .procmailrc. All John would
do is let you type 'change' to select to use this filter, and maybe give you
the opportunity to add the whitelist at the same time, unless he has other
ideas. I am sure you can manage it without him.
|
gull
|
|
response 169 of 480:
|
Dec 10 00:07 UTC 2006 |
Re resp:164: Grex isn't an ISP.
Personally, I find I don't see much spam in email addresses that I
don't list on a webpage or use as domain name contacts. Addresses that
I do one of those two things with quickly become spam magnets.
|
cross
|
|
response 170 of 480:
|
Dec 10 00:23 UTC 2006 |
Regarding #165; Suggesting that someone move on from grex becasue they have
a legitimate complaint is not very productive.
Look. I've been a sysadmin before; I'm qualified to say when I feel that
we're doing a substandard job. And we're doing a substandard job here. Part
of that is because the job is so hard, but part is because staff just doesn't
want to make any changes. Rane is right: each person doing spam filtering
*by themselves* is a huge waste of resources. Really, staff ought to get off
their duffs and do a better job, or let someone who is willing to do a better
job, and is capable, do it for them.
|
spooked
|
|
response 171 of 480:
|
Dec 10 02:48 UTC 2006 |
Dan is spot on with his comments about the status quo of individual user
responsibility for spam management being a huge waste of resources**, and
staff being slack here (and equally or more bad unwilling to change).
That's primarily why I resigned from staff. It is also a big reason why I
have informed staff I'm willing to rejoin staff - I want to change the
poor/apathetic/slack culture of Grex staff.
**FOOTNOTE: TO be fair to remmers, he is working on an opt-in
spam-filter solution which will be a lot better than the current no
solution default. I'm also happy to improve the opt-in solution where
possible if staff can agree to take me back on board (however, like most
things concerning staff on Grex, they are taking their time...)
|
cross
|
|
response 172 of 480:
|
Dec 10 14:45 UTC 2006 |
Like Mic, I have also volunteered to do a number of projects on grex. Also
like Mic, so far, all I've heard are cricket's chirping.
|
mary
|
|
response 173 of 480:
|
Dec 10 15:02 UTC 2006 |
(Whosh)
|
cross
|
|
response 174 of 480:
|
Dec 11 00:11 UTC 2006 |
Is that supposed to have meaning?
|
remmers
|
|
response 175 of 480:
|
Dec 11 14:35 UTC 2006 |
To recapitulate, and to slightly rephrase a response I entered in the
Agora system problems item: There is currently a system-wide
spam filter running - spamd, the "daemonized" version of SpamAssassin.
It works like this: Operating in conjunction with procmail, it reads
configuration files from the user's home directory (.procmailrc and a
few other things) to decide what to do with incoming mail messages for
that user. In particular, the user can decide how aggressive the
filtering should be. SpamAssassin also has Bayesian filtering features.
Sindi has it exactly right: What I've volunteered to do is write a
simple menu-style interface that makes it easy for a user to set this up
for his or her account.
The problem with this approach - and I can certainly understand Rane's
and others' concern - is that if you own the configuration files that
control how your spam filtering is done, you also own the job of keeping
them up-to-date. The fear that this could be a bothersome task is a
legitimate one. I've been experimenting the last few days with
SpamAssassin-based filtering in my own account, and I must say that my
experience contradicts Sindi's. The same filtering options that caught
over 90% of my spam a few months ago are now catching less than 20% of
it. I woke up this morning with over 100 uncaught spam messages that
had accumulated over the last day. If I'm going to make my filtering
more effective, I'll have to tune my SpamAssassin options, or add stuff
to my .procmailrc, or something. Now, I'm something of a computer geek
and enjoy tinkering, so I might find that a bit fun, even. But I can
certainly understand why people don't want to bother with it.
So that raises the question: What can Grex do on a global level,
independent of any user preferences, to control the spam problem? And
do we have the resources - computing power and personnel - to do it?
Offhand, I don't know the answer to that.
|
cross
|
|
response 176 of 480:
|
Dec 11 16:10 UTC 2006 |
Computing power? Yes. Personnel? No. Culture? No.
|
remmers
|
|
response 177 of 480:
|
Dec 11 16:38 UTC 2006 |
Boy, we really suck, don't we.
|
tod
|
|
response 178 of 480:
|
Dec 11 18:16 UTC 2006 |
I'd venture to say that so long as Cindy is tweaking her script then it
wouldn't be too much scripting to propagate those rulesets globally.
Computing power? I dunno..I think grex email is a bad idea altogether.
|
keesan
|
|
response 179 of 480:
|
Dec 11 18:44 UTC 2006 |
Remmers, have you set spamassassin to three points instead of the default five
points? I also set my .procmailrc to put anything on the spamcop list in a
/spam folder, after running spamd, along with anything with 2 points (which
is sometimes real mail). Some real mail ends up in /spam folder.
My tweaks are aimed at the residual 20% or so. I change the stock spam filter
every couple of days, for instance, and throw out HTML with Windows charsets
or 3D or embedded images (some of which comes from people I know).
|
keesan
|
|
response 180 of 480:
|
Dec 11 18:45 UTC 2006 |
Remmers, can you set up a script that will automatically update .procmailrc
for everyone using spamd, if you do want to keep tweaking? Or just include
that option in a change program (update spam filter)?
|
rcurl
|
|
response 181 of 480:
|
Dec 11 20:11 UTC 2006 |
You still have to check all your mail, spam and all, before deleting, don't
you? Or do you send any of it just to dev/null without checking?
|
keesan
|
|
response 182 of 480:
|
Dec 11 20:14 UTC 2006 |
I send things to /dev/null but keep a non-verbose log which I check every few
days in case my filter threw out something it should not. It is much more
aggressive than spamassassin itself, which has never thrown out anything it
should not, using 3 points. I send 2-point mail to a spam folder, and about
1% of the time that is real mail. Ditto for anything on sorbs or spamcop
lists. My aggressive filter is somewhat tempered by a long whitelist of any
friend whose mail got dumped previously by the filter. But you can just use
spamassassin without a log file, set to 3 points, and probably catch 90%.
|
keesan
|
|
response 183 of 480:
|
Dec 11 20:19 UTC 2006 |
The log file is a list of where the mail came from (The 'From' line, which
spammers often fake), the subject line (spammers tend to use creative lines
like 'Hi' or 'Re: Re: Re:'), and where it went (/var/mail/keesan, or
/dev/null, or /var/mail/keesan/spam). Every few days I check over the
/dev/null lines quickly and then delete the log file, at which time it is
around 50K (pages and pages). If you are curious why things go to /dev/null
you can set the log file to verbose and it will tell you which filter(s) the
mail passed through and where it was caught. I used to do that until the spam
count increased. If you notice a lot of the same spam slipping through for
a few days, you can add your own filter before spamassassin.
|
rcurl
|
|
response 184 of 480:
|
Dec 11 20:19 UTC 2006 |
But you still scan the subject of everything, don't you? That's all I do.
I suppose your system just prevents your inbox from filling - or, does it?
I don't expect a perfect system is possible, but so much spam is so obviously
in *classes* of spam, and getting rid of those immediately would be a great
help. Users that want to could keep tweaking a separate filter for themselves.
|
keesan
|
|
response 185 of 480:
|
Dec 11 20:25 UTC 2006 |
My system sends 90% of spam to /dev/null and most of the rest to a spam
folder. I scan really quickly, just what went to /dev/null and came in less
than 2 copies. Nothing to move or delete or save afterward, it is already
gone. This week I got one false positive, since I filter on anything
that looks like a webpage and the sender designed it to look like a webpage
but will consider next time sending out holiday greetings as a text message
with the link to a website.
|
keesan
|
|
response 186 of 480:
|
Dec 11 20:44 UTC 2006 |
I just revised my simple filter and added lots of explanations. See
procmail.simple for instructions how to set up a spam filter using spamc.
Copy one line to .forward and the rest of the file to .procmailrc after
editing out my name and putting in your own. All your mail will be forwarded
to procmail, which runs your filter, sends any whitelisted mail directly to
the inbox, assigns spam points to the rest, dumps anything with 3 points,
sticks anything with 2 points in a spam folder, and puts the rest in inbox.
It keeps a logfile in the mail directory of where mail came from, subject
line, and where it ended up, which you can read and delete every few days.
You should also read the spam folder and delete it once in a while. Adding
additional filters catches more spam but sometimes also gets real mail.
|
cmcgee
|
|
response 187 of 480:
|
Dec 11 21:20 UTC 2006 |
Actually, there are already some system-wide spam filters in effect. Marcus
designed some, and they return an email to the sending system. To track what
filter rule was applied, Marcus put in a cryptic quotation.
I ran across these a few years ago when I was a list admin for a list on a
different system. Some of the mail from that list bounced back from my Grex
address. As admin, I saw the bounced email and made inquiries, but no one
knew what the cryptic quotations actually referred to.
|
tod
|
|
response 188 of 480:
|
Dec 11 21:24 UTC 2006 |
Its a countdown sequence for the alien invasions.
"Haven't you ever wanted to be part of something special?"
|
spooked
|
|
response 189 of 480:
|
Dec 11 21:41 UTC 2006 |
I very much doubt Marcus' quotations/basic spam analysis have been ported
across from SunOS/Sendmail to our current system featuring procmail.
The DEFAULT way to cut spam for Grex (OR SHOULD BE!) is at our mail server
on Grex, BEFORE it reaches user's mailboxes. I have been reading a
technical book dedicated to combatting spam and believe I have the technical
capability to hugely reduce the current epidemic problem of spam facing
Grex user's currently.
The question remains, and the options simple. Does Grex want to move
forward (in anything other than microsteps)? If that is a yes, than staff
should IMMEDIATELY reinstate Dan and myself to staff.
|
mcnally
|
|
response 190 of 480:
|
Dec 11 23:49 UTC 2006 |
re #189: procmail is a local delivery agent. the program which fills
the sendmail niche in our current configuration is exim.
re #187: that sounds typically Marcus. I'm sure if pressed for an
explanation of why he chose cryptic quotations rather than informative
error messages he'd've given a compelling-sounding explanation about
how readable error messages would have helped the spammers when the
reality is that long after he lost interest in the project other
people were reduced to trying to figure out his cryptic and non-standard
configuration choices.
re #189 (again):
> I have been reading a technical book dedicated to combatting spam
> and believe I have the technical capability to hugely reduce the
> current epidemic problem of spam facing Grex user's currently.
I'm all ears. Perhaps you could write a brief summary to tell us
how you would approach the problem, what resources would be required,
what e-mail would be affected, and what the potential downsides of
your approach would be (in your opinion.) If the approach looks
promising I'm betting that the board will support it, but running it
by the user community for comments would be a great first step.
|
spooked
|
|
response 191 of 480:
|
Dec 12 00:18 UTC 2006 |
Yes, 'exim' is our current MTA (Mail Transfer Agent) component - thus my
strong doubts the customisations to 'sendmail' that Marcus made on our last
machine would have been ported over to this machine with a new MTA.
'procmail' is a Mail Delivery Agent (MDA) component (sitting between the
MTA and user mailbox software, aka Mail User Agent (MUA) component, albeit
a very powerful one. Customisations to 'exim' and server-interface
'procmail' components is the way to go.
I would require staff re-instatement before I committed my services any
further. It appears as though they do not value my input and technical
capabilities, however.
|
mcnally
|
|
response 192 of 480:
|
Dec 12 01:55 UTC 2006 |
I was under the impression that you resigned from staff. Its not
surprising if nobody has reinstated you if no request has been
made and you haven't previously indicated a willingness to rescind
your resignation.
I'm not trying to be difficult, though, but if you were evaluating
someone else's proposal, would the "I have a secret plan to reduce spam.
Make me staff and I'll tell you what it is.." be one that you found
persuasive?
|