|
Grex > Coop13 > #376: The problems with Grex, e-mail and spam | |
|
| Author |
Message |
| 25 new of 480 responses total. |
glenda
|
|
response 105 of 480:
| 
Nov 29 00:28 UTC 2006 |
No, we shouldn't reject all Mexican ISPs! Or any other blanket block of ISPs.
If there is a particular IP that is causing nothing but problems, yes. Notice
that I said IP not ISP, a significant difference.
|
mcnally
|
|
response 106 of 480:
|
Nov 29 01:22 UTC 2006 |
re #98: picospan has had the "scribble" segv at least since we moved
to OpenBSD.
|
cross
|
|
response 107 of 480:
|
Nov 29 03:01 UTC 2006 |
Picospan is dead software; it will eventually stop running as assumptions it
made about the underlying system become untrue as the underlying system
evolves.
I've volunteered to work on some of grex's quagmire of email. Slynne said
a month or two ago she was going to talk to baff about giving me some staff
privileges to work on some projects. I don't know what became of that....
|
gull
|
|
response 108 of 480:
|
Nov 29 03:54 UTC 2006 |
What's really necessary is for some staffer to make this their pet
project and bang something together. Dan seems like as good a choice
for that as any. Going after the problem piecemeal is not going to be
nearly as effective.
|
cyklone
|
|
response 109 of 480:
|
Nov 29 03:57 UTC 2006 |
And he would seem to be an obvious answer to #101, as well.
|
maus
|
|
response 110 of 480:
|
Nov 29 08:45 UTC 2006 |
I have two silly questions regarding filtering spam:
- Is there a tutorial or primer on setting up one's account to use
spamc or some other intelligent mail-cleaning bot?
- Do the mail-cleaning bots use a shared (system-wide) corpus of
mail-examples to learn from, or does it have to learn seperately for
each user?
|
cmcgee
|
|
response 111 of 480:
|
Nov 29 13:31 UTC 2006 |
re:103 Well, there is that!
If Dan is willing to do this, I'd encourage Board and Staff to give him the
access he needs to get it done.
|
remmers
|
|
response 112 of 480:
|
Nov 29 14:05 UTC 2006 |
Re #110: I posted an item a while back on how users can set up spamc.
It's in last summer's Agora conference. See item:oldagara,236. For
convenience, I've copied the item text to the file ~remmers/www/
spamc.txt.
You can also read it on the web at http://grex.org/~remmers/spamc.txt .
The process is pretty simple and involves creating a couple of files.
You can configure how "aggressive" you want the spam filtering to be.
I think that a reasonable way to make this approach accessible to more
people (who don't necessarily read agora or even know how to edit files)
would be to have a standalone program that a user could run to create
the necessary files and specify the level of spam filtering wanted. The
existence of the program could be mentioned prominently in the motd. It
could also be made accessible via a "menu" option, or even a web form.
Enabling spamc in your account means that every mail message you receive
is first filtered through SpamAssassin. Some staff members have
expressed concern that this could swamp Grex's CPU if too many users do
this. Although this may be true, spam has become so annoying to so many
users that my inclination would be to try it anyway (unless somebody
comes up with a better solution Real Soon Now), but make it "opt-in" by
requiring users to run the program, rather than making spam filtering an
automatic default for all users.
In addition to the concern about system load, there's also the question
of what to do with spam messages. Discard them? Save them to a
separate "junk" folder in the user's directory, so that the user can
examine them for false positives? The latter approach could eat up disk
space real quick.
If people think this is a reasonable approach, I'm willing to work on
implementing it.
|
remmers
|
|
response 113 of 480:
|
Nov 29 14:06 UTC 2006 |
(Colleen's #111 slipped in.)
|
remmers
|
|
response 114 of 480:
|
Nov 29 14:14 UTC 2006 |
I'll add that another idea discussed at last night's board meeting was to
offer users the option of turning off external mail, i.e. only receive
mail from other Grex users. Once implemented, presumably that choice
could also be built into a general-purpose mail configuration program that
users could run.
|
remmers
|
|
response 115 of 480:
|
Nov 29 14:33 UTC 2006 |
Corrected link to my Agora post on spamc: item:oldagora,236
|
maus
|
|
response 116 of 480:
|
Nov 29 15:07 UTC 2006 |
Thank you for the instructions. While I have no problem with the
instructions given (they look very straight forward), I can see how they
would give someone who has never really worked in a UNIX/BSD environment
fits (I remember how alien pipes and regexes were to me all those many
years ago).
I applaud your offer to make a script or tool to allow users to easily
set this up for themselves, and if I make it out to AnnArbor, I'll buy
you a beer for your effort. It does not look like a horribly nasty
script, but I would not trust myself with other folk's emails (I know
how pissed off I would be if a well-intentioned but badly executed
script from a college kid blew away a letter from a long-lost friend),
so I am not stepping up to write it myself.
|
bru
|
|
response 117 of 480:
|
Nov 29 15:10 UTC 2006 |
MAUS!!!!!
|
maus
|
|
response 118 of 480:
|
Nov 29 15:16 UTC 2006 |
Just wondering, is there a mechanism by which one could mark a message
as spam if it gets through the filter so that the filter's engines learn
from it? I know spamassassin is supposed to be adaptive and adaptable.
How can one help train the system?
|
maus
|
|
response 119 of 480:
|
Nov 29 15:16 UTC 2006 |
Re #117: Squeak!
|
remmers
|
|
response 120 of 480:
|
Nov 29 15:52 UTC 2006 |
Re #117: Not the "maus" you think it is, I suspect.
|
maus
|
|
response 121 of 480:
|
Nov 29 16:36 UTC 2006 |
Probably not, but who would turn down such a warm greeting?
I am not the small, cute rodent from Mnet or The Well. I am not the grad
student with huge boobs. I am, however, the small, cute rodent who has
been inhabiting cyberspace.org for a fair number of years, but who was
too introverted to participate in the discussions until recently.
|
keesan
|
|
response 122 of 480:
|
Nov 29 16:54 UTC 2006 |
I wrote up a small easy filter that you can just copy from my home directory
to yours, along with my .forward file. cp ~keesan/procmail.simple
./.procmailrc. Then change 'keesan' to your own login, and change the
'jdeigert' in my whitelist to the name of someone you want to get mail from.
This filters on anything assigned five points by spamassassin but I would
change it to three points (*/*/* instead of */*/*/*/*) because I never got
a false positive that way. Someone else copied this but did not let me know
yet if it worked. A slightly more complicated sample is procmail.sample .
I think I set this to send */*/* to /dev/null and */* to a spam folder.
Today I got no spam in any folder, after adding a few more filters on such
things as Windows character sets, embedded images, From: debora .
I don't recall if my sample filter keeps a log of what went where, but I have
my own filter set up to keep a short version, which is running 20 pages a day
of mostly spam (at 3 lines per entry).
|
gull
|
|
response 123 of 480:
|
Nov 29 22:34 UTC 2006 |
With spamc the main concern, load wise, is to make sure you're not
running excessively large messages through it. On systems I configure
I generally bypass spamc for messages larger than 1 megabyte. Its
memory and CPU usage goes up rapidly with message size.
|
remmers
|
|
response 124 of 480:
|
Nov 29 22:41 UTC 2006 |
Using the method I described, it's easy to incorporate that.
|
keesan
|
|
response 125 of 480:
|
Nov 29 23:16 UTC 2006 |
I used to dump any message over 100K and now I forward them somewhere else
before running spamc. remmers, are you working on some way to let people set
up a filter without knowing how to copy and edit a file?
|
remmers
|
|
response 126 of 480:
|
Nov 30 13:36 UTC 2006 |
Yes.
|
tsty
|
|
response 127 of 480:
|
Dec 1 09:00 UTC 2006 |
glad i started something progressive ... keep it up - thank you.
,.
|
naftee
|
|
response 128 of 480:
|
Dec 2 19:37 UTC 2006 |
tajnxxxxxxxxx tws
|
remmers
|
|
response 129 of 480:
|
Dec 6 15:55 UTC 2006 |
There's an article in today's New York Times about the recent upsurge in
spam and why methods of dealing with it that were reasonably effective a
few months ago are now failing.
http://www.nytimes.com/2006/12/06/technology/06spam.html
According to the article, spam volume has doubled in the last year, 90%
of internet email messages are spam, and spammers have developed new
techniques that are very effective in getting past existing spam
filters. The article has interesting details on how spammers are
foiling the filters and why they remain motivated -- there are still
enough suckers who fall for their scams to make them money, often a 5%
or 6% return in just two days.
Anti-spam companies are scrambling to develop techniques to filter the
new breed of spam, but they have a way to go to catch up. If and when
they do, spammers will invent new techniques to get around the new
filters, judging from past patterns.
My own experiments with spam control on Grex tend to bear out what the
article is saying. A few months ago, SpamAssassin filtered over 90% of
the spam coming to my mailbox. I reactivated the filter yesterday, and
it was catching less than half of it. In fact, the spam score of most
of the junk messages was 0.0, meaning that SpamAssassin didn't think the
message was suspicious at all.
SpamAssassin has a "learning" feature (the "sa-learn" command; you can
tell it that messages it let through are in fact spam, and that's
supposed to make it smarter about filtering in the future); I've been
playing around with that and will see if it really improves things. But
it's somewhat cumbersome to use. I'm sure users want a spam solution
that "just works" rather than something that requires constant care and
feeding.
The trouble is, nobody has such a solution. Given that companies that
specialize in spam filtering and actually pay their programmers are
having such poor success nowadays, I'm pessimistic about Grex's
prospects of effectively controlling spam, at least in the near term.
Giving users the option of turning off inbound mail entirely seems more
and more desirable.
|