response 427 of 547:

Jun 24 03:49 UTC 2003

It might be easier to deal with spam by accepting the first N pieces
of mail from a novel host, then delaying the rest; if the mail starts
showing up in the UCE bin, further mail from that host is refused for
an extended period (or perhaps permanently).  This takes care of
hijacked relays as well, while passing the occasional e-mail from an
odd host without any delays at all.

While spam may not be a big bandwidth load, it sure isn't going to
stay small unless we act; it really behooves us to frustrate spammers
if we can.

Jan, is there any way to get Backtalk to salt its pages with fake
e-mail addresses that would be picked up by spam harvesters?  That
would be one way to be certain that a host was sending spam to Grex.

response 429 of 547:

Jun 24 15:34 UTC 2003

I think it's probable that email addresses are being harvested from Grex, but
I'm not sure of the extent of the problem.  Grex backtalk has a robot.txt file
that requests honest robots not to harvest it, which is why google searches
don't find Grex items (somewhat mixed blessing).  Obviously dishonest spammers
are likely to disregard this, and I have seen indications of robots walking
through Backtalk on Grex.  In most Backtalk interfaces, clicking on the user
name will give user bio page - but on Grex that is just the .plan file, and
won't contain in clickable email addresses, which are probably the spammer's
favorite thing to harvest (on other systems Backtalk does have clickable email
links, an issue that I need to address).    Some people will have their
email addresses in their .plan files, but most probably don't or have email
addresses for systems other than Grex, so spam to addressesharvested from
there would go to non-Grex addresses and be hard to recognize.

A spammer might be smart enough to go through the Backtalk pages, pick up login
names and attach "@grex.org" to the end of each.  But I'd think this would be
uncommon.

I'm not really inclined to think that seeding a lot of bad addresses is
going to help enough to be worth the ugliness.  However, you are certainly
welcome to include <A HREF=mailto:uce@grex.org>send spam here</A> links
in all your HTML postings on Grex.

response 431 of 547:

Jun 28 00:22 UTC 2003

More results from using this at work - over 75% of spammers do not come
back within 24 hours.  No evidence that any legit e-mail has been lost.
Looking through the tuple database, substantial spam attacks are *really*
obvious...suggesting automated means to keep 'em locked out after their
tuples age out of probation...i think we're using 5 minutes for this now.

Yes, like any anti-spam technology, this has downsides and costs for both
the infrastructure & users.  But *not* using any anti-spam technology 
also has downsides & costs - like overflowing "in" boxes of left-'cause-
there-was-too-much-spam ex-users.

response 433 of 547:

Jul 2 01:08 UTC 2003

We rolled our own (adding a few lines of C to our mail server software
with MySQL on the back end).  Graylisting is hardly more complex than
a bubblesort routine, and it took fewer lines of code than any bubblesort
that i can recall.

We started it in "observer, record, & report what you'd do" mode.  We've
added more simple features (whitelist, etc.) to it as they've occured to
us.

It looks like spammers are far more impatient on retrying than legit mail
servers - we're hoping to add the rule "if graylisted e-mail comes back 
before the graylist time expires, then add a minute to the interval it 
came back within, compare that sum to the remaining graylist interval, 
and update the graylist interval to the greater of those two".  Tweaking
the numbers promises to let almost all legit e-mail avoid additional
delays from this, hopefully spam with delay itself to death (or blacklist).

response 435 of 547:

Oct 17 17:23 UTC 2003

It's stalled; too many staff have too many other things going on.  I was
going to propose, and I suppose here is as good a place to do it, that
we move nextgrex from Jan's house to the pumpkin after the new version
of OpenBSD comes out.  I think that'll make it easier to test and debug
subsystems, and ultimately easier to transition over from old grex to
new grex.  Once we do that, we should set a schedule; a reasonable one,
that we can stick to, trying to anticipate people's time demands, for
switching over within, say, no more than three months.

Major subsystems I see needing configuration before we can switch are:

        1) The BBS.  Someone has to port PicoSpan (mdw?) or provide
           an alternative (YAPP?  Frontalk?)
        2) Mail.  We have to build out the mail system.
        3) set up newuser
        4) Configuration changes.

That's about it.  Everything else, we can do once we've transitioned.

response 439 of 547:

Oct 17 19:20 UTC 2003

I agree.

Re #437: My understanding is that Grex is running a legal copy of Picospan,
though we do not have an official license.  The validity of our copy comes
from Marcus, so he needs to explain how that works.

response 443 of 547:

Oct 18 02:05 UTC 2003

Regarding #440; We're not going live without a working mail system.
It's not rocket science to put one together, but it does take time,
and that's what everyone is lacking right now.

Regarding Remmers's comment about hardware configuration; As far as
I can tell, with the exception of the ethernet interface (which seems
flakey), it's pretty much set.  Everything left is purely software
configuration.

response 445 of 547:

Oct 18 04:49 UTC 2003

Hey, I agree with you 100%.  It's ridiculous that it's taken this long
to get things rolling.  I think we should be able to move over to next
grex within three months, and I can think of no reason we shouldn't be
able to: this has stalled long enough.

There is some work happening, but you won't see it if you just read coop.
Most of it happens in garage, and some (to a much lesser extent) in
other places.  In particular, despite juggling kids and other time
commitments, Jan has made some stellar progress setting up facets of
nextgrex.  Regardless, we're stalled right now.

Here's my take on some of what happened.  There's been disagreement
among staff about how _best_ to proceed in certain technical areas.
But traditionally, certain staff members have maintained domains of
responsibility comprising various subsystems on grex (like how Marcus
maintains PicoSpan).  Some of this is necessary (Marcus and PicoSpan is a
good example; he's the only one with the source code), some is contrived.
Those parts have been, if not off-limits to other staffers, then at least
considered that individual staffer's responsibility and left up to them.
So, despite some staffers having more time than others, certain areas
of nextgrex remain untouched pending the staffers who have less time,
but traditional responsibility of those areas, to become available.

I don't think this is working out too well.  I think maybe we should
consider just saying, ``screw it; we need to get the new machine online.
Let's figure out the quickest way to do that and go from there.''

*THAT* said, we also have to be careful.  Grex, right now, is a real
mess in my opinion.  There are patches upon patches upon bandaids upon
kludges upon hacks upon more patches stacked up so high, it's difficult
to see over them all.  I think it's scary for newer staff (well, for
me anyway) to *do* anything because everything is so customized.  A lot
of the work Jan has been doing on nextgrex is meticulously documenting
*everything* he's done so that rebuilding grex from scratch is going to
be an easy process for a reasonably competent Unix system administrator.
This is good, and necessary, and we really do need to do this with all
the major components of the system so that, moving forward, we don't
end up in the hole we're in right now (it really is a hole).

I think we can get back on track if a couple of us agree to donate a few
hours or so a day for the next couple of weeks, to get OpenBSD 3.4 to
where we're at now with OpenBSD 3.3 (note: OpenBSD 3.4 comes out on
the first of November).  If we then put the machine in the pumpkin, we
should be good to go with getting nextgrex online in under three months.
hour or so a day for the next few weeks to g

response 447 of 547:

Oct 18 14:46 UTC 2003

I've proved the security of grex's password hash to be the same as that
of sha1, which is provably mathematically superior to MD5.

Also, isn't YAPP shareware?  I thought you had to pay a significant
amount of money for anything other than casual use?

response 449 of 547:

Oct 18 19:06 UTC 2003

Blowfish in OpenBSD is pretty slow.  I've argued many times that building
our own scrambling algorithm was a bad idea, and I certainly would have
done it differently, but it happened before I came on board.  Sorry,
them's the breaks.  Sometimes you just have to accept what you're given
and work with it as best you can: we've got anywhere from 20,000 to
40,000 users whose passwords are hashed with it.  But, we've also gotten
it working with OpenBSD (I've got it working with login.conf on nextgrex).

Jamie, I think you have a lot of good ideas on how to move grex forward,
but don't blow it by being your polemical self.  Three months *is*
a reasonable amount of time for a complete build out of nextgrex.

And I don't care how long it took Salcedo to get mnet up and running.
This isn't mnet, and we want to have an amount of downtown that's a
minimal amount longer than the time it takes to transfer the data from
oldgrex to nextgrex (five weeks just to move things over---now that's
outrageously long, in my opinion).  I don't even live anywhere *near*
Ann Arbor (otherwise, I'd volunteer to take in the hardware and crank
away on it over the next two or three weekends), and other staffers
don't have a lot of time, so it's unlikely we'll be able to half the
amount of time I already projected.

As for YAPP, I'm all for it.  We need something up and running sooner
than later.  However, ``you-think-you-heard-someone-say-maybe'' isn't
a license agreement, and if your primary objection to PicoSpan is a
licensing issue, then trading that can of worms for a mess that is a
verbal licensing agreement for YAPP doesn't seem much better.  Now,
if I had my druthers, I'd just as soon see FrontTalk built out and
used as a replacement for both.  We're sure of the license for that;
unfortunately, that would require someone talking a lot of time to
make happen, and, as we know, time is an issue.

Look, you're preaching to the choir here.  So cut the confrontational
crap and let's figure out how to move forward.

response 451 of 547:

Oct 18 20:19 UTC 2003

I would like to see us stay with picospan.  I left m-net when they switched
to yapp.  I tried it for a couple of weeks, hated it and never logged in
again.