Grex Oldcoop Conference

Item 4: brainstorming solutions for the full disk problem

Entered by valerie on Thu Mar 1 04:28:19 2001:

valerie Jan  8 06:34:18 2004 Valerie Mates valerie Jan  8 06:34:18 2004 Val
valerie Jan  8 06:34:18 2004 Valerie Mates valerie Jan  8 06:34:18 2004 Val
valerie Jan  8 06:34:18 2004 Valerie Mates valerie Jan  8 06:34:18 2004 Val
valerie Jan  8 06:34:18 2004 Valerie Mates valerie Jan  8 06:34:18 2004 Val
valerie Jan  8 06:34:18 2004 Valerie Mates valerie Jan  8 06:34:18 2004 Val
valerie Jan  8 06:34:18 2004 Valerie Mates valerie Jan  8 06:34:18 2004 Val
valerie Jan  8 06:34:18 2004 Valerie Mates valerie Jan  8 06:34:18 2004 Val
valerie Jan  8 06:34:18 2004 Valerie Mates valerie Jan  8 06:34:18 2004 Val
valerie Jan  8 06:34:18 2004 Valerie Mates valerie Jan  8 06:34:18 2004 Val
valerie Jan  8 06:34:18 2004 Valerie Mates valerie Jan  8 06:34:18 2004 Val
valerie Jan  8 06:34:18 2004 Valerie Mates valerie Jan  8 06:34:18 2004 Val
valerie Jan  8 06:34:18 2004 Valerie Mates valerie Jan  8 06:34:18 2004 Val
valerie Jan  8 06:34:18 2004 Valerie Mates valerie Jan  8 06:34:18 2004 Val
valerie Jan  8 06:34:18 2004 Valerie Mates valerie Jan  8 06:34:18 2004 Val
valerie Jan  8 06:34:18 2004 Valerie Mates valerie Jan  8 06:34:18 2004 Val
valerie Jan  8 06:34:18 2004 Valerie Mates valerie Jan  8 06:34:18 2004 Val
valerie Jan  8 06:34:18 2004 Valerie Mates valerie Jan  8 06:34:18 2004 Val
valerie Jan  8 06:34:18 2004 Valerie Mates valerie Jan  8 06:34:18 2004 Val
valerie Jan  8 06:34:18 2004 Valerie Mates valerie Jan  8 06:34:18 2004 Val
valerie Jan  8 06:34:18 2004 Valerie Mates valerie Jan  8 06:34:18 2004 Val
valerie Jan  8 06:34:18 2004 Valerie Mates valerie Jan  8 06:34:18 2004 Val
valerie Jan  8 06:34:18 2004 Valerie Mates valerie Jan  8 06:34:18 2004 Val
valerie Jan  8 06:34:18 2004 Valerie Mates valerie Jan  8 06:34:18 2004 Val
valerie Jan  8 06:34:18 2004 Valerie Mates valerie Jan  8 06:34:18 2004 Val
valerie Jan  8 06:34:18 2004 Valerie Mates valerie Jan  8 06:34:18 2004 Val
valerie Jan  8 06:34:18 2004 Valerie Mates valerie Jan  8 06:34:18 2004 Val
valerie Jan  8 06:34:18 2004 Valerie Mates valerie Jan  8 06:34:18 2004 Val
valerie Jan  8 06:34:18 2004 Valerie Mates valerie Jan  8 06:34:18 2004 Val
valerie Jan  8 06:34:18 2004 Valerie Mates valerie Jan  8 06:34:18 2004 Val
valerie Jan  8 06:34:18 2004 Valerie Mates valerie Jan  8 06:34:18 2004 Val
valerie Jan  8 06:34:18 2004 Valerie Mates valerie Jan  8 06:34:18 2004 Val
valerie Jan  8 06:34:18 2004 Valerie Mates valerie Jan  8 06:34:18 2004 Val
valerie Jan  8 06:34:18 2004 Valerie Mates valerie Jan  8 06:34:18 2004 Val
valerie Jan  8 06:34:18 2004 Valerie Mates valerie Jan  8 06:34:18 2004 Val
valerie Jan  8 06:34:18 2004 Valerie Mates valeri
203 responses total.

#1 of 203 by keesan on Thu Mar 1 04:48:25 2001:

I vote for number 3, but allow people to download the files, not just read
or delete them.  I sometimes fetch files up to 1 or even 1.5M and occasionally
forget to delete them immediately after downloading, or even forget to
download.  I thought the size of file, rather than the type, was what you were
after (no on 4).  No on banning ftp - I thought only paid members to could
ftp files to grex anyway.


#2 of 203 by valerie on Thu Mar 1 04:58:52 2001:

This response has been erased.



#3 of 203 by carson on Thu Mar 1 06:20:44 2001:

(I'd like to see a combo of #3 and the present system.  the way I see it,
if such a program as described in suggestion #3 ran once or twice a day,
(at least), that would relieve some of the burden on staff.  give the
user about a week or so to correct the problem before going in manually,
if necessary.)

(is there an upper limit on the size of files that can be FTP'd/emailed
to Grex?  having that might deter someone from transferring a large
file, as it would have to be broken into pieces first.)

(also, I seem to remember a "bad neighbor" list of the top 20 or so
disk users.  did that become more trouble than it was worth?)


#4 of 203 by scg on Thu Mar 1 06:26:46 2001:

I haven't been directly involved in the disk problem, but I'm guessing the
"bad neighbor" list is one of those ideas that doesn't scale well.  When Grex
was a small system, where most users participated in the conferences or in
party, or had friends who did, showing up on such a list was presumably a
motivation to do something about it.  With a system the size Grex is today,
with many of the disk hogs "just passing through," and not knowing what Grex
is or anything about anybody else who uses or runs it, I'm guessing such a
list would have much less of an effect.


#5 of 203 by carson on Thu Mar 1 07:42:16 2001:

(that was my hunch.  still, it was always good to have a scale of the
problem, which is something that is readily obvious to staffers, but not
so apparent to the rest of us until we can't read conferences.)  ;)


#6 of 203 by krj on Thu Mar 1 15:07:47 2001:

Could a file limit size be set on FTP which would be sufficient for 
people to upload "reasonable things" but which would block most 
software packages and binary files?   Could exceptions to that file limit
size be given by staff upon application?
 
If you were worried about people mailing the stuff to Grex, you'd have
to put a similar size limit on mail files.


#7 of 203 by keesan on Fri Mar 2 02:00:50 2001:

What is wrong with a small binary file?  I use grex to fetch and download and
upload binary files all the time - wordperfect/DOS, small DOS programs, an
occasional picture of someone's kids that arrives in e-mail.  Why is the
nature of the file important rather than the size of it?  Why are 100 pages
of text acceptable but 10K of binary file not?  


#8 of 203 by remmers on Fri Mar 2 02:50:05 2001:

There isn't anything wrong with small binary files that just pass
through Grex once on the way to somewhere else.

People aren't supposed to make image files web-accessible, but
that's a separate issue.  Once they're web-accessible, they pass
through Grex's internet link every time somebody accesses the
person web page.


#9 of 203 by keesan on Fri Mar 2 03:56:25 2001:

Not if they are linked to the main page (you have to select them to see them).


#10 of 203 by mdw on Fri Mar 2 07:55:47 2001:

How does the web server know that?  How do we enforce this with a
minimum of staff time?


#11 of 203 by keesan on Fri Mar 2 19:21:21 2001:

How is it any different what sort of file you have on your home page, whether
a 100K text file or a 100K image file?  Set a size limit, not an arbitrary
statement that images are bad and other things are good.  (I am about to help
grex out by cutting up my 15K website into smaller pieces, only one of which
will load at a time with links to the others.)  Do you want to set some limit
on the total bytes allowed in the www directory?  


#12 of 203 by scott on Fri Mar 2 19:56:11 2001:

People tend to create cluelessly huge websites because of graphics, not text.


#13 of 203 by keesan on Fri Mar 2 22:25:20 2001:

So write a program that detects huge websites.  I see nothing wrong with
having a couple of small graphics files linked from a home page that is also
small, the whole totalling less than many text only pages.  Or make up some
rule about the maximum size of a home page (index.htm(l)) and find a way to
enforce that.


#14 of 203 by gull on Sat Mar 3 03:49:48 2001:

Re #6: There *is* a limit on the size of incoming mail files.  I've had 
a few digests of an email list I'm on bounced because they were over the 
limit.

I think a limit on the size of incoming FTP files would be reasonable, 
and would slow down some of the people trying to bring in eggdrop.  I 
occasionally FTP things here that I need to mail (resumes, mostly) but 
they're all well under 200K.


#15 of 203 by keesan on Sat Mar 3 04:03:45 2001:

I have used grex (and am currently using mnet) to get files up to 2M, that
I fetch with ftp andthen download with ymodem or kermit.  I would find this
sort of restriction a real pain, and it would encoruage me not to be a member
since I would then have to pay an ISP for the priviliege to use FTP to get
files.  I try not to do this during peak times such as 4 pm or 8 pm or
midnight.  Just occasionally issuing warnings to people who are over the limit
significantly (1M plus email) should be adequate, I think.  It wokrked for
me when
I forgot to empty old files out (I just deleted about a third of what I had
to
get down to 700K, smoe of which is binary files).  I have two small image
files that are smaller than my text-only website, which I email to people,
and would object strongly if there were a ban on binary files being received,
sent or stored.  I also have a small DOS program and sometimes have others.
They are not causing anybody any problems.  Can some program be written that
will recognize
eggdrop and other common banned programs and delete them?  


#16 of 203 by gull on Sat Mar 3 05:36:54 2001:

What if the restriction on FTPing large files was only on non-members?  
Members are unlikely to fill up the disk anyway, since they're aware of 
the problem.


#17 of 203 by pfv on Sat Mar 3 20:30:32 2001:

To one subset of users:
        I have not spoken to Val about this recently, this is her own spin.
        I *did* email Anne a week before this item - it's getting silly.

To another subset of users (Keesan and her clones):
        Sending your junk to grex to move elsewhere, via ftp one way and
        *modem another is just plain stupid - which comes as absolutely no
        suprise. The same is true for attachments to email.

The following is a _SIMPLE_ example of a dolt:
==================
genius : units
==================
  3:11pm  up 12 days, 23:41,  41 users,  load average: 2.59, 2.20, 2.04
User     tty       login@  idle   JCPU   PCPU  what
genius   ttys4    12:54pm         5:55     13  pico.real -z psybnc.conf 
------------------
total 11
drwxr-xr-x   3 genius   units         512 Mar  3 14:17 .
drwxr-xr-x  67 root     wheel        2560 Mar  3 08:03 ..
-rw-r--r--   1 genius   units         778 Mar  3 08:03 .cfonce
-rw-r--r--   1 genius   units         664 Mar  3 08:03 .cshrc
-rw-r--r--   1 genius   units         725 Mar  3 08:03 .login
-rw-r--r--   1 genius   units        1245 Mar  3 08:03 .mailrc
-rw-------   1 genius   units         232 Mar  3 08:03 .plan
drwxr-xr-x   4 genius   units         512 Mar  3 14:20 prv
-rw-------   1 root     daemon          0 Mar  3 09:20 root-deleted-your-bnc
------------------
total 1172
drwxr-xr-x   4 genius   units         512 Mar  3 14:20 .
drwxr-xr-x   3 genius   units         512 Mar  3 14:17 ..
-rw-r--r--   1 genius   units      194560 Mar  3 12:37 bnc.tar
drwxr-xr-x   2 genius   units        1024 Mar  3 13:50 bnc2.8.2
-rw-r--r--   1 genius   units      983040 Mar  3 14:18 psyBNC2.2.1.tar
drwxr-xr-x   9 genius   units         512 Mar  3 15:06 psybnc
==================

This clown has HAD a warning, and persists. Why persist in trying to be
polite?

Further, this clown is not a bbs-user, or a party-contributor.. Why would
Grex wait even a MOMENT before smearing it into sludge?

It's not a contributing guest; it's not a member: It's a twit.. Staff has to
deal with these in ungodly numbers, (as I ain't a staffer, I can't define it
further - ask them).


Another point: some deviates actually scream for "more disks and space" - as
though this is a solution. The lack of space is a symptom.

A final point I'll make, and then I'll let you "folks" deal: certain
"gentle-folk" want a higly parameterized solution.. They actually demand
software. Wonderful, software is a Good Thing (tm) - Who is to write it?
When? To do what? How long will it take? (occasionally, folks try to suggest
*I* write a solution - umm, yeah.. right: staff wants me like Bubonic Plague
- I am (and remain) Politically Incorrect.

Val has specified a problem. This problem is both technical, (meaning
solvable), and political, (meaning you better grasp Reality with both
hands). She's asked for SOLUTIONS - not "votes" and not parameters.. She's
OFFERED some possibilities - which means she expects YOU GUYS to discuss
them.

She's already mentioned Quota will not work.. And, to date, not one user to
include scg has, in party, made a goddamned bit of sense.

You have a MAJOR PROBLEM - no, it is NOT trivial when shit locks up or dumps
users.. Time to think it over.


#18 of 203 by gull on Sat Mar 3 20:51:26 2001:

Re #17: How is using Grex to email my resumes "stupid"?  It's the only 
really stable email account I've got.

You've done a great job pointing out the problem.  Where's your 
suggestion for fixing it?

Here's a suggestion that I suspect will be unpopular:  Assign one disk 
for paying members, assign another for non-members.  Run a script that 
deletes all files that are larger than a set size on the non-member 
disk, regardless of content.  Alternatively, just delete the home 
directory of any non-member that's over a certain size.


#19 of 203 by pfv on Sat Mar 3 21:04:10 2001:

re 18:
        Are you a member? FTP, like email and lynx (aka web ftp) is a
        privelege. Far be it from me to define "membership".
        If grex is the "only really stable email.." then it's time to think
        about membership, to my mind..

Solutions:
        I'm noty a root, and almost anything I mention immediately gets
        beaten to death by grexies.. At this stage, it's better I do NOT
        point out solutions.

Your idea:
        yeah, but the drive itself means little - would work as of now.. I
        suspect the crazies will jump all over the idea, in any event.



#20 of 203 by gull on Sun Mar 4 00:20:23 2001:

Re #19: I think, though, that if you're going to lambast people for not 
doing anything, the onus is on you to show how they could do something. 
 If you can't think of any solutions, it's hardly fair to criticize 
others for not coming up with any.

No, I'm not currently a member.  If Grex decided to ban incoming FTP and 
email attachments altogether, than I wouldn't have any use for the place 
and I'd have no reason to become one, either.  As it is, I've been 
planning on it as soon as I get money ahead.  Currently I'm in the 
middle of a not particularly successful job hunt.

And yes, the idea would work whether or not the two were on seperate 
drives.  Moving members to a seperate drive (or just a seperate 
partition) would, however, make the script easier to write, and would 
also prevent members from being inconvenienced if some vandal fills up 
the non-member drive.

I think if I *were* a member I'd hesitate more to suggest this. As it 
is, I'm not really calling for any special privilage for *myself*.


#21 of 203 by keesan on Sun Mar 4 03:15:46 2001:

Re 17, I do not 'send my junk to grex', I download files from other places
so that I can then transfer them (via Kermit) to my own computer and use them
there.  They are not much good to me on someone else's computer.  I receive
attachment in my email because people send them to me - what is so stupid
about letting people send me email?  I think you have some assumption that
is not correct.  I can also get 1M files without ftp, by regular download from
a website.  I try to remember to delete them as soon as I have downloaded them
to my own computer.  I much prefer pine and lynx to Netscape.  I also send
attachments to people, some of whom are grex members, and these attachments
are binary files, and they spend some time in my home directory.  And I store
some files there, some of which are binary, so that I can access them from
various phone numbers.  I think the biggest files I have are text files.


#22 of 203 by aruba on Sun Mar 4 03:25:03 2001:

It's not really practical to put members' directoies on one disk and
nonmembers' on another, because people move on and off the membership list.
We'd have to move their directories whenever that happened, which seems like
a lot of trouble.


#23 of 203 by mdw on Mon Mar 5 02:40:40 2001:

Creating a separate "members" disk also creates a feeling of "I paid for
it, so I deserve better treatment" rather than "I donated for the good
of grex", which means members end up feeling little responsibility to
improve the lot of non-members, which in the long run reduces the amount
of new blood.


#24 of 203 by aruba on Mon Mar 5 16:39:53 2001:

Right, that's been Grex's general philosophy about member perks since the
beginning.


#25 of 203 by davel on Tue Mar 6 01:05:42 2001:

I agree, pretty strongly, but I think this means Marcus & Mark &
I are part of what pfv means when he says "the crazies will jump all over the
idea".

(OK with me, if so.)


#26 of 203 by swa on Tue Mar 6 07:22:06 2001:

More frequent reminders might be a good place to start.  I don't bring
over huge files to my Grex account, but I use it for mail, which has the
tendency to accumulate over time.  I go through and delete bunches of old
mail when I happen to think of it and notice I'm over the disk space limit
-- but generally I notice before staff does; only once have I received a
reminder about this.

This won't solve the problem of disk hogs who are just passing through,
but I suspect I'm not the only person out there who does care, is
absentminded, and responds well to gentle pressure.




#27 of 203 by ryan on Tue Mar 6 17:46:44 2001:

This response has been erased.



#28 of 203 by remmers on Tue Mar 6 20:30:25 2001:

I think we have spare hard drives as it is.  No additional money
needed to install, just some staff time...


#29 of 203 by pfv on Tue Mar 6 20:35:14 2001:

money, space and staff time were the original "bottom-line"

Add space? Same problems will grow.

Older users? Same-same.

NEW USERS? same-same..



#30 of 203 by ryan on Tue Mar 6 20:35:19 2001:

This response has been erased.



#31 of 203 by gull on Tue Mar 6 20:42:10 2001:

I think we just happened to get a good deal on a bunch of 2-gig drives?

pfv's probably right.  Adding space will only delay the problem, not fix it.


#32 of 203 by ryan on Tue Mar 6 20:49:42 2001:

This response has been erased.



#33 of 203 by gull on Tue Mar 6 21:46:26 2001:

The problem is never one twit filling the drive.  It's that we get a 
constant influx of twits who each fill a little chunk.  Staff warns 
them, but being twits they don't care, and the accounts generally aren't 
deleted.

Anyway, from the looks of it we're going to talk this to death and 
ultimately do nothing, again.


#34 of 203 by aruba on Tue Mar 6 21:50:52 2001:

I feel the need to comment on Ryan's comment that Grex has a lot of money in
the bank: we do have money in the bank, but not as much as we used to.  See
the treasurer's report for details.  I think it's time we shed the notion
that Grex is rich.


#35 of 203 by ryan on Tue Mar 6 23:22:00 2001:

This response has been erased.



#36 of 203 by scott on Tue Mar 6 23:23:08 2001:

There *is* some kind of 2GB filesystem limit.


#37 of 203 by ryan on Wed Mar 7 15:12:43 2001:

This response has been erased.



#38 of 203 by scott on Wed Mar 7 16:30:34 2001:

Something like that.


#39 of 203 by keesan on Wed Mar 7 22:56:40 2001:

Is there also a limit on number of partitions?


#40 of 203 by mdw on Wed Mar 7 23:21:17 2001:

Yes.


#41 of 203 by keesan on Thu Mar 8 00:40:08 2001:

Are you going to tell us this limit?


#42 of 203 by gull on Thu Mar 8 03:31:57 2001:

Maybe you should call Sun and ask them.  It seemed to work for
Ameritech.


#43 of 203 by other on Thu Mar 8 05:00:40 2001:

There is a rational philosophy to keeping disk sizes small and numbers 
great.  To wit, the smaller disks limit the potential loss in the event 
of disk failure.

The limiting factor then becomes the controllers and their connections to 
the CPU.  

We need to add another SCSI controller so we can add disk space by adding 
more small (<=2GB) disks.  We should not start obtaining and installing 
much larger disks, because we do not have the resources for redundant 
systems to protect larger disks from data loss.


#44 of 203 by scott on Thu Mar 8 12:15:23 2001:

Re: 42

Regardless of how you feel about the now infamous Ameritech call, that was
just rude.


#45 of 203 by mdw on Thu Mar 8 17:53:44 2001:

I thought I'd wait until somebody actually asked for the limit, although
it's readily available from !man sd or !man dkio, and NDKMAP in
/usr/include/sun/dklabel.h .  I'm not sure if it's obvious from those,
but partition "c" is usually reserved to mean the whole disk, although
that's probably not a big issue.

Another limit is less obvious, and that's the number of access paths and
other performance constraints to a given amount of space.  One 9 G drive
won't perform the same as 4 2 G drives, and for grex, that can be
important at times.  We have extra sbus disk controllers, so in theory
there's no reason we can't have more than 7 scsi devices.


#46 of 203 by keesan on Thu Mar 8 19:01:40 2001:

Thanks, Scott (43) but I think the response in 42 was due to my not putting
a :=) after my second question to make it obvious that I realized that Marcus
was playing 20 questions (his joke).  
Or answering like a computer to a yes/no question.  No hard feelings on my
part and I will try to make my jokes more obvious.


#47 of 203 by gull on Thu Mar 8 19:09:41 2001:

I'm sorry.  It was meant as a somewhat sarcastic joke.  I understand Sindi
is only trying to help, but I sometimes get the impression that the only way
to make her happy would be to make her Dictator of Grex and let her set all
policy herself, from how the checking accounts are run right out to dialin
settings and disk space allocation.


#48 of 203 by pfv on Thu Mar 8 19:11:20 2001:

        Isn't there a login and logout script run for every user - telnet or
        ssh? Someone care to illuminate this?


#49 of 203 by gull on Fri Mar 9 01:36:07 2001:

It's shell-dependent, I think.  csh will run .login and .logout, 
respectively.  bash will run .profile and (in some versions) .bashrc, 
but I don't think it has a provision for a logout script.  I could be 
wrong, though.


#50 of 203 by keesan on Fri Mar 9 15:19:31 2001:

I have just realized that there is no longer any need to move files from the
web via my home directory.  With the new improved Kermit it is possible to
download directly to my own computer.  The older version required typing in
some settings to run it at more than glacial speed and to get around the Y2K
problem (it would not download files at all without doing something about the
date attribute).  This way I will not be forgetting to remove 1M files from
'local disk'.  Kermit is Option 2 for downloads from the web with lynx.
Steve Weiss - thanks again for the new Kermit.  


#51 of 203 by i on Sat Mar 10 04:16:55 2001:

If i understand correctly, most of the problem comes from just-using-us-
as-they-pass-through folks, and longer-term users (members or not) are
pretty well behaved.  Could we have a two-tier systems that was newcomers/
non-newcomers (instead of non-members/members), with some pretty draconian
limits just on the former and a "we're very sorry, but due the the huge
number of people just looking for a system they can abuse in a hurry..."
up-front message explaining it?


#52 of 203 by gull on Sat Mar 10 05:18:06 2001:

Maybe we just need to reap inactive accounts quicker.  What's the current
time for that?


#53 of 203 by aruba on Sat Mar 10 18:45:29 2001:

3 months.  If Walter's idea could be implemented in some reasonable way, it
sounds promising.  It gets around the whole "perks to members" problem.


#54 of 203 by pfv on Sat Mar 10 19:19:42 2001:

        Using a small program and dbopen() would work well and quickly: just
        use the logid for the key, and a date-in-the-future for tha data.
        Alternately, have a root-owned file in the user-space? Or, just base
        it from his account-creation date?

        Now, what would Tier Zero do, (assuming that Tier 1 is what guests
        currently have)?

        What programs would be affected? How would you affect them?
        ( I suspect you'd need a wrapper around lynx, w3m, etc - that checks
          that database/those-files. What about file-attachments?)

        Tiering sounds a lot like adding a new, more limited group akin to
        member or guest.

        Being promoted to "guest" would still leave the OTHER wastrels, but
        I suspect most will have been winnowed out at Tier Zero.

        Isn't there a global, system-defined .logout and .login file that
        grex uses? On a per-user basis, it seems to me that cleaning right
        up after a user would be very light on the system. Cleaning up or
        denying them .login might also be of use, (not sure in the latter
        case).


#55 of 203 by keesan on Sat Mar 10 19:46:32 2001:

Wasn't there some discussion of dropping people in less than three months if
they had only logged on once?  Say in one month instead of three?  And maybe
only if they did not have any email.


#56 of 203 by scott on Sat Mar 10 21:13:32 2001:

It already happens.


#57 of 203 by carson on Mon Mar 12 00:17:47 2001:

(is there still an auto-reaper, or has that gone by the wayside?)


#58 of 203 by don on Fri Mar 16 04:26:42 2001:

A completely unorthodox and probably not-very-useful suggestion: probably the
biggest consumer of space for non-malicious userrs is mail, right? Most
people's reason for not deleting mail is so that they can refer back to it.
Space could be readily freed up if these mail archives were removed. So, a
possible solution would be to activate POP mail services. With a local copy
of their mail, people would be much more inclined to remove their mail from
grex. There wouldn't be any increaesd bandwidth used, since either way the
mail goes from grex to the user. The only drawbacks I could see with this
would be reduced incentive to participate in the grex community (although
opening elm or pine at a bash prompt doesn't provide much incentive either)
and an increase in freeloaders looking for free POP mail (this doesn't seem
to be a problem with m-net's POP service, although they have a slightly
different situation over there). A small advantage would be freeing up ports
due to a lower number of people logging in to read mail.


#59 of 203 by pfv on Sat Mar 17 18:15:35 2001:

        I don't believe it's mail.. Unless you use procmail, the mail is on
        a different drive.

        Do we have any consensus, yet?

        I've been tinkering with scripts, which I hate, and - as you would
        expect - it's not a LOT of users.. There are a small number of users
        that use vastly more space than they need; it's a small number of
        users that simply refuse to read newuser; it's a small number of
        users that download jpg's, jpegs, gifs, mpegs, and even mp3's. Exe's
        and com files.. It's also "dead files" that never get reaped.

        Someone feel like posting a member/staff overview?


#60 of 203 by russ on Sun Mar 18 14:35:41 2001:

Are there any common elements between the people filling up the disk,
like downloading eggdrop?  Or are they all different?

If there are one or a few common profiles we could do something to
address them.  Just having ftp look for common eggdrop file names and
just making a symlink to a local copy would save tons of bandwidth and
disk space.  If it saved a record of doing this, a daemon could follow
up later and remove the mess automatically.

These people have nothing to offer Grex.  The faster we convince them that
Grex is a waste of their time, the faster we can get rid of their files.

I like the idea of keeping a "serious users" partition which hosts
long-term users, people who participate in BBS, and the like.  Let
the newusers fill up the newuser partition.  Better yet, put the
login names of offenders in the MOTD and hold them up to public shame.
Nothing like a few dozen nastygrams in the mail to wake people up.


#61 of 203 by gull on Sun Mar 18 18:52:57 2001:

I'd bet most of the offenders never come back once they find out eggdrop
won't run.  They probably never read their mail or the MOTD, and could
probably care less what the rest of us think of them.  We should just reap
their accounts quicker.  Anything else is just a waste of time, probably.


#62 of 203 by pfv on Sun Mar 18 19:11:14 2001:

        This is why, above somewhere, I asked about ".logout".

        If, (at logout), they have ANY files we object to - remove them,
        (f we REALLY object to them, just reap the account).


#63 of 203 by jp2 on Thu Mar 22 14:36:15 2001:

This response has been erased.



#64 of 203 by tenser on Fri Mar 30 08:09:09 2001:

Regarding #63.  Perhaps I'm being naive, but I'm almost willing to bet
that the people who are downloading lots of pictures, copies of eggdrop,
etc are not imaginative enough to name them ``addressbook.txt'' in
order to disguise the fact.  Probably, they download eggdrop, realize
it won't run (if they can even compile it), and split leaving a pretty
hefty tarball in their home directory.  What's more, said tarball will
be named something like, yup, you guess it: eggdrop.tar.gz.

A skulker script that runs nightly or on the weekends ought to be able
to look for certain file globs, and delete them if it finds them.  (eg,
eggdrop*.tar.gz, nekked*.jpg, *.mp3, etc).  If the list of acceptable
deletable file names is small enough, and widely published, I don't
see how many conflicts would arise in practice.  The only one I can
realisticially think of is eggdrop_soup_recipes.tar.gz :-).

Another solution is to turn on disk quotes.  Maybe not the best idea for
SunOS 4 (Although I used it back in the day; I don't recall it being
that bad, but I never tried it with 30,000 users.  Others I knew did,
though, and didn't complain.  :-)  If the soft quota were set to, say, 1
meg, and the hard quota to, say, 10 megs, then it seems that those users
wishing to move transient files around would still be accomodated while
the goal of putting an upper `cap' on a user's total disk usage would
still be met.  (Note for those unfamiliar with how Unix disk quotas work:
You can have two numbers; one is the ``hard'' quota which you are not
allowed to violate.  The other is the soft quota, which you are allowed
to violate for a short amount of time, but which becomes the hard limit
if it's violated for too long.)

Moving to, eg, Solaris could help make this a reality.  While I think
that it's kind of cool that grex still runs SunOS 4, I just don't see
that being a realistic thing to do for much longer (read a few years
at the max).

In order to gather more meaningful data, I'd be interested in what
the output of, eg, ``quot -v /dev/sd<up> | sort -nr | fgrep -vf \
list_of_staff_accounts'' is.  Could someone from the staff please post
the output of that command somewhere?  (If it's felt this would be too
much of an invasion of privacy, then perhaps strip out the login using
awk or cut first).  It would be interesting to see what percentage of
the user population accounts for what percentage of the space, and also
what percentage of the total disk space hasn't been accessed in the last
3 months.  Also, what's the correlation between disk usage and activity
on the system?  Can output of sa(8) of something similar be matched
up against the output of the above pipeline?

One other thing which might help is to move staff home directories to
their own filesystem.  I say this because staff users have a legitimate
need to use largish amounts of disk space during the course of their
work (imagine compiling emacs :-), but it seems unfair for this work
to infringe upon general user space.  Also, from my own days as a Unix
admin, it helped to have administrative users on their own FS in order
to prevent the possible damage from an ``oops.''  I won't elaborate on
that, as it brings back bad memories.  :-)

Another thought for those using grex to stage files from the Internet: you
might find it convenient to consider downloading the files to a public,
but much smaller, filesystem such as /tmp/$USER, or maybe a new /scratch
or /var/stage or something.  These directories can be policed pretty
heavily, and with the understanding that files left in them absolutely
will be deleted after a day or two.  (I say it's convenient since the
burden for cleaning up can be left to a nightly script....  :-)

Finally, on a somewhat related note....  Might I suggest putting script
wrappers around the most popular MUA's that conditionally print out a
message saying something like, ``Hi, before we start the mail program,
may we suggest that if you're only here for the free email, you instead
use something like Hotmail or Yahoo!'s free email service?  The load
email places on our server is very high... (etc)'', waits a few seconds,
and then starts the MUA?  Printing of the message could be related to
the existence of an environment variable or file in one's home directory.
Eg, in /usr/local/bin/elm:

        #!/bin/sh
        [ -f $HOME/.email_only ] && cat /etc/email_only_notice && sleep 10
        exec /usr/local/real-elm

or something similar.  The ``in your face'' nature of this scheme might
help in convincing people that there are better places to get email....
The delay helps, since a >1sec delay has the effect of making people
lose track of what they were doing, thus making them do things they
might not normally do, like read text on the screen.  :-) Those who
really need to read mail on grex can simply ``rm $HOME/.email_only''
to avoid the notice and the delay.

        - Dan C.



#65 of 203 by carson on Fri Mar 30 08:53:46 2001:

(wow.  that's a lotta suggestion.  kewl.)


#66 of 203 by gelinas on Fri Mar 30 14:08:51 2001:

I don't like Swiss Army knives; they look good, but they try to do too much,
in my opinion.  I feel the same away aboout most tools.  So suggesting
HoTMaiL and its ilk isn't going to impress me.  I want my kids to have e-mail.
Right now, I can't run a mail server in the house.  So grex is the next best
thing.  To the best of my knowledge and belief, they are not using picospan.
Eventually, they may.  Discouraging them from using grex because they are
just reading mail seems to me short-sighted.


#67 of 203 by pfv on Fri Mar 30 14:33:48 2001:

        I suspect the general use of mail is not a problem, and I'm aware
        that grex mail is size-restricted anyway, (although I think it gets
        the whole thing and THEN complains about oversize).

        Otoh, spam-mail MIGHT be a problem.. And, certainly, pine and elm
        (and sendmail?) leave droppings around in many directories.

        Further droppings are left via crazy web-users saving freaky "pages"
        (the names are wild, and I suspect the content as well).

        Downloading gifs and jp[e]gs and suchlike.. And, yes.. The infamous
        eggdrops as well as ssh, bnc, pysbnc, irc, bitchx, mIRC and lord
        knows what else. Further, it isn't JUST the tarballs, it's the
        expansions.. the objects.. the useless compiles, hell the cpu wasted
        FOR the compile..

        I'm still waiting for a staffer to report if there is a global
        .logout being run, or CAPABLE of being run.. Yes, a cleanup therein
        is probably too little, too late - but it also means that the
        staffers could waste less time, (and grex less cpu), by checking on
        a per-user basis.

        Perhaps "login" should fork a root process that checks 'df' and can
        check 'last' for, ohh... the last 1000 lines? Just checking THOSE
        user-spaces might manage to clean up most of the mess.



#68 of 203 by scott on Fri Mar 30 14:43:59 2001:

Perhaps Grex should just stop allowing people to use the system; it would
certainly be easier to administer.  ;)


#69 of 203 by carson on Fri Mar 30 15:17:27 2001:

(that's a great idea, scott!  it seems to work for M-Net.)  ;)


#70 of 203 by pfv on Fri Mar 30 15:19:41 2001:

        <shrug> Could be, but that wasn't suggested anywhere else.


#71 of 203 by gelinas on Fri Mar 30 15:34:58 2001:

(Thank you, Pete, for reminding me *why* my umask is set to 077.)


#72 of 203 by pfv on Fri Mar 30 15:39:58 2001:

(You are quite welcome.)


#73 of 203 by tenser on Fri Mar 30 18:27:28 2001:

Regarding #66.  I'm sorry, but I don't quite follow what you were
saying.

        - Dan C.



#74 of 203 by tenser on Fri Mar 30 19:23:41 2001:

Regarding #67.  It's been my experience that email can be a pretty big
resource hog.  It's not just the disk usage, but also the memory, CPU,
and network bandwidth which is consumed, as well as the load on the
I/O system's bandwidth for spooling and generally moving data around.
Consider, for instance, what happens if grex goes offline for a few
hours and then comes back on....  The load associated with the resulting
``mail spike'' is nontrivial.

(Note also that I'm not advocating turning off email access; just,
by default, making a little more inconvenient.

Csh and derived shells, when invoked as a login shell, will run a
``.logout'' file when a user logs out if it exists in the user's home
directory and is readable.  However, the user can ``turn this off''
by removing ~/.logout, and it's not entirely clear what would be in
it anyway.  For instance, does it do a find and look for common ``junk''
directory and file names?  What does it do then?  Wouldn't a ``normal''
user be kind of bothered if they had to sit there trying to log out
for 5 minutes while something walks their directory tree and does a ton
of stat(2) calls?  Do let it run in the background after the (On this
machine, I'm not sure that's unlikely even for a user with a relatively
small directory...).  shell has exited, and risk it being killed by this
``robocop'' program, possibly leaving the user's directory in a terribly
inconsistant state?

No, it's much better to run such things in a controlled way, via an
administrative program invoked by cron.

As for having login fork and run a program that invokes df and last,
what exactly does that do?  I'm confused.  :-)

Any sort of skulker script which looks around the filesystem and deletes
things could look for directories named things like eggdrop* (in addition
to files) and remove them.

As for the CPU cycles lost in people compiling things that just won't
run on grex.   Yeah, but what can you do about that, realistically?
You could disable the C compiler, but that seems overly harsh.  Maybe a
solution would be to move it to a directory that's not ordinarily in the
user path (Solaris puts it in /opt/SUNWspro/bin, but then the compiler is
an add on product under Solaris), and then put a note in the FAQ saying,
``if you want to use the C compiler, put the following in your PATH:
....''  That might solve a lot of problems anyway, but is a pain.


#75 of 203 by gelinas on Fri Mar 30 20:04:36 2001:

Dan, in #73:  #66 is a statment that I like a tool to do one thing and
do it well.  If I need to do something else, I'll use a different tool.
Web browsers display HTML.  They should do that very well, including,
reasonably, invoking other applications to display certain HTML-encoded
content.  Expecting them to also display, and compose, mail (or news
or chats or et cetera ad infinitum ad nauseum) is misguided at best.
Suggesting that I use a use one one for those purposes is laughable.
*Requiring* me to use one for those purposes is doomed to failure. If I
want to read mail, I'll use a mail client.  And so will those I teach.

You *can* use a hammer to drive a screw, but it's going to take more
effort, and the screw won't work as well as nail would, or as it would
had it been driven with a screw driver.


#76 of 203 by tenser on Fri Mar 30 21:34:08 2001:

Regarding #75.  Joseph, I'm not sure what I said that made you think that
I was suggesting *requiring* someone to use one of the web based mail
services, and not allowing them to use grex for mail.  All I'm suggesting
is that, by default, a message pop up that enumerates alternatives and
waits a few seconds before proceeding so that the user is sure to see
that this isn't the *only* way.  Note also that I gave a very easy way
to turn off this ``mail inconvenience feature.''

To justify, and in the spirit of your point, I would humbly submit that
grex is a tool for computer conferencing, not providing free email.
Services like hotmail, on the other hand, *are* tools for providing free
email, and as such, are better for those seeking as much.

As for your assertion regarding web interfaces vs. more traditional MUAs.
Well, you're right; a web browser is a tool for displaying HTML...and
accepting form-based input from a user.  Note, however, that this is all
that Hotmail et al do.  That is, accept form input from a user and return
HTML which is rendered and displayed by the browser.  No violation of
single use principles there!  It's just that embedded within that HTML
is a user's email.

What's more, I've run across many users who have eschewed Unix-based
CLI-oriented MUA's for the free web-based services.  Their justification
is usually that they found the CLI-based MUAs too difficult to use.  If
this interface works better for them, then that's fine.  In other words,
``what makes a good email client'' is subjective.  If an application
running within the context of a web server works for someone, then who
am I to knock it?  I like using MH, and would be terrible if forced to
suffer the abomnible interface of pine, for instance.  Others might feel
(vehemently) different, however, and utterly rebel if forced to use MH.

So, to summarize, I'm not advocating forcing anyone to do anything, other
than wait a few seconds when reading email until they figure out how to
turn that ``feature'' off, or decide that there's a better alternative.

However, if email really isn't a significant load on the system, then
it's a moot point, right?


#77 of 203 by keesan on Sat Mar 31 02:26:26 2001:

I like Pine.  What is MH?  We have signed up lots of people to dial in to grex
and use email.  They have never used a computer before and find the
conferencing too confusing, but can handle Pine.


#78 of 203 by gelinas on Sat Mar 31 02:57:46 2001:

Dan, I read your suggestion in 64 to mean that people who just want to read
e-mail shouldn't do it here.  "Suggesting" a different tool, and then making
the preferred tool "inconvenient" is well down the road towards 'requiring'
them to use the tool you prefer they use.

And "form-based input" is a perversion of HTML.  HTML is a *markup* (hence,
display, i.e. output) tool.

Sindi, MH is the "Rand Message Handler", a collection of single-purpose
tools for converting e-mail messages into individual files for reading,
filing, deleting, etc.  (And single files aren't always one message, but
usually.)  Since each message is a separate file, all the standard Unix
file-manipulation tools work.  It's my preferred MUA (Mail User Agent),
but I use Pine to delete messages I don't want to reply to or otherwise
waste diskspace on.


#79 of 203 by gull on Sat Mar 31 04:38:27 2001:

Re #78:
> And "form-based input" is a perversion of HTML.  HTML is a *markup*
> (hence, display, i.e. output) tool.

Yeah!  And email is a perversion of computers!  They're calculating 
machines, not chatting devices! ;>


#80 of 203 by tenser on Sat Mar 31 08:34:19 2001:

Regarding #78.  Well, you are of course free to interpret what I say any
way you like.  However, I'm not sure you are really giving my suggestion
fair consideration.  In particular, the delay and message can *both*
be turned off, simply by removing a file.  Only the default is the
``inconvenient'' mode.  If a person really decides that using mail on
grex is right for them, then they simply ``rm ~/.mail_only'' and carry
on as before.  However, an assurance is made that they are at least
aware of alternatives.

I *really* don't see how this is ``requiring'' anyone to do anything
(other than possibly removing a file), and it fits in with Grex's mission
statement.  It's a gentle hint that free email isn't grex's main purpose
in life, and that other services are probably more suitable.

Regarding your feelings about web sites/HTML.  All I can say is, ``that's
your opinion.''  It's a completely valid opinion, but I think that you
need to bear in mind that 90% of the rest of the world disagrees with
you, including many, many users who have switched from more traditional
MUAs to a web based interface.

Also, if you're going to take the stance that grex ought to help people
get email, then why not advocate for a POP server?  Or even an IMAP
server?  That way, people could use clients on their desktop machine's to
get their cyberspace.org email.  For the vast majority of users, something
like Eudora is far, far easier to use than elm, pine, mailx, etc.

To me, a web browser is a colorful equivalent of an IBM 3270 terminal;
get screenful of data at one time, transmit a screenful of data at one
time.  Let a block oriented I/O monster on the other end deal with it.
This model really works for some people (obviously you're not in that
category; I'm not either :-), so how can you really knock it on an
absolute scale?

btw- I'm still curious as to how much of a load email places on grex.
If it turns out that the answer is, ``not much....'' then it's hardly
worth arguing about.

(oh, ps....Don't get me started on what a mess HTML and HTTP are, but
let's be realistic; a lot of people use them and like them.  Who are we
to judge?)


#81 of 203 by scott on Sat Mar 31 13:03:19 2001:

Email puts a pretty big load on Grex.  However, offering free email is
basically part of Grex's charter, that of providing Internet access to people
who otherwise couldn't afford it.


#82 of 203 by mary on Sat Mar 31 13:23:34 2001:

Agree.

Dan, are you suggesting mail doesn't fit with Grex's mission?  If so, I
disagree.  I would hate to see us only support projects which keep members
of a Club Grex talking to each other in conferences.  Free email and
public access to Unix are wonderful services that we are proud to offer. 
Some day things may change and we may have no alternative buy to eliminate
mail, open Unix, or even dialups.  But that would be a real shame.



#83 of 203 by swa on Sat Mar 31 19:01:53 2001:

I agree wholeheartedly with #81 and 82.  I'm also fairly certain that
I *have* seen a message in newuser or someplace pointing out that there
are other free e-mail services available for those only looking for such
things.  My understanding is that Grex's raison d'etre is to foster
communication and sharing of ideas, and to provide a place for those new
to the Internet, to Unix, to computers in general, etc., to learn and ask
questions.  As I see it, e-mail fits quite neatly into both of these two
purposes.




#84 of 203 by tenser on Sat Mar 31 19:52:21 2001:

Regarding #82.  Mary, no, I'm *NOT* suggesting that mail doesn't fit in
with grex's mission (sorry to shout, but I want to be absolutely clear
about this, since I feel that people keep misinterpreting what I'm saying.
Probably my fault for not explaining it well enough).

It's just that it's my understanding that ``free email for the the
world'' isn't what grex is all about.  I got that understanding from
reading from the following ``staff notes'' web page:

        http://www.cyberspace.org/staffnote/about.html

which has this to say about the matter:

        ``Grex doesn't run a POP server. Providing mail service to
          the universe is not our primary goal, even if it is much of
          what we end up doing.''

And then goes on to say,

        ``We want people to get involved in our virtual community,
          so we want them actually get on the system to get their mail.''

Which I interpret as, ``we provide free email for people who are
interested in an online community, but we're not hotmail.'' If I'm
mistaken, please let me know.

All I'm really saying is that, for people who are *only* interested
in free email, and not participation or potential participation in
a community, there are other, perhaps even better, alternatives, and
that if email was using so many resources that it was making the system
unusable for people, it might make sense to encourage those people to
explore those alternatives.  (And that that's a big if; I've not seen
any quantitative data on it pointing one way or the other, though Scott
has indicated that it's a fairly big load.)

Never have I suggested that offering email isn't a good idea, or that
grex get rid of email (or Unix shell access; I'm not sure where that
came from?).  On the contrary, I think they're great services, and I'm
glad that grex provides them.

I'm only offering suggestions for how to reduce disk usage, doing so
under the assumption that mail is one of the largest consumers of disk
resources.


#85 of 203 by gull on Sun Apr 1 00:46:48 2001:

I don't think email is a big problem.  Lately our problem has been full 
disks, not a bogged-down system.  Grex has seemed acceptably fast to me 
pretty much all the time, lately.


#86 of 203 by russ on Sun Apr 1 04:08:33 2001:

Re #81:  Flogging free e-mail to people who can access *any* *other*
*email* *provider*, at the expense of other parts of Grex's mission,
probably isn't the best thing for us to do.  Hotmail and Yahoo and
Angelfire have the bandwidth to deal with ten thousand people shuffling
multi-megabyte attachments around all day.  We sure don't.  People who
want that and nothing more should definitely be directed elsewhere.
Anyone with a TCP/IP connection will be just as well off doing that.

Don't tell me that "we can do it".  There was a time not long ago when
Grex would shut down in the middle of the day to process mail backlogs.
Our "community" functions were being cut off to provide services to
people who had no interest in our community and gave no support.

Grex is best as a discussion forum and community, not a blind mail drop.
We can also provide mail service to people who do not have an ISP
(and thus do not come to Grex over TCP/IP).  There are a lot fewer of
those, but they are Grex's natural constituency.  Offering the rest
a little bit of encouragement to use more resource-laden providers
would make everyone's life better.

Speaking of logs, does ftp keep a log of what it transfers?  If all
we had to do to catch the worst disk hogs was to grep an ftp log
for .tar, .tgz, .gif, .jpg, .mp3 etc. and target them for early and
frequent cleanup, life would be a lot easier.


#87 of 203 by aruba on Sun Apr 1 05:26:34 2001:

Re #84: Don, I for one appreciate your attempt to address the problem this
item was created to talk about.  I don't think you have misinterpreted what
Grex's mission is or what our attitude is toward offering email.

Part of the thing is that we have gone over these policies many times, and
so lots of opinions have become calcified.  That's not to say it isn't good
to bring them out into the open.

My problem with your suggestion that we tell people about hotmail, etc. when
people run pine is that it assumes they are freeloaders before they've had
a chance to demonstrate otherwise.  If I were new to Grex, I'd find that
rather standoffish, and I wouldn't be inclined to join the community at that
point.  But maybe that's just me.

Some staff member please correct me if I'm wrong, but I think at the moment
it's the home directory disk that usually fills up, not the mail disk.  So
incoming mail is less of a problem right now than eggdroppings.


#88 of 203 by mary on Sun Apr 1 15:00:34 2001:

Calcified, eh?  Thanks a heap. ;-)

I don't use Grex mail as my primary email address.  But I'd be 
interested in hearing from folks that do depend on Grex for
mail service as to why they are here as opposed to, say, hotmail.com.


#89 of 203 by pfv on Sun Apr 1 15:57:50 2001:

I can think of several reasons for grex as a Primary Email, (if not grossly
mime):

        1) Local User on a Dialin
                (Been here, done that - God Bless);
        2) "In Search Of.. (A Decent ISP)"
                (same-same, all one need do is alter .forward);
        3) grex-user email (local email)
                (staying in contact with Those We Miss);
        4) "Email As a Persistent/Delayed-Delivery Telegram To A Grex-User"
                (this also allows for code-fragments and data-captures)


#90 of 203 by aruba on Sun Apr 1 16:21:50 2001:

I use Grex as my primary email because
1. I can dial in without using up my alloted minutes with my ISP,
2. Marcus's spam filters do a good job of limiting my spam intake, and
3. I think it's good for the Grex Treasury messages I send out to come from
   an address at cyberspace.org.


#91 of 203 by keesan on Sun Apr 1 16:24:33 2001:

I dial in and check email and the bbs.  I prefer PINE to other mail programs
as it is keyboard instead of mouse.  I use lynx as my browser because it gets
rid of the graphics and is fast, and Pine is the same phone call.


#92 of 203 by pfv on Sun Apr 1 18:08:35 2001:

        pine is a pain, and lynx wasn't even discussed. *sigh*


#93 of 203 by robh on Sun Apr 1 18:26:24 2001:

I still use Grex as my primary e-mail address - in addition to
the reasons mentioned by others,

* cyberspace.org is a very cool domain name

* I can access Grex through a modem/telnet connection from durn near
  any computer, without needing a PPP/SLIP connection.  Case in point,
  my PC's Internet programs all stopped working on Thursday night, and
  I haven't yet gotten the problem fixed - but my e-mail access
  was uninterrupted.  And if my PC had crashed completely, I could
  get here from my fiancee's computer with minimal effort.


#94 of 203 by i on Sun Apr 1 18:42:46 2001:

Grex is my primary e-mail address because...
 - I don't have TCP/IP net access here at home; grex is a local call
 (Neither my tight spending habits nor indifference to leisure-time
 web surfing incline me to go out and sign up with a much-pricier ISP.)
 - I'm here anyway for the community
 - Minimal junk-mail & no electronic diseases get through here
 - Cool e-mail address & warm feelings without the Big Brother marketing
 dept. or semi-competent staff issues of a commercial provider.


#95 of 203 by gull on Sun Apr 1 20:30:07 2001:

I use Grex for most of my email for several reasons:

- I've had this address longer than any other; many, many people know
  it.

- Grex doesn't seem to be going anywhere.  Free web-based email
  services go bankrupt at a rate of about one a month, it seems.

- I prefer a text-based interface to a web-based one.

- Grex is more reliable than most web-based email services.  The one
  I use for receiving files from people -- mailandnews.com -- just
  went down for two weeks with no warning.  Hotmail also has sporadic
  problems with being unable to send mail, according to people I know
  who use it.

- Unlike most free email services, Grex doesn't sell my name to mailing
  lists.

- Unlike hotmail, Grex doesn't have a reputation for being a place
  people use to hide their real identity.  There are mailing lists
  and such that no longer accept people with hotmail addresses for
  that reason.


#96 of 203 by tenser on Sun Apr 1 23:18:37 2001:

Regarding #87; Mark, thanks for letting me know that my interpretation
was correct.  I hear what you're saying about the problem my suggestion
has with possibly scaring off new users.  Hmmm....  I don't have a good
idea on how to address that.  Maybe the wording of the message could
be made as friendly as possible, or the verbage could be moved into
the user account creation process (someone mentioned that it might be
there already).  Then again, maybe it's best just to forget the idea
since it doesn't seem very popular.  :-)

btw- Regarding the user space disk versus the mail disk....  Programs like
elm, pine, and MH's inc all take mail out of the central spool directory
and place it into files in the user's home directory (typically into
``folders'' under the mail or Mail subdirectories).  So while the MTA
(eg, sendmail) dumps mail into /var/spool/mail it should be assumed that
at some point a good chunk of the data in that filesystem will migrate
itself over to /a and /c for ``permanent'' storage.

The df command tells me that there's about 1.1GB of data currently
in /var/spool/mail, but only about 300MB free between /a and /c.
Assuming that 70% of it will get deleted and thus not filed, there's
still not enough space on the user disks to hold mail + user files.  :-(

btw, here's another suggestion.  While there's only about 300 megs free
between the two existing user disks, there's somewhere on the order of
600 MB free on /var/spool/mail.  Most modern MTA's can be configured to
deliver mail directly to a user's home directory.  One way to get some
more usable space for user's might be to move mail delivery into the
user's home directory, and then reallocate the disk that /var/spool/mail
lives on to home directory space.

This would have the advantages of giving user's some more space, and
splitting the I/O load associated with both mail and user access over
more disks.  It would have the disadvantages of increasing the load
on user disks, and possibly affecting mail delivery if the user disk's
filled up (though there's no reason the mail disk couldn't fill up and
cause the same problem if enough people got enough mail in a short enough
time period).

Another suggestion would be to tune the filesystems to reserve less
space for cylinder group reallocations (ie, reduce the ``minfree''
parameter).  It looks like it's set to the default 10%, which is kind of
high considering the size of the user filesystems.  Setting it to 2% or
thereabouts would free up some space, and I don't think it would unduly
affect performance.  Taking the machine down into single user mode and
doing a ``tunefs -m 2 /dev/rsd..'' on a couple of key filesystems might
help some.

        - Dan C.



#97 of 203 by russ on Mon Apr 2 01:01:12 2001:

Re #87:  Is there any way to detect someone downloading eggdrop, etc.
other than by scanning people's directories?


#98 of 203 by tenser on Mon Apr 2 01:17:08 2001:

Regarding #97; Russ, it depends.  Back in the days of yore, I seem
to recall that certain FTP daemons (like WU) would allow you to use
a configuration file to do things like specify regular expressions of
files that you didn't want people uploading or downloading.  The problem
with those sorts of schemes is that it doesn't prevent someone from
sidestepping the them by just renaming the file when they transfer it.
Also, it doesn't stop them from using eg SSH or email for their transfers.

Then again, if someone is sharp enough to think that they can run an IRC
bot here, they're probably also too smart to know how to get around it.
:-)

Perhaps one of the grex staffers can comment further.  I'd be interested
to know if anyone has studied HOW copies of eggdrop etc get onto the
system.  Is it really via FTP most of the time?


#99 of 203 by gull on Mon Apr 2 02:13:43 2001:

I'd be in favor of a maximum file size on FTP PUT operations.  Many 
places now do this.  We have a limit on incoming mail attachments, why 
not on incoming FTP files as well?


#100 of 203 by cmcgee on Mon Apr 2 13:46:28 2001:

I use Grex as my primary email address because:

I don't have any other ISP

I can get to grex from anywhere in the US with a long distance phone call
for dialin access.  For the amount of time I am on email when I'm out of
town, a long distance call is much cheaper than any other access to the
'net that I've found.

I can get to grex from anywhere in the US with a long distance phone call.
I don't need any fancy internet access, and there are fewer technology
links to fail.

"cyberspace.org" is the neatest domain name I know.  Since the internet was
opened to everyone, and .edu is no longer high-status, .org reflects my
values.  cyberspace is simply easy to remember.

All the privacy and filter  stuff that other people said

The staff is the coolest bunch of people I've met in a long time.  They are
patient with me, and impatient with the nasties that sometimes get through
to my mailbox.  And they respond, fast!

Checking my email and checking bbs are close to the same thing.  I like
getting "messages" on bbs and staying up with what my friends are doing.  



#101 of 203 by gelinas on Mon Apr 2 14:55:37 2001:

I chose grex for my children's mail boxes simply because I don't like web
browser-based mail and because I don't want their e-mail on the machine
they happen to be using at the moment.  Yes, I have an IP connection, so
we telnet in to grex and so could use just about anything in the world.
But grex has the kind of facilities I want them to learn to use.  And its
affordable.

I know the advantages of POP and IMAP.  And their disadvantages.  As long
as we all have to share a computer, we'll stick with Pine (and MH for me)
on UNIX boxes.


#102 of 203 by jared on Mon Apr 2 16:27:52 2001:

What i've looked at doing (and am in the process of doing) on nether.net
is I have it mount a (the) users partition from a NFS server that is
x-over 100Mb/s FE to the machine.  I see low utilization on this nfs link
and allows me to set up a PC w/ 256M ram (memory is cheap these days :)
and a 80G ide disk.  This means that we have the users on "cheap" disk
that has this ~256M buffer inbetween the disk and everything else.
It works out quite well, not great I/O performance for the users who want
to suck down files as fast as possible to disk but you just set up
rpc.quotad and you have your quotas, etc.. It means you're not buying
a expensive scsi disk to do mass file storage (which is what any
home directory system is).  The disk needn't be fast, just reliable.

Plus you can also do a hot-backup or nightly copy to another 80G
ide in the box providing an online backup.  Reduces restore times quite
a lot.


#103 of 203 by mdw on Mon Apr 2 21:34:58 2001:

We've certainly considered mail-to-user-directories, but haven't done
it.  Here are some of the issues:
 (1) security - if mail spool files are directly in home directories
        there are more security issues to face, and more potential
        for bad stuff to happen.
 (2) separating mail from home directories means that one thing filling
        up doesn't break everything.
 (3) we'd like to back up home directories.  we don't want to back up mail.
 (4) having a separate mail directory facilitates moving into a
        distributed environment.  One possible direction would
        be similar to Umich: mail delivery & processing on
        a separate mail machine, home directories in AFS, and
        a pool of login machines that share access to the common
        resources.
Also, regarding NFS,
 (1) NFS is not secure.
 (2) mail and NFS don't play well together.


#104 of 203 by tenser on Tue Apr 3 00:04:59 2001:

Regarding #103; Marcus, thanks for the informative note.  However, as
usual, I have some questions.  Regarding your point 1, what security
problems are encountered by moving mail to the user's home directory?
It's been my experience that this generally makes a system more secure
by avoiding all of the problems with /var/{spool/,}mail and friends.
Indeed, qmail has adopted this approach by default in order to *avoid*
security problems.

Point (2) is definately a problem.  I'm not sure I agree with (3),
but mail does suck up a lot of tape feet....

Regarding point (4), I'm not sure how having a seperate mail area
facilitates moving to a distributed system, while mail in the home
directory doesn't.  Going with AFS opens a pretty hefty can of worms,
not the least of which is licensing (though OpenAFS might help with that).
On the other hand, if mail exists in the user's directory, it just sort
of automagically follows him/her around.

Also, recent versions of NFS *are* pretty secure.  Indeed, Solaris 8
includes code to authenticate NFS via Kerberos.  This probably puts
it on par with AFS.  Neither system, however, does encryption of file
data as far as I am aware.

Also, as long as NFS locking works (which it does in newer versions
of Solaris, though not really well in SunOS), it and mail are reasonably
good playmates.

Then again, Dan Bernstein, author of qmail, has the following to
say about NFS:  ``NFS is the most unreliable computing environment
ever invented''.


#105 of 203 by scott on Tue Apr 3 00:09:05 2001:

I can answer point (2).  We don't want to back up mail for legal/social
reasons, mainly because we don't want to be on the ugly end of a "subpoeana
all his/her email activity" demand.  Plus inboxes are often the places that
a failed (created the account then disappeared) user fill up most.


#106 of 203 by scg on Tue Apr 3 00:15:25 2001:

That's not the reason for not backing up mail.  Mail is transient enough that
backing it up every few weeks won't really do anything useful.

This does, I suppose, make a user's old mail harder to subpoena, but that's
a side effect.  We've always been pretty clear that Grex is not a system for
use for illegal purposes.


#107 of 203 by tenser on Tue Apr 3 00:21:34 2001:

Regarding #105; Thanks Scott, that makes a lot of sense.  Hmmm....

Regarding #103; Jared, that model (single 80GB IDE disk) will work,
but it won't scale.  In particular, you end up stuffing close to 80
gigs of data onto one single drive, which then gets absolutely
hammered.  The seek time alone will kill you if you have more than
a modest load....  It's much better to use a set of disks, and
spread the rotational latency and seek time across a set of spindles.

Also, even if you put multiple ``cheap'' IDE disks in a machine (only
two per controller, though), you end up with another problem: the IDE
controller can't do overlapping seeks (ie, I can't tell one drive to
seek somewhere and then tell the other drive to seek somewhere while
the first is still seeking), which means that access to the bus is
effectively serialized.  At least, this is how it used to work.
Obviously, this will make the solution rather slow as time goes by
(especially on a timesharing system).

The reason for going with SCSI isn't so much that it sounds cool, but
that it does away with these limitations, leading to a much higher
performance I/O subsystem.

Then again, maybe some of these issues have been addressed in more
recent versions of the IDE standard.  Somehow, I kind of doubt it,
though.  Think of IDE as the SunOS 4 SMP of disk architectures.  ;-)


#108 of 203 by gelinas on Tue Apr 3 01:40:25 2001:

File permissions on /var/spool are controlled by people who actually
understand unix file permissions.  File permissions on user directories
are controlled by people who quite likely can't even define the term, much
less manage them.  Dropping mail into a user's directory can be embarrassing
at best.


#109 of 203 by tenser on Tue Apr 3 02:24:20 2001:

Regarding #108; Okay, so what exactly prevents me from doing `chmod a+rw'
to my maildrop in /var/spool/mail?  What about if I do it, ``because
someone told me to....''  (``But he said he was Grex staff and he was
trying to fix a problem with mail!  His login was mbw, that's Marcus
Watts, right?'')

On the other hand, what prevents me, a malicious user, from attempting
all sorts of race condition attacks against /var/spool/mail.  Or from
attacking a whole slew of newly setgid or setuid programs, including
many MUA's.  Suppose I find a race in temp file creation in a program
which was originally intended to be run as the user, but must now run as
root to create a file in /var/spool/mail.  The list of potential problems
with /var/mail and friends goes on, as does the list of problems which
have been realized over the years.

Grex has taken steps to prevent much of this from happening, but I don't
think the measures are perfect, if for no other reason than the staff
must expend large amounts of time caring for the changes anytime they
want to upgrade almost any part of the mail system.

What's more, a very well respected (err, sorta [*]) paranoid security
freak has done a thorough analysis of the problem and determined
that delivering mail into $HOME isn't that bad, and is preferable to
the alternative.  Delivery into $HOME is thus the default for qmail.
My own study of the problem over several years leads me to agree with
him.  Perhaps you'd like to tell Dan Bernstein that his decision is
``embarassing at best''?

One point I'd like to make is that discussions of arbitrary user screwups
in setting permissions on directories, files, and maildrops is misguided.
No technical solution can be found to problems that involve inadequate
user education, especially in the area of social engineering.  Users who
don't even know who Unix is to ask him for permissions to read a file
aren't likely to just play around with the chmod command for no reason,
and thus aren't likely to mess anything up.

And I've seen a few professional sysadmins screw up permissions in
various places in a big way.  Sometimes decentralizing something (like
mail) can significantly reduce the impact of such mistakes, resulting
in a net increase in overall system security.

As a final aside, please note that most MUA's move mail out of /var/mail
or /var/spool/mail and into folders in the user's home directory.
In particular, you've mentioned that you use MH elsewhere in this
discussion.  MH's inc and rcvstore by default place mail into folders
which live beneath $HOME/Mail.  Has this caused a problem in the past?
If not, then I see very little reason to assume apriori that moving
the MTA's mail delivery into $HOME will cause a problem in the future,
especially since other sites have been doing this for years now.
In particular, the Unix permissions argument doesn't hold up, since
folders would suffer from the same problem.

-- 
[*] DJB is repsected in terms of his ability to write secure software.
His ability to win friends and influence people is considerably less
respected, however.  It's telling, however, that he's willing to put
some of his own money on the line to prove his point; he's willing to
pay a cash reward to anyone who finds a security bug in qmail.


#110 of 203 by jared on Tue Apr 3 03:25:06 2001:

Marcus, when was the last time you used nfs?  v3 or v4?  I suspect you
are missing out on recent updates.


#111 of 203 by gelinas on Tue Apr 3 04:47:13 2001:

The 'default' umask most users get saddled with is '022': everyone gets to
read any file created.  Until the user learns enough to change the umask,
they have no privacy.  Since MH honors umask, yup, it's a problem.

Yes, 'social engineering' is a problem.  At least the 'engineer' has to
introduce certain concepts to compromise /var/spool directories; $HOME is
compromised by default.


#112 of 203 by tenser on Tue Apr 3 17:35:30 2001:

Regarding #111; Actually, MH ignores umask (or, rather, resets it
to something ``safe'') when creating directories.  Have a look at
sbr/makedir.c in the NMH sources.  Note that at least inc makes use of
this when creating folders (see uip/inc.c).

So, even if files were created by MH with world read permission, (and
note that they're chmod'ed to 0600 anyway) the fact that the directory
they're in is mode 0700 means that no one can read them other than the
owner of the directory (and root).  About the only files in MH that
I know of which default to listening to the umask are .mh_profile, and
the global context file (the latter lives under a protected directory
anyway, though, and the default .mh_profile is hardly earth shatteringly
important enough to keep secret).

If that weren't the case, this would have been a problem for years on
grex and many other systems.  However, this has not been a problem for
years on any other system that I know of.  In fact, I've seen MH *not*
work if it felt that permissions on certain files were *too* lax (this
happened to a friend of mine once whose .mh_profile was group writable.
Certain MH commands flat out refused to work).

Regarding social engineering, I'd like to know what ``concepts'' an
attacker would have to introduce to make use of a social engineering
attack in /var/mail?  The only one I can think of was the one introduced
by PT Barnum nearly a century ago, that of: ``There's a sucker born
every minute.''

Also, I fail to see how $HOME is ``compromised'' by default in any way
while /var/mail isn't.

Finally, whatever any MUA (such as MH) does when creating files is
independent of what an MTA would do if configured to deliver into a
user's directory.


#113 of 203 by scg on Tue Apr 3 19:03:17 2001:

The Grex staff gets tons of mail from people who have messed with their home
directory permissions and locked their accounts.  We don't seem to get this
sort of complaint about /var/spool/mail much.  I don't know why.


#114 of 203 by tenser on Tue Apr 3 19:17:48 2001:

Regarding #113; Interesting.  btw- how do you define ``ton'' in this
context?  Typing random octal numbers to chmod can definately do weird
things to one's permissions.  I think that's independent of the
security of any maildrop scheme, though.

Hey, some quantitative data on what staff sees most frequently might
be really interesting.  Can someone post any here?


#115 of 203 by remmers on Tue Apr 3 21:24:04 2001:

Staff is too busy answering the messages to count them.  :)


#116 of 203 by tenser on Tue Apr 3 21:41:07 2001:

Regarding #115.  Heh, what you mean you don't have a helpdesk system
like RT set up?  :-)


#117 of 203 by scg on Tue Apr 3 22:58:10 2001:

Nope.

Tons was probably an overstatement.  I haven't been paying much attention to
staff mail for the last few years, beyond skimming the ones with interesting
subject lines, but my guess is it's probably not more than one or two a week,
if that.  Still, that's many more people than screw up the permissions on
/var/spool/mail.

This is generally a matter of people trying to lock down the privacy of their
home directory, and locking it down so far that they can't even get in
themselves.


#118 of 203 by russ on Wed Apr 4 00:16:18 2001:

Re #98:  Banning certain filenames just provokes countermeasures.  What
I'd like to see is a log which can be checked later, to clean up after
eggdroppers ASAP.  Scanning mail for uuencode headers of such files might
also be useful for marking accounts for automatic scrutiny and cleanup.

Idea:  Keep copies of eggdrop &c on-line, perhaps in CVS.  Tell people
where they are, and that they won't work here.  Follow up after the CVS
logs every few hours with a daemon and delete the files and binaries.
This should keep the total inventory down to a few copies at most.


#119 of 203 by gull on Wed Apr 4 01:48:13 2001:

Re #118: Nah.  When they tried the CVS version and it didn't work, 
they'd just assume we'd done something to it to break it, and download 
their own 'clean' source to try.


#120 of 203 by mdw on Wed Apr 4 08:56:44 2001:

Putting mail in home directories creates mail race conditions, with
people renaming things.  Sure, there are ways around it, but it's more
of a problem than when mailboxes live in a directory people can't munge.
It's not an unsolvable problem, and indeed, we're still living with some
of these security concerns, because we honor .forward's in people's home
directories.

Perhaps I should describe one, just to make people sweat: originally,
grex used the actual sunos mail programs, and even after we mostly
switched away, we were saddled with one remainder, /usr/ucb/mail.
/usr/ucb/mail, for ugly reasons, required write perms on
/usr/spool/mail, which meant /usr/spool/mail perms were in fact set like
/tmp--the sticky text bit was set.  This meant user A couldn't mess with
user B's file if it already existed, which was fine.  However, user A
could *link* his mailbox to user B's file, if it *didn't* exist.  This
was not good, but livable.  We lived with it for a while.  Every so
often, weird stuff would happen because people would discover this, and
try to exploit it.  When we moved to hierarchal mail files, we finally
got around to fixing /usr/ucb/mail, and got rid of write perms.  End of
problem.  Now, if we move back to putting mail in people's home
directories, people can now create accounts, and do a hard link to each
other's mail files.  This can be fixed, sort of, but I'm not sure it can
be fixed in such a way that there aren't race conditions.  Perhaps they
can be, and perhaps the qmail author has fixed them, but I, for one,
sure didn't want to think about them hard enough to be *sure* the result
was right.

So far as NFS/AFS goes.  I'm not sure I want to spend a lot of time on
this, but yes, I know NFS has been advancing.  Indeed, I know some of
the folks (at CITI) who are working on this, and I know NFS is evolving
in the direction of AFS--including much of its functionality, though not
always in the same way.  There's an interesting conflict here though,
which is "secure by default" vs. "compatible with the past".  We
obviously want "secure", but we'd have to be very careful to be sure
it's secure enough, and to avoid the temptation to relax something
because we got some legacy product (a terminal controller or backup
device) that required some insecure mode to be turned on in order to use
NFS.  AFS supports integrity checking (by default) and will encrypt
things over the wire (if requested).  There are definitely features in
AFS that are not yet in NFS (pt groups, global name space), although NFS
is catching up on caching and (maybe) security.

An issue with both NFS and AFS is file locking and reliability.  This
becomes a major problem with mail, because these are both crucial to
reliable mail operation.  It is hard to do file locking right with any
sort of network based scheme, because you've now introduced network
delays and retries, which can slow things down a great deal.  I don't
believe NFS originally supported locks at all (granted, this now purely
historical); modern versions support file locking, iff the server
doesn't crash...  AFS isn't really much better about locking, although
since it's never tried to pretend file transactions are "stateless" it's
probably at least architecturally more "honest".  Reliability gets to be
an interesting problem - what do you do when you're delivering mail, and
the file server goes down? Do you just drop it?  Requeue it?  How do you
know that the mail wasn't in fact stored before the server crashed and
you just didn't get the notification?

NFS and AFS file semantics are naturally going to be different than UFS,
so that means the mail delivery client has to know about these changes.
With AFS, for instance, files get flushed to the server upon fsync or
close.  With NFS, this is implementation dependent and depends on on the
caching strategy - I believe some modern implementations don't give you
any control at all over when things get flushed, because the API doesn't
permit it.

Security is another messy area.  With any secure network filesystem
(where "secure" = secured by kerberos), there is no such thing as SUID
perms, and "root" becomes an interesting problem in definitions.  One
solution is mail is delivered by "the mail guy" and you give "the mail
guy" read/write perms on your mailbox, if you want mail to work.
Another would be to forge kerberos credentials -- doable, but real
scary.  Now, there *are* sites, some of them quite big that have mail
working acceptably via NFS or AFS.  There are plenty of other sites that
have had scary results trying to do this.

There is one other point to consider in all of this; grex is
*different*.  With virtually any large institution, users can be
classified into one of two groups, "us" and "them".  Any of "us" that
misbehaves badly enough can be turned into one of "them".  You're
forkbombing everytime the campus president logs in?  Off with your head.
You filled up the disk? You get a huge charge in your next student bill,
for all the file space you're "using".  This leads naturally to the
"firewall" mentality, which says, put all of "us" inside the magic fire
ring, and don't let any of "them" cross the ring.  Doesn't work with
grex.  Any of "them" just has to take out a new account.  Users have to
be presumed hostile, even if almost always innocent as lambs. This has
interesting implications for NFS.


#121 of 203 by jared on Fri Apr 6 00:00:25 2001:

(My suggestion is only meant to get "cheap" decent disk attached to grex.
despite the performance losses of IDE disk, there are not a lot
of applications (Except for mail) that are very I/O intensive and
demand performance out of the disk that grex needs).

I'm not for providing an insecure NFS/AFS/"Networked File System"
but the lack of hardware support in the current playform for the IDE
disks makes it more complicated/dificult.

One could purchase one of these new "SunBlade" workstations
($1k list and takes IDE disk which can be obtained.. and
at the price of memory these days $88 for a 256M dimm) and turn it
into the "next grex" and have ditch all the present disk.  For
a complete investment of $3k (memory, disk, system) there could
be 100G+ of disk and ~1G of ram online in the system.

(here's the url for the sunblade system:
http://store.sun.com/webconfig/BuildConfig.jhtml;$sessionid$WHDHN2IAAAS3BAM
TA1ESQ1T5AAAACJ1K)

Someone w/ campus/student afiliation can get the solaris-8 src
and modify the kernel for the tcp/ip restrictions and we can be
off and running w/ more disk, faster system, etc.. :)


#122 of 203 by mdw on Fri Apr 6 01:29:21 2001:

I believe IDE still doesn't support disconnected disk seeks, doesn't
support more than 2 disks per controller, doesn't support long cables,
and most IDE controllers are built into a motherboard or cards are
available for just a few bus architectures (ie, not sbus, not vmebus,
which is what we could use today).  Modern IDE drives (with the right
controller) do support DMA and linear block addressing, which at least
removes two of the most serious traditional faults of IDE.  That makes
IDE competive performance-wise with SCSI for low end systems.

For better or worse, grex does a fair bit of disk I/O - enough that disk
is at times the performance bottleneck, and enough to shorten the lives
of even fairly rugged disks.  We have a fairly heavy mail load, pine is
a pig, conferencing and in particular backtalk regularly appear in the
process logs, and of course there are the endless parade of stupid
wannabe eggbotters.  We also have a pile of cheap SCSI hardware,
including drives and controllers, which would do just fine on grex.  The
only real bottleneck here is staff time and system downtime.

Putting everything on one spindle of a drive intended for a low duty
cycle operating environment, losing the expansion possibilities, and
incidently throwing out the rest of our hardware & software environment,
does not sound like a good solution for reliability, stability, or
performance.  I suppose, of course, that if we grew sufficiently
unreliable, the resulting drop in user demand would drop to meet the
performance.


#123 of 203 by valerie on Fri Apr 6 20:20:08 2001:

This response has been erased.



#124 of 203 by pfv on Fri Apr 6 20:26:43 2001:

        I sympathize, Val.. I really do.. You'll not GET "a solution" and
        the way things are arranged, no one that WOULD help you COULD help
        you - even tentatively. Between calcification and perms, a potential
        solution can't even be tested.

        I'm sorry, I wish I could help.


#125 of 203 by i on Fri Apr 6 22:42:37 2001:

Was there any plausable way to implement my very-limited-newbies/not-so-
limited-old-timers idea (back around #51)?  (Did it have enought basis in
fact to be really usable anyway?) 

If we could get a fair fraction of a cure by putting a few more of our 
2GB drives on line, and the too-limited time of the few staffers who grok 
the hardware well enough to do that is the problem, could we organize a 
volunteer drive to mow the lawns, watch the kids, wash the cars, etc. of 
those staffers while they get the extra drives on line?

If Technical Solution X would sufficiently cure the problem, but a lot of
folks are highly opposed to X, do we need to go to a segregated system 
where users pick between partition /X_but_usable and /non-X_often_full
and live with their choice? 


#126 of 203 by tenser on Sat Apr 7 20:24:01 2001:

Regarding #120; Sorry I haven't had time to respond recently; it's that
great American passtime of chasing the all-mighty dollar.  :-)

I don't understand your example using hard links.

In particular, it's easy to see how the *original* problem worked
(ie, I have two users, mary and joe; joe sees that Mary's maildrop in
/var/spool/mail doesn't exist and so does ( cd /var/spool/mail && ln joe
mary ), causing all sorts of merry problems [pun intended]).  However,
when moving mail to the user's home directory, this attack no longer
works because joe *cannot* write to mary's directory to create a hard
link to his maildrop.  Sure, he can hardlink *her* file to his own, if
her's already exists, but what does that get him?  His mail goes to her?
I'm not sure that would even work....I don't know for sure, but I'm
willing to bet that even sendmail isn't stupid enough to deliver into
a file if it's neither owned by the intended user nor owned by root.
He certainly can't read it, since the permissions in the inode don't
give him that access (recall that two files that are ``hard linked'' to
each other are just two directory entries that point to the same inode,
and that permissions are stored in the inode).

So, either Joe's email would go to mary (and what would the point of
that be?  The only malicious thing I can think of would be a denial
of service attack, but there are other ways of doing the same thing),
or Joe's email would just bounce (again, only really good for denial
of service, this time against grex itself, but there are other ways of
doing the same thing).

Regarding NFS vs. AFS....  I was not aware that AFS does encryption;
does OpenAFS?  Can you point me to a documentation reference so that I can
turn it on?  :-)  Certainly, there are features in AFS that don't exist
in NFS (in particular, the AFS caching model remains superior to that of
NFS) but then they're not the same product, and there are ``features''
in NFS that aren't in AFS (paying attention to the Unix permission bits,
for instance).

You bring up good points regarding locking, semantics, and legacy
support.  Yes, any network filesystem must have different semantics
than the ``normal'' Unix file model presents.  It's not clear that one
must teach *every* program how to deal with that, though, especially if
compatability hooks are provided (eg, an fsync(2) call that ``does the
right thing'').  Indeed, even under Unix, just beause write(2) returns
doesn't necessarily mean that my data is on disk.  I must still fsync(2)
or close(2), and applications should already be written assuming as much.
The jump from that to a network filesystem (as long as it has decent
locking) isn't really that big for most code.

Regarding reliability for mail and network filesystems, it would seem
that on a system like grex, the best thing to do would be deliver
mail onto the file server.  If using NFS, you end up dealing with the
traditional FFS semantics for reliability by doing so.  It's worth
nothing that mail is weird; it's really something that needs ACID style
transactional semantics, but the Unix file model isn't really robust
enough to provide that.  Certainly, I don't know any DFS's that are.

Yes, the concepts of root and setuid become particularly hairy when
extended over the network, but both concepts are flawed.  root was an
architectural necessity at the time Unix was written, and setuid was
a workaround for limitations in the overall Unix model (in particular
the lack of per-process file namespace manipulation and a complete form
of IPC; half-duplex pipes aren't really good for much other than simple
coroutine sequences).  It's worth noting that in the follow on systems
that have been developed at Bell Labs, both concepts have been eliminated.

In the modern Unix model, we now have decent IPC and ways to work around
a lot of the root issues, and indeed, network filesystem access provides
hooks (all though of a rather limited nature) to weaken or eliminate
the concepts of root and setuid.

The only other thing I have to say is that I disagree with the idea that
grex is unique.  Grex represents the union of several things, which when
combined may be somewhat unique, but the discrete components are not.
In particular, the idea of untrusted access by possible miscreants is seen
by any university that uses Unix.  Many have been using networked Unix
systems in these environments for decades now.  Another thing is high I/O
requirements.  Most database servers running Oracle under Solaris see 10
times the load that grex sees with few problems.  Many of these also exist
in untrusted networked environments (consider a populare database-backed
web site).  I think it's a mistake not to leverage the wisdom that these
sites have gained over the course of years and apply it here.  Indeed,
both the user load and the I/O load are somewhat light in comparison to
many other sites which have successfully employeed network filesystems
for an untrusted userbase for years, with few problems.


#127 of 203 by mdw on Mon Apr 9 02:39:48 2001:

I haven't time to respond to all that just now, but one quick thing: I
*work* at a large university.  Trust me, universities *can* kick people
out for doing bad things with computers, and the "big stick" model works
very well there.  I suspect sites that use oracle in production don't
order bottom-of-the-line IDE drives.


#128 of 203 by tenser on Mon Apr 9 04:46:22 2001:

Regarding #127; Yes, but I used to work for a large university as well.
(In fact, a football rival of yours, but I digress....).  Yes, you can
kick some students out, but not all of them, and it's better to assume
evil rather than good in such a setting.  I certainly know that it's
easier to do so than to clean up after some overzealous undergraduate
trying to impress the ``cute girl'' in CS 101.

Is grex ordering bottom of the line IDE drives now?  I never suggested
such a thing, though someone else did.  In fact, I argued against it.
btw- you'd be surprised what some folks run Oracle on in production....
Hmm, on second thought, perhaps ``mortified'' would be a better word.
But the point is that it's just not accurate to assume apriori that grex
is in some way unique with respect to load or user behavior profile.


#129 of 203 by mdw on Mon Apr 9 04:58:01 2001:

I see you aren't done sweating.  Yes, it's true, *in theory* you
shouldn't be able to hardlink *to* other people's directories.  "Should"
is an important weasel word there; there are people who carelessly
permit their home directory 775 or even 777, and there is the
ever-present trojan horse (granted, there are other more obvious
problems with trojan horses.) With symbolic links, it's possible to have
more "fun"; exactly what depends on how other things are implemented.
For instance, does the mail delivery agent run as root?  Or as the user?
In what order and using what calls are the file mode/perms checked?
There isn't an atomic way to open a file and avoid symbolic links in
Unix, so whatever set of calls are used, there's probably some trickery
that can be done.  Also, I never said it was impossible to do all this
right, just difficult.  The real problem is what we have here is a
"complex" problem, and in the security world, "complex" is a bad word.


#130 of 203 by tenser on Mon Apr 9 05:44:53 2001:

Regarding #129; if I chmod my directory 777 or even 775, I have far
bigger problems than someone hard or symbolic linking my mail spool file
somewhere else.  The attack you describe requires this as a prerequisite
for working, but at this point, I'm already compromised.

No, I'm not done sweating, but then, I never really began sweating.
That's because extremely qualified people have spent quite a bit of time
analysing this problem already, and they have done the sweating for me.
Those people have determined that it's perfectly safe to deliver into
$HOME, and indeed, that doing so is safer than delivering into /var/mail.
Some of them even offer cash rewards to those who can prove otherwise.
If that's not intellectual honesty, I don't know what is.

On the other hand, vague rumblings about hard links and soft links and the
ordering of system calls are far less convincing, since the techniques
for doing all this securely have been well understood for a decade.
These remarks are especially unconvincing if they are predicated
upon users doing something which completely compromises themselves
as a prerequisite.  They become nearly inconsequential when put into
the context of, ``all the hard work has already been done for you.''

Yes, email is complex because of all the security domains it crosses,
but I really don't understand why you seem to continue implying that it
hasn't been done already, and with a demonstrated good security track
record, to boot....

btw- the point of going to home directory delivery is to avoid running
things like delivery agents as root, in addition to freeing up space.

One other thing I'd like to point out is that the large, monolithic,
setuid root program that grex currently uses to deliver mail has been
demonstrated over the course of many years to have a far worse track
record with respect to security than almost any other commonly used
software package on the net.  It seems to me that, in the interest of
security, one ought to be seeking to replace that as soon as possible
instead of worrying about largely theoretical attacks against already
compromised users.

http://www.qmail.org/
http://www.postfix.org/

(btw- Why do I feel like I'm on USENET all of a sudden?  :-)


#131 of 203 by mdw on Mon Apr 9 05:56:08 2001:

Actually, grex doesn't use sendmail to deliver mail.


#132 of 203 by tenser on Mon Apr 9 16:00:21 2001:

No, true, sendmail isn't what actually writes to the user's mailbox,
that's another horribly insecure piece of code (unless you've really
hacked mail.local).  But then I meant ``deliver mail'' in the all
emcompassing sense.

But semantic nits doesn't mitigate the security problems that running
sendmail presents.


#133 of 203 by gull on Mon Apr 9 17:37:06 2001:

This debate is interesting, particularly to someone like me who's 
fascinated by security issues.  But it doesn't really address the 
problem.

Since quotas, automatic clean-up programs, and limits on file transfer 
sizes have all been rejected as solutions, I'm going to suggest that we 
simply reduce the amount of time an account can be idle before it's 
reaped.  Since most eggdroppers probably give up on Grex after realizing 
eggdrop won't work here, this would reduce the amount of disk space 
consumed by causing their accounts and the associated files to be 
removed more quickly.

I'm not sure what the reap time is currently, but 60 days with no 
activity is pretty common on free web sites.  If you're concerned about 
inadvertantly reaping college students during summer vacations, etc., 
there are two options:  Allow people to request that they be exempted 
from reaping, or make the reap time longer for people who have logged in 
more than, say, 10 or 20 times.  I'm guessing few eggdroppers stick 
around that long, though it'd be interesting to research that.  Maybe 
pfv could check, since he seems to enjoy locating this stuff.

I'm less concerned about things like stored mail.  Those are reasonable 
uses for the system.  If we get rid of most of the abuse problem, and we 
still have space problems, then it's time to add more disk.  But we need 
a better way to manage the abuse problem than staff manually going 
through the disk.


#134 of 203 by scott on Mon Apr 9 18:17:51 2001:

Reap time is something like one week if an account is only ever logged into
once.


#135 of 203 by steve on Mon Apr 9 23:38:01 2001:

   Reap time is after 90 days of inactivity, OR if the account was used 
only in the first 24 hours after its creation, it can be reaped after 21
days.


#136 of 203 by mdw on Tue Apr 10 05:07:36 2001:

We really hacked mail.local.  Actually, *I* really hacked mail.local,
hence my lack of enthusiasm for changing horses.  If we were doing this
"from scratch" today, it's very likely sendmail wouldn't be our first
choice.  Back when we switched *to* sendmail, it really was the best
choice (um, you do know why we switched *to* sendmail, don't you?)  At
this point, sendmail (now sadly multilated from its original form) seems
to be serving people well enough functionally speaking.  I think if
nothing else, we can agree that switching from sendmail to something
else is not going to help the home directory disk usage problem in the
slightest, which means it's not really relevant to the topic at hand
here.

I had a look at the kernel config.  So far as I can tell, we'd need to
build a new kernel that knows about a 2nd controller, but the controller
should then be just a drop-in solution.  We might be able to install up
to 4 controllers, if we wanted.  I'm not sure whether we have any
cabling issues, but I believe we have enough spare disk enclosures to
have more drives (even 3.5" drives, if we had any.)


#137 of 203 by tenser on Tue Apr 10 23:03:23 2001:

I assumed you switched to sendmail to get rid of the abomination that
was sendmail.mx.  Ahh yes, one still sees it on the disk, all though
chmod'ed to 400 now.  I myself escaped on at least a couple of systems
via the Zmailer, smail, or MMDF routes.  The latter being, as Henry
Spencer might put it, ``a particularly revolting invention of satan.''
(Actually, I never throught it was that bad, but it was weird.)

I disagree that you wouldn't save any disk space, as you could put
mail drops in user home directories at that point (at the expense of
many wasted feet of backup tape), which is what the whole argument was
about, anyway.  But I, too, am tired of discussing it, so it's probably
best to just agree to disagree.  :-)


#138 of 203 by i on Wed Apr 11 00:21:38 2001:

Ignoring for the moment the really-cool-to-read technical discussions,
are there any semi-decent partial solutions to the original problem in
the offing?  If so, what are they?  What's between us and having them
implemented?


#139 of 203 by tenser on Wed Apr 11 04:11:32 2001:

Regarding #128; Walter, it seems that the problem is creating a net big
enough to let the legitimate users come in and feel comfortable, but
small enough to catch the IRC weenies, thus encouraging them to go away.
It seems that a lot rides on stopping the incoming flood of, eg, eggdrop,
but we really don't know.

Let me ask this.  Valerie, you said you spent all day the other day
cleaning up grex's disks....  What did you actually delete?  Knowing that
gives us the basis for figuring out where to best target our resources.
Also, does anyone know how this stuff actually gets on the system?  Ie,
is it via FTP, email, SSH, or something else?  Do people frequetly use
lynx to download eggdrop or other goodies?  (Don't worry, I'm not going
to suggest making lynx go away...  :-)

If a lot of the problem is junk coming in via FTP, then perhaps one thing
to do would be modify the FTP daemon so that users can't transfer files
larger than a certain size (say, 30-50KB) on the first day that they
have an account.  *After* the first or second day, the limit disappears.

The idea here is that legitimate new users have enough access so that
they can bring over personalized dot files or programs they've written
or what have you, but eggdrop and friends is too big to fit through
the hole.  For legitimate users, they'd probably never even notice.
One the other hand, the one-off IRC weenie looking for an easy kill will
get discouraged and split, probably forever, but without leaving behind
large eggdroppings.


#140 of 203 by mdw on Wed Apr 11 08:02:02 2001:

Actually, we went from sendmail, to smail, *back* to sendmail.  smail
turns out to have really *bad* behavior in the face of congestion, and
was generally less efficient.  Evidently most places are used to a
considerably more favorable machine/user ratio.  Sendmail, with its
early production history on VAX-11/750's and other old big slow
expensive hardware, turned out to be better adapted to our needs.

While I wouldn't say that we've done a *lot* of experimentation with
eggdrop users, there is some evidence that these people know they are
doing something "bad", and that they're willing to try whatever it takes
to get around the annoying technical blocks.  I haven't looked at this
recently, but at one point, I had the impression that a lot of the
vandal tools we found on grex were due to eggdrop users who tried
running it here, found that it didn't work, and decided to try to break
root so they could "fix" the problem.  It's as if there were a FAQ out
there somewhere that included directions like "... find a free shell
provider.  Install eggbot.  If it doesn't work, become root and fix the
problem. ..." I should say the problem isn't *just* egg-drop; there were
a number of other "bot" packages (some of them sounding decidedly
non-friendly) that seemed to be just as popular.


#141 of 203 by russ on Thu Apr 12 02:13:58 2001:

Re #139:  Banning large files is definitely the wrong thing to do.
It's too easy to work around (use "split" and ftp the resulting
little files, uuencode them and send them by mail, etc.).  What it
would mostly do is make the operations harder to find and clean up.

Having ftp, ssh etc. *watch for* and *log* files over a certain size 
or having names matching certain patterns would be helpful.  You
could schedule automatic cleanups of 'bot programs, or trigger alarms
if vandal tools were detected.  (More subtle interventions, such as
carefully buggering the source of the vandal tools so that they'd
compile OK but core dump when they got to the right spot, are possible.
I doubt that anyone here has time to do that.  I sure don't.)

In the case of people using vandal tools, reporting them to their ISP
and mentioning that they violated our terms of service might be helpful.
Nothing like getting someone kicked off the 'net to keep them from
bothering us for a while.


#142 of 203 by aruba on Thu Apr 12 14:46:07 2001:

The idea of logging large transfers and then cleaning up after them later
appeals to me.  If we waited, say, a day after large transfers before
deleting the files, That would give the vandal time to find out that
eggdrop doesn't work and go away.  (It also gives legitimate users
leeway to transfer files for short periods, and delete them quickly.) 
I think Russ is correct that if we try to stop transfers as they happen,
then people will find a way around.

I also thought tenser's suggestion that we get some hard data on what
files are problems and how they arrive was a good one.


#143 of 203 by pfv on Thu Apr 12 15:22:22 2001:

re 142:
        Why not leave the hogs there only until the user leaves or gets
        disconnected? (Even 24 hours can be dangerous, when several of these
        folks decide to spend the day downloading/extracting/compiling,
        since it can fill the drive quite quickly).



#144 of 203 by aruba on Thu Apr 12 16:13:15 2001:

Maybe we could start at 24 hours, and then move the deadline back if we have
problems.


#145 of 203 by pfv on Thu Apr 12 19:27:31 2001:

        I agree, but I wondered about the technical problems, (remember I
        asked about 'logout' above?).

        What triggers what and HOW and WHEN seems to be the foremost issue.


#146 of 203 by jared on Sun Apr 15 18:33:20 2001:

Hmm.. 80G of disk for $234 including 2-day shipping.
nether.net is getting its disk upgrade this week :)


#147 of 203 by gull on Sun Apr 15 20:34:43 2001:

"I buy the smallest disks I can find, because I just don't believe you 
can reliably squeeze 80 gigs into such a small package."  -- the author 
of SpinRite, in a radio interview.


#148 of 203 by jp2 on Mon Apr 16 13:31:20 2001:

This response has been erased.



#149 of 203 by carson on Wed Apr 18 22:15:39 2001:

resp:148

(very much so.)  :^)


#150 of 203 by valerie on Fri Apr 20 00:08:04 2001:

This response has been erased.



#151 of 203 by mdw on Fri Apr 20 02:28:22 2001:

A fair number of people drag stuff over using Lynx or other web
browsers.  This is particularly popular for graphics, but it is also
common for software (it's easier to setup apache than ftpd), and of
course for vandal tools (httpd logs are generally not as useful.) I'd be
very surprised if even 50% of the bloat on grex comes via ftpd these
days.


#152 of 203 by gull on Fri Apr 20 02:48:27 2001:

<grin>  I can't help but think of this bit from "Know Your System 
Administrator -- A Field Guide":

--SITUATION: Low disk space. --

TECHNICAL THUG: Writes a suite of scripts to monitor disk usage, 
maintain a database of historic disk usage, predict
future disk usage via least squares regression analysis, identify users 
who are more than a standard deviation over the mean,
and send mail to the offending parties. Places script in cron. Disk 
usage does not change, since disk-hogs, by nature, either
ignore script-generated mail, or file it away in triplicate.

ADMINISTRATIVE FASCIST: Puts disk usage policy in motd. Uses disk 
quotas. Allows no exceptions, thus crippling
development work. Locks accounts that go over quota.

MANIAC:

# cd /home
# rm -rf `du -s * | sort -rn | head -1 | awk '{print $2}'`;

IDIOT:

# cd /home
# cat `du -s * | sort -rn | head -1 | awk '{ printf "%s/*\n", $2}'` | 
compress

(Courtesy of http://www.cs.bgu.ac.il/~omri/Humor/sysadmins.html)


#153 of 203 by gelinas on Fri Apr 20 04:47:28 2001:

I use ftp to get files OFF grex; I'd not appreciate losing access to it.


#154 of 203 by pfv on Fri Apr 20 13:09:56 2001:

Val - 
        I just went back about 100 posts, and I can't find anything about
        stats.. (I admit I'm having the first cup of coffee)..

        The last thing I found was the idea of a "logout cleaner".


#155 of 203 by gull on Fri Apr 20 15:01:36 2001:

I'd suggested making incoming FTP operations drop altogether after a 
certain number of kilobytes, instead of just slowing down.  If most 
files are now coming in via lynx, that wouldn't help, though.  (And I 
suppose if we did limit FTP PUTs, lynx use would probably jump.)

I really think the only permanent, reliable solution to this would be 
disk quotas.  It's a classic disk quota situation.  That may need to 
wait until we get newer hardware and/or a newer OS, though.


#156 of 203 by pfv on Fri Apr 20 16:27:13 2001:

The idea of "quota", (on a system that can't run "quota"), is why I still
suggest a root-managed global .logout script..

As they leave, it could run a nohup program, (owned by root), that simply
scans their space and proceeds to thermonuke oversize files or subdirs with
bot/irc/bnc crap.

I can't imagine that a background job like that is going to slow/inhibit
the disconnect of a user. And, the program itself can 'nice' itself.


#157 of 203 by jp2 on Fri Apr 20 16:56:18 2001:

This response has been erased.



#158 of 203 by gull on Sat Apr 21 03:34:37 2001:

Consider the limited time the Grex staff has.  Now consider the hours 
(nay, days) of labor it'd take to move all of Grex's custom software and 
custom network patches over to another platform.  It took months to move 
from the Sun 3 to the Sun 4, IIRC.


#159 of 203 by mdw on Thu Apr 26 16:51:30 2001:

NetBSD's kernel architecture is less advanced than SunOS: it doesn't yet
support even asymmetric multi-processing.  That means it won't make as
good a use of our current very cheap hardware.


#160 of 203 by jp2 on Thu Apr 26 17:43:01 2001:

This response has been erased.



#161 of 203 by mdw on Thu Apr 26 19:07:00 2001:

Well, considering this decade is only about 4 months old, that means
expensive *new* hardware at close to list prices.  To get something
roughly equivalent to the current hardware, we'd be looking at SCSI disk
and ECC (or at least parity) RAM, and a non-intel instruction set CPU.
There are a number of fine makers of such hardware, but it would be a
stretch to make our budget cover any of them.

Keep in mind when comparing to the 3AM "act now" TV offer computer that
what you see on TV probably comes with one spindle of IDE disk, no tape
backup, virtual parity RAM, intel instruction set CPU, a lot of
software, video hardware, etc., that we don't need, and is generally
optimized to perform well as a slightly flakey computer for *one* person
who is probaby not doing a lot of multi-tasking.  Grex probably *could*
afford that system, but spares would be another question.

IDE disk is cheap, but doesn't do overlapped seeks - not a big issue for
IDE since it only supports 2 disks at most.  Grex has a history of
expanding to fill its disks - no upgrade path seems pretty dubious to
me, especially given that the excuse for this whole project is so we can
store an infinite number of copies of eggbots and graphics.

Backups - I think it goes without saying that these are important.  We
currently have a very nice SCSI tape drive, less than 1 year old.  We'd
certainly want to continue to use that if at all possible.

"no parity" ram works fine when it works.  Of course, you'll never know
when it doesn't.  We've already lived through flakey memory on grex, but
at least we *knew* it was causing kernel panics.  Memory that was
flakey, but *doesn't* crash the system.  Do we really need it?

"Intel instruction set".  Don't get me wrong.  I have a lot of respect
for Intel.  Nevertheless, the pragmatic fact is (1) intel does not have
a pristine bug record, and (2) *every* vandal under the sun is trying
exploits using intel machine language.  Using almost anything else will
buy us at least 2 months of time to patch holes, and completely dissuade
80% of the vandals out there.  Also, if you price out intel hardware
with SCSI disks, sufficient ECC RAM, etc., you start looking at prices
that are pretty comparable with the non-intel server market.


#162 of 203 by jp2 on Fri Apr 27 02:09:52 2001:

This response has been erased.



#163 of 203 by tenser on Fri Apr 27 03:44:57 2001:

Actually, I offered to donate some Sun hardware to grex about a
month back, but, alas, I never heard back from anyone, and the
hardware is no longer available.  Specifically, I was offering a
Sun Ultra 2 with dual 170 MHz processors, 640 MB of RAM, and 40
GB of SCSI disk spread across two controllers.  I also offered
to throw in an Ultra 1 with 128MB of RAM and 22GB of disk, plus
a SPARCstation 5 and SPARCstation 2.  That would certainly have
mitigated some of these problems, and would have also provided
an upgrade path.  Plus, grex could have continued to use its
current tape drive.

I'm not sure why no one got back to me about any of this.  All
I asked was that grex pay for shipping from New York City.  Of
course, to effectively use the dual processor Ultra 2, grex
would have had to move to Solaris.  Perhaps that had something
to do with it.

Marcus, I'm not sure how you can rate SunOS 4 as more advanced
than, say, NetBSD, just because it lacks the hack multiprocessor
support of the former.  Consider that NetBSD has many features
that SunOS doesn't, such as large file and filesystem support, a
better VM system, soft metadata updates for UFS file systems
(which, IMHO, would be of huge benefit on grex), file flags, kernel
security levels, and more.  Not to mention active maintenance.  In
fact, I'm willing to bet that a single Sun Ultra 1/170 running
NetBSD would blow the doors off of the current Sun running SunOS
in all respects, would use less power, and only cost a few
hundred dollars.  I'll even help grex finance some or all of it,
if desired.


#164 of 203 by jp2 on Fri Apr 27 13:54:19 2001:

This response has been erased.



#165 of 203 by i on Sat Apr 28 01:25:58 2001:

Continued very interesting technical discussions.

New/fast/sexy hardware for grex would be nice to talk about.

It would take at least hundreds of hours of staff time - time that
doesn't seem to be available - to get grex working on any new hardware.

Ergo, i don't think that new hardware is a viable strategy for dealing
with the disks-perpetually-filling-with-crud problem.


#166 of 203 by tenser on Sat Apr 28 04:14:18 2001:

Regarding #165, it remains that grex is sitting on ``dead end'' hardware
and software.  Those hundreds of hours of staff time required to move to a
new system are partially a result of having to deal with that.  Having an
expandable hardware and software platform with which to work alleviates
a significant number of those problems.  For example, if running Solaris,
then moving from a Ultra 1 to an Ultra 2 is relatively easy.

What's more, I read statements in coop seemingly quite frequently wherein
staff members make mention of how they simply don't have hardware to
do this or that.  That's just not true.  The fact that I was prepared
to donate ~US$4,000 worth of gear to grex for the price of shipping
demonstrates otherwise.  Alas, this cannot happen now, since the hardware
just isn't available anymore.  :-(  (Though I'd still like to donate
something to grex!)

One of the problems with having such a ``customized'' system is just this;
it's impossible (or at least *really* hard) to upgrade.  The most common
accepted solution to that problem is to simply not customize the system
so much; that is, find ``standard'' solutions to problems and use them,
thereby shifting the burden onto someone else whose generally willing to
accept it and thus leverage economies of scale in the maintenance arena.

What's more, going with a newer system eliminates the need to do many
of the things which grex has to do (for instance, installing GNU versions
of most of the common utilities in order to work around SunOS
deficiencies).

It's a different way of thinking about the system maintenance problem.


#167 of 203 by gelinas on Sat Apr 28 04:22:36 2001:

(Right; they just have to install GNU versions because that's what most people
use.)


#168 of 203 by tenser on Sat Apr 28 04:26:21 2001:

Regarding #167; Oh really?  I've run a lot of Unix systems over the
years; I haven't installed GNU versions of df or ls in, oh, 6 years
maybe.  Then again, I haven't installed SunOS 4 in about that long,
either.

I don't know many users who ``expect'' GNU versions of most of the
basic utilities.  Now things like gcc on the other hand....


#169 of 203 by gelinas on Sat Apr 28 04:31:43 2001:

I don't remember using GNU versions of df or ls on SunOS 4 systems, either.


#170 of 203 by tenser on Sat Apr 28 04:43:16 2001:

Regarding #169; then you must have a very short memory:

: grex 48; ls --version
ls (GNU fileutils) 3.16
: grex 49; which ls
/usr/local/bin/ls
: grex 50; 

I suspect that if you were to type, ``which ls'' you'd see the
same thing.  I suppose I could be wrong, though.

/usr/local/bin/df seems to be a symbolic link to /usr/bin/df.  I
remember absolutely needing GNU df on AIX 4.1.3 in order to avoid
going crazy looking at poorly formatted information about NFS
mounted filesystems.  But things have progressed quite a bit in
the last 6 years.  Anyway, the fact that /usr/local/bin/df exists
is probably an indication that the GNU version was probably on
the disk at one point.


#171 of 203 by jp2 on Sat Apr 28 14:21:40 2001:

This response has been erased.



#172 of 203 by tenser on Sat Apr 28 16:14:25 2001:

Regarind #171; Errm, yeah, that to.  My point is that with a more
modern version of Unix, most of the functionality of the ``GNU
enhanced'' tools already exists in the base system; there's no need
to install (and maintain) a bunch of other stuff.


#173 of 203 by gull on Sat Apr 28 18:59:23 2001:

I usually install GNU tar, because 'tar xvzf filename' is much more
convenient than 'gunzip -c filename | tar xvf -'  And of course if I have to
install gunzip and GNU tar anyway, then I need GNU make and gcc to work
around whatever broken compiler the OS shipped with.  Can't say I've ever
had a burning desire to install GNU df or ls on any machine, though the
ability of ls to flag files and directories differently is really handy.  I
usually also install bash, because shells with no command recall or tab
completion really annoy me.

A lot of Grex's custom stuff has nothing to do with GNU, though.  There's
the custom hacks to limit outgoing Internet services.  And there's Picospan,
which we don't even have the source for to recompile it for a new
architecture, if my understanding is correct.


#174 of 203 by tenser on Sat Apr 28 20:33:03 2001:

Regarding #173; Note that NetBSD comes with virtually all of the
software you mentioned (GNU tar, gcc, gzip, etc) already included in
the base system.  I'm not sure why one would need gcc or GNU make to
install gzip; I just did it as a test using Sun's SunPro C compiler and
/usr/xpg4/bin/make under Solaris 7, and it passed all tests.  As an aside,
I'm astounded that the uncompressed shell archive which makes up GNU tar
is 4.7MB in size, which speaks to the poor implementation of the package.
As far as shells go, tcsh has both of the features you mention, and
ksh can be coerced into doing both command recall and file/command name
completion (I don't know if it uses tab for the latter).

But, I'm less concerned with the merit or dismerit of any particular
package from GNU, and more with the maintainability of the system as
a whole.  Right now, it's not too good, from what I can see.

Regarding the kernel blocks and picospan; again, much of this
functionality can be assumed by other packages.  For instance, with the
hardware I wanted to donate, one could easily make, say, the Ultra 2 be
``grex'' and disable all outgoing network connections (except for http
and DNS or whatever is intentionally left open) using Sun's built in
firewall software.  Then, set up, say, the SPARCstation 2 as a machine
which only staff and grex ``members'' can login to, and which has no such
restrictions on outgoing network traffic.  This eliminates the need for
making any modifications to the kernel, and the machine could double as
something else, too (for instance, a web server).

Similarly, the functionality of picospan could be taken over by another
package.  Say, an NNTP server (NOT part of the USENET in general)
running on the Ultra 1 I wanted to donate, or by the notesfile package.
The former would have the benefit of allowing, eg, grex and m-net to
share conferences.  party could be replaced by a private IRC server
and specialized client.  Orville write by the zephyr messaging system.
Note that at this point, most services are being replaced by ones
which extend over a network, allowing grex to scale by moving from
a single big computer to a cluster of smaller ones (this is called
``Client/Server Computing;'' perhaps you've heard of it?  :-).  The DBM
password database could be replaced by the builtin DBM password database
in NetBSD, or it could be replaced by a secure LDAP server under Solaris.
Authentication could be handled by a dedicated Kerberos server (like the
SPARCstation 5 I wanted to donate, which could also do backups via the
AMANDA package [if AMANDA, which can use Kerberos authentication and do
network encryption of backup data during the dump process, incidentally]
ran on the Kerberos server, one could still get network backups while
minimizing the exposure of the Kerberos server).  The password hashing
scheme currently used would be a problem, but there are ways around
that, too.

Note that doing this *would* require some changes in the way that the
system was used, but I think that in the long run the added functionality
and ease of maintenance would be well worth the short term discomfort.

The overriding principle in doing any of this would be to come up with a
system that's more easily supported than the current one, while providing
the same or analogous functionality as the current one.  I have no doubt
that this could be done, and I'm somewhat disappointed to see grex NOT
take this approach.  I fear that much of the reason why has less to do
with technical feasability, and more to do with NIH.  That'd be a shame.


#175 of 203 by jp2 on Sat Apr 28 21:29:17 2001:

This response has been erased.



#176 of 203 by tenser on Sat Apr 28 22:08:09 2001:

So is INN, c-news, the UIUC notesfiles implementation, and many, many
others.  What does picospan offer that these don't?  I'm really very
curious.


#177 of 203 by jp2 on Sat Apr 28 22:11:53 2001:

This response has been erased.



#178 of 203 by krj on Sat Apr 28 22:15:00 2001:

"Welcome to Grex, the Living Computer Museum!"  as I usually say in party.


#179 of 203 by tenser on Sat Apr 28 23:08:08 2001:

Regarding #177; so is the UIUC notesfiles implementation.  Also, INN
running on a fast machine will be inperceptably slower than picospan.


#180 of 203 by jp2 on Sun Apr 29 04:21:26 2001:

This response has been erased.



#181 of 203 by tenser on Sun Apr 29 04:27:33 2001:

Regarind #180; I'm sorry, but I must still be missing your point.
Assuming you meant to say, ``I can cut...'', well, I do that all
the time with trn.  Coming from that perspective, I find picospan
slow and tedious.  Have you tried using the notesfile programs?


#182 of 203 by scg on Sun Apr 29 04:46:42 2001:

Marcus has source to Picospan, and has done some stuff with it for Grex in
the past.  The problem, as I understand it, is that the current ownership of
Picospan is rather unclear, and the license that was transferred to Grex was
for the binary, not the source.

The OS debate, like many architechture issues, is an issue where there's no
clear, one size fits all, answer.  Most systems have plenty of advantages and
disadvantages, and plenty of advocates and critics.  Some points of view are
better thought out than others, and some ways of doing things are better than
others, but it's often very hard to agree on which is which, and often the
advantages to be gained from doing things the "right" way aren't nearly as
great as its advocates think.

Most of these decisions are actually made based on local factors.  The
strongest of those, in established organizations, is the legacy factor.  If
a system is already running, changing it can be a lot of work.  In addition,
if that system is working well, bugs introduced in the changeover can easily
make things worse, rather than better.  Another strong factor is that people
tend to use what they're familiar with, and that's not an entirely bad thing.
The best system in the world run by somebody who doesn't know what they're
doing probably won't work nearly as well as a bad system run by somebody with
a clear understanding of how to make it work.  All quality and even price
issues aside, this is a large factor in why so many companies hiring their
system administrators straight out of high school run Linux -- it's what the
kids have been playing with and gotten comfortable with.  The other free
Unixes tend to pick up smaller chunks of the same market, for the same reason.
People coming out of Solaris heavy institutions tend to use Solaris when given
a choice at a new job.  People who have been using Windows all along tend to
lean toward NT.  In Grex's case, it's original staff members were comfortable
with SunOS, and Grex started running SunOS.  Because Grex has always run
SunOS, it doesn't seem to need fixing, and both the older and newer members
of the staff have learned it and gotten comfortable, Grex continues to run
SunOS.  It's certianly not what I would do if I were in charge and starting
from scratch, but that doesn't necessarily mean it's wrong.

The bigger question, then, is whether the problems people here have been
pointing out with SunOS, override it already being known to the staff, and
override it already being up and running.  Would the improved quota support
in some newer systems be enough of a timesaver that it would save the staff
more time than it would take to implement it?  Would there be more good people
volunteering to be on the Grex staff if Grex were running an OS that more
people new how to administer?  That sort of thing.


#183 of 203 by jp2 on Sun Apr 29 14:46:03 2001:

This response has been erased.



#184 of 203 by gull on Sun Apr 29 18:40:18 2001:

I'd hate to see Picospan axed to turn Grex into yet another private 
USENET heirarchy.  I happen to like the Picospan interface, and I think 
it works better for the kinds of discussions we have here than a 
newsgroup heirarchy would.  I also haven't found a text-based news 
reader that was as easy and quick as Picospan's interface.  They're all 
either bloated, clumsy, cryptic, or all of the above.  tin used to take 
three minutes to start up, on my MTU account.

I actually alternate using Picospan and Backtalk.  If I'm going to go 
through all the new responses, giving my full attention, I use Picospan. 
 If I'm going to be more leisurely about it, I use Backtalk, because I 
can let it sit for 20 minutes while I do something else without timing 
out my connection.  It's a tribute to how well Backtalk is designed that 
I can go back and forth between it and Picospan with no problems.


#185 of 203 by slynne on Sun Apr 29 19:45:45 2001:

You know what? I feel the same way about backtalk and picospan. 


#186 of 203 by mdw on Mon Apr 30 01:38:11 2001:

Dan's new Sun's aren't available any longer?  Damn.  I know staff was
planning to take him up on that - life got in the way (in my case, a
broken clavicle.)

SunOS 4 had primitive multi-processor support in it.  SunOS 5 (aka
Solaris) has much less primitive multi-processor support.  Both of these
beat the multi-processor support in OpenBSD, which is zip.  This isn't a
really big issue, except that our current hardware has 4 slow processors
in it, so "upgrading" to OpenBSD would result in throwing away 3/4's of
our CPU.  Technically, the "more advanced" architecture of Solaris (or
the "newer" architecture of OpenBSD) doesn't really buy us much.
Solaris has kernel threads, a much more sexy user thread model, &etc -
and is somewhat notorious for being less efficient on the same hardware.
OpenBSD would give us source so we could port the tcp & fork bomb logic
forward more easily, leaving us with essentially the same capability we
have today.  In either case, we'd still have a *lot* of homework to do.
There isn't a quantum leap forward here that will give us anything we
don't have today.


#187 of 203 by tenser on Mon Apr 30 04:03:29 2001:

Regarding #186; I'm still looking around for some Sun hardware to donate
to grex.  It seems that another batch of systems might be coming off line
in the near future; these look a lot like the old systems, a Sun Ultra 2
with dual processors, a Sun Ultra 1, and 2 SPARCstation 5's (as opposed
to last time, where it was a SARC-5 and a SPARC-2).  I'm not sure about
the disk and RAM configurations, though.  Probably all the Ultra proc's
are running at 170MHz, and the SPARC 5's at 85MHz.

The quantum leap forward you would get with a newer OS is significantly
reduced maintenance cost; many of the features retrofitted into SunOS
4 by the grex staff come standard with newer software (for instance, an
indexed password database and shadow passwords are a standard feature in,
eg, *BSD.  LDAP could serve the same purpose under Solaris).  The need
to install GNU fileutils and friends goes away under both systems; *BSD
has it's ports/packages collection to make installing 3rd party software
significantly easier, while prepackaged collections of useful 3rd party
utilities for Solaris exist around the net.

What's more, Solaris source *is* available (though I wasn't able to
penetrate the legalize of the license to understand exactly the terms
it's offered under) for download from Sun.  What's more, Solaris is
pretty zippy on newer hardware.

One last thing; grex has 4 processors (I thought only 1 processor card
was installed?  Doesn't it only have 2?), but each one runs at 40MHz
or so.  A single Ultra 1/170 already has more power than the sum total
of all processors currently in grex.

Granted, the current machine should theoretically be able to do 4 things
at once versus the Ultra 1's 1 thing, four times faster.  However,
given the poor quality of SunOS 4 SMP support, I'd bet that the Ultra
1 would still beat out the slower multiprocessor.


#188 of 203 by jp2 on Mon Apr 30 13:09:56 2001:

This response has been erased.



#189 of 203 by mdw on Tue May 1 02:22:30 2001:

Uh, what "maintenance cost"?  This is why we have all those extra
hardware bits.  Yup, one quick processor would beat 4 slow ones.
Solaris doesn't come with GNU userland, so no win there.  There
definitely are attractions to newer stuff, but it's not as simple as you
think.  For instance, stock solaris "passwd" is *much* slower than our
current "passwd" pgm, probably too slow for us.  *bsd's "master.passwd"
database scheme would need similar work as well - newuser is run often
enough that we can't afford to run "pwd_mkdb" every time.  LDAP has some
obvious DOS attacks; it's not an attractive option to us.

On a uniprocessor machine, we'd almost certainly pick openbsd over
solaris, but this still leaves a formidable task ahead of us, in terms
of: lining up hardware spares, identifying, porting or adapting software
for our special needs, and making sure we don't leave any nasty security
holes in what is perhaps the ultimate challenge for any OS: active use
by lots of potentially hostile users.


#190 of 203 by tenser on Tue May 1 05:03:07 2001:

The maintenance effort in terms of time, labor, and sweat all translates
into a cost, even if it's personal as opposed to financial.  All though,
we saw a grex staff person post in this very thread about she spent a
working day cleaning up grex's disks, thus leading to a realized loss
for her financially.

Many of the things which you list as making the job of moving to a newer
machine more complex are precisely the things which I think grex ought
to do away with in the interest of making its staff's lives easier.
The password database is a prime example; is it really that inefficient
on modern hardware to make use of whatever the system comes with?  In the
case of *BSD, I'm having a hard time believing that pwd_mkdb running
on a modern machine, with a modern disk subsystem, is going to be so
much less efficient than what grex currently uses as to be unworkable.
Also, how often would it be run per day?  Even 200 times, at 15 second
run times, is less than an hour a day.  The same holds true for passwd(1)
(besides, one would hope that grex would take this opportunity to move
to Kerberos and it's kpasswd program).

As empirical evidence that the Solaris password scheme is also workable,
it *appears* that the nether.net system, running on a Sun4d with 8 (!!)
processors uses the stock Solaris passwd subsystem (file and associated
commands, no dbm indexing).  It appears quite zippy, and supports twice
the number of entries as grex (large UID's would be another useful feature
of newer OS software).  Anyway, I'd think that a dual processor Ultra
2 would be comparable in performance to a SPARCcenter 1000.  ahh, yes;
each of those processors runs at 40MHz, a lot like grex's.  I'd put my
money on the Ultra.

This is reminiscent of a useful maxim in software engineering, about
optimization.  The first rule of optimization is this: Don't do it.  Why?
Because most of the time you don't need to, and it just makes maintaining
the program significantly more difficult.  A parallel can be drawn to
system administration.

OpenBSD on a uniprocessor UltraSPARC would be problematic, since there is
currently no Ultra port of OpenBSD.  I agree that it would be somewhat
of a challenge to move grex to other hardware, but I don't think it
*has* to be as complex as you seem to be making it.  In particular,
if you guys cut out a large amount of the custom software, particularly
in the ``hacks to the OS and associated subsystems'' arena, it becomes
signficantly easier.  I think that making a move now represents a good
opportunity to do just that.

Another useful software engineering maxim is this: Don't diddle the code,
find a better algorithm.  In my opinion, a lot of what grex has done to
fix problems in the system is akin to diddling code; moving those hacks,
which might very well have made sense on SunOS 4, to a new system seems
counter productive.  The kernel patches to restrict outgoing network
packets are a good example of this.  Instead of putting that logic
into a custom kernel hack, put it into rules in a vendor's existing
firewall offering, and then give members access to another machine
without those restrictions.  You never have to worry about ``porting''
the changes again.

I guess what I'm saying is that grex, in an absolute sense, probably
has far fewer special needs than it's users and maintainers believe it
does.  One can exploit that to ease the maintenance burden put on its
staff in moving to new hardware.


#191 of 203 by mdw on Tue May 1 07:38:59 2001:

When the Well had its famous break-in, they had to change everyone's
password.  This was on a Sparcstation 20 running Solaris.  The Solaris
passwd program was sufficiently slow that they ended up having to run a
special reset process with some sort of replication scheme to populate
the real database, with a delay of several hours in it.  This was
hardware that was, for its day, state of the art.  It would be
interesting to try to bench-mark it on modern hardware, but I would not
take it as a given that it will, in fact, work with grex's load.  Grex's
passwd program updates things "in place" if it can - this is a *lot*
less disk I/O than the vanilla Unix password program, and disk speed
hasn't improved nearly as much as CPU.  Yes, running kerberos would be
an attractive alternative - although it doesn't completely fix the
passwd file problem because there's also newuser and chfn to worry
about.  Newuser is definitely something that has no vendor equivalent,
and for which there are no short cuts to making it work right (believe
me, they've all been tried, many of them right here on grex.)

Replacing the kernel blocks with firewall rules would be, um,
interesting.  Right off the surface, the first obvious problem is we'd
have to install a firewall, because we haven't got one today.  It
certainly wouldn't duplicate the current functionality - mail delivery,
and "what to do with members" would be just two of many issues to
address.  Running a 2nd machine for members has its own ugly problems -
for instance, that means we'd have 2 login machines, with different
features, to administer.  That would make a shared filesystem
attractive, but that introduces yet another set of problems to worry
about.  I'm not sure we'd want to pull this string too far, it looks
kind of sticky.

There are a lot of different directions we could go with this machine
upgrade thing - the problem with any of them is they take time to
investigate, and aren't guaranteed to work.  That doesn't necessarily
mean they're a worse solution than what we currently have, but it does
mean they're not necessarily automatically better.


#192 of 203 by gull on Tue May 1 12:58:13 2001:

A seperate machine for members would also cause us to end up with something
like the mnet "reserved telnet ports" situation, with nonmembers waiting in
the telnet queue for their machine while members slip by into their own special
private box.  While that isn't really a problem, exactly, I've gotten the
impression that giving members that amount of special privilage isn't really
something Grexers have been comfortable with.


#193 of 203 by blaise on Tue May 1 13:43:16 2001:

Re: #191, #192.  You wouldn't need a second system for members.  Restrict the
ports based on what group the user is a member of.  If they're not a member
of group "members", then don't allow them access to whichever ports you don't
want to allow them.  Most firewall software can do that; I know that ipchains
can and it is included with FreeBSD.


#194 of 203 by jp2 on Tue May 1 13:52:38 2001:

This response has been erased.



#195 of 203 by blaise on Tue May 1 15:48:51 2001:

Sorry, I used the wrong name.  I remembered that there is a firewall package
included in each of FreeBSD and Linux, and my brain farted out ipchains as
the name.  But my point was valid.


#196 of 203 by gull on Tue May 1 18:05:45 2001:

IP chains *was* what Linux used.  It was a replacement for ipfwadm, which
was deprecated because of some limitations it had.  Now they're apparently
replacing it with iptables, to make sure everyone's good and confused and
compatibility is all shot to hell. ;> I don't know what advantages iptables
has.  FreeBSD also used to use ipfwadm, I think.  I don't know what they use
now.

I kind of like OpenBSD's ipfw functionality.  At least it has *real* state
tables, instead of the user-space hacks you have to use with ipchains.


#197 of 203 by carson on Thu May 3 20:57:16 2001:

(the only thing that disappointed me about Dan's offer to donate 
hardware was the apparent lack of response to it.)

(I wouldn't see new hardware as an immediate solution.  I understand 
it takes time to hammer out upgrade details.  if my memory were 
better, I'd remember the move to the Sun-4.  :^)  what I do remember 
was that it wasn't immediate, and took many months, but was worth it 
once the move took place.)

(I don't want Grex to pursue upgrades in speed for speed's sake, and 
wouldn't want Grex to drop a few grand on new hardware.  *but*, if 
that hardware's being offered essentially free, then it seems to me 
that there's no harm in trying to get the newer hardware working.)

(I think I'd like to see the discussion of possible hardware upgrades 
take place separately from what might be done *now* about disk space.  
would it help staff to have more people willing to do the [admittedly 
tedious] work of policing disk use?)


#198 of 203 by i on Thu May 3 23:55:24 2001:

It would be nice to have the hardware/software/upgrade/etc. discussion
in the garage cf., instead of here, where i suspect that it just shuts
down discussion about the issue actually at hand.

Staff & board can handle the problem; it'd go better if they could get 
good user input beforehand.


#199 of 203 by jesuit on Wed May 17 02:14:17 2006:

TROGG IS DAVID BLAINE


#200 of 203 by styles on Fri Jul 14 21:31:49 2006:

276 newresponse items and 44 brandnew items

it's been a while since i've been here, but i suspect wild expurgation and
scribble madness.


#201 of 203 by styles on Fri Jul 14 21:32:17 2006:

blah> fix
blah> j coop

         You have now left the Co-op conference.


              Welcome to the Co-op Conference [Ver 13]
           The conference for users to decide Grex's future
                    Fair Witness: cmcgee

    All important decisions about Grex are discussed and made here.

           For basic information, type "help introduction"

First item 1, last 342
blah>


#202 of 203 by scholar on Fri Jul 14 22:24:45 2006:

yay!


#203 of 203 by naftee on Sun Jul 16 01:16:04 2006:

blah !


There are no more items selected.

You have several choices: