Grex Coop10 Conference

Item 20: Staff Reports

Entered by janc on Mon Jul 21 14:36:51 1997:

Once upon a time someone complained that staff wasn't reporting enough about
what we are up to, and suggested that we report more stuff in co-op.  I think
we did this for a little while and started forgetting about it again.

I thought I'd revise the practice.  I hope I'll remember better in the future.
16 responses total.

#1 of 16 by janc on Mon Jul 21 14:45:03 1997:

Recently the Grex staff, led by Marcus, redesigned the way passwords are
encrypted on Grex.  We had to rebuild a number of programs on Grex (login,
passwd, su, newuser, webnewusers, backtalk's authenticator, etc.) to use
the new encryption method.  Users shouldn't notice any difference at all,
but this will be an improvement in account security, and more importantly
it should ease a future transition to a distributed network authentication
system (like kerberos).

I screwed up on the modifications to webnewuser, so that for a few days,
accounts created by webnewuser had their passwords set incorrectly.  The
problem has been fixed, and the new system seems to be working smoothly.

If you'd like the small added security of having your password encrypted
by the newer, tougher encryption, just change your password.  All newly
set passwords use the new encryption, while old ones still use the old
encryption.

Since passwords have to be reset at least once a year (due to the password
expiration system we use here), all user passwords will use the new
encryption algorithm within a year.


#2 of 16 by janc on Mon Jul 21 15:09:31 1997:

The next project, also initiated by Marcus, is a change-over to hierarchical
mail directories.  A while back we changed user's home directories into a
hierarchical structure, so that instead of being under /home/janc, my account
is under /a/j/a/janc.  This is more complicated for humans, but much more
efficient for the computer to access.  Under the old system, when Grex looked
for my home directory, it had to search for "janc" among the 15,000
directories in the /home directory.  This requires on average about 7,500
tests.  With the new structure it first searchs under /a for the first letter
of my name "j" (there are 26 entries here, so it takes on average 13 tests),
then it looks under /a/j for the second letter of my name (another 26 entries
so another 13 tests on average) and then it looks under /a/j/a for "janc"
(there are about 15000/(26*26) = 22 entries here, so that's another 11 tests
on average).  So this takes about 13+13+11 = 37 tests on average instead of
7,500.  Since Grex has to look for users home directory a lot, the
hierarchical structure does a lot to speed up Grex.

So now we are going to do the same thing to the mail spool directory. 
Currently a user's unread mail is kept in a file under the /var/spool/mail
directory.  Any user that has unread mail on Grex has a file there.  Mine is
/var/spool/mail/janc.  Currently, there are about 8,100 files in that
directory.  Whenever anyone reads their mail, or whenever a piece of mail is
delivered to a user, Grex has to search through those 8,100 files to find the
right mail file.  So we are going to apply the same solution as we did to the
home directories.  This should make a significant performance difference on
Grex.

To do this, almost all the programs that deal with mail need to be worked on.
A few are smart enough to do this without modification, but many needed to
be revised.  Marcus has revised the mail delivery programs and the "mail"
mail reader program (that was a tough one because the version we were running
here is original SunOS, which we don't have source to, and the public domain
ones were inferior and not very compatible.  Marcus put a lot of work into
making one that will work right).  I did "pine" last week, and dang is working
on "mh" even as we speak.  With that one done, we should be ready for the
change-over.

If all goes well, users should see no change, except maybe a small improvement
in Grex's speed.

This is kind of a cool project.  I've never seen or heard of a Unix system
with a hierarchical mail directory, but it's a good way to squeeze a little
more performance out of an overloaded machine.  Most people solve such
problems by throwing money at them.


#3 of 16 by remmers on Mon Jul 21 19:08:45 1997:

(Grex is better endowed with creative technical expertise than
with money. A number of very talented people are working *real*
cheap...)


#4 of 16 by scg on Tue Jul 22 05:19:32 1997:

A bit more than a year ago now, two Chase IOLan terminal servers were donated
to Grex.  At first, they seemed to be having lots of problems, and since
nobody really had much time to work on them, they were put aside.  I started
experimenting with the one of them that we named groupie a few months ago,
and tried a bunch of things.  After some amount of trial and error, when
Grex's serial card started having problems, we got the point where we were
sort of confident in it, and desperate enough to try it.  After some more
trial and error, it seems now to run very well.  Hopefully at some point soon
we will be able to move all of Grex's modems over to it, thus taking the
burden of dealing with modems off of Grex, and allowing all of our modems to
run at 14.4 kbps.  Before that can be done, we need some changes done to the
telnetd differentiate between users on the terminal server and users coming
in over the net link, and not make dial-up users wait in the telnet queue.


#5 of 16 by valerie on Tue Jul 22 16:26:32 1997:

This response has been erased.



#6 of 16 by valerie on Tue Jul 22 16:29:03 1997:

This response has been erased.



#7 of 16 by remmers on Tue Jul 22 16:59:50 1997:

(I've wondered the same thing. It's a handle which I believe I
first noticed Jan using while he was at TAMU.)


#8 of 16 by valerie on Tue Jul 22 17:12:24 1997:

This response has been erased.



#9 of 16 by janc on Tue Jul 22 22:39:39 1997:

Oops, name carried over from enigma.

Valerie and I tried sticking another memory card into Grex.  Klaus did some
tweaking of the power supply a week ago, and we are hoping that Grex will be
stable with more than 32Meg of memory now.  So we have it up to 64Meg.  If
it doesn't start crashing left and right, we'll try restoring the memory up
to the full 128Meg again.  Grex should run much faster with more memory.


#10 of 16 by srw on Wed Jul 23 05:22:12 1997:

I do some of the same things Valerie does to maintain the smooth running 
of the system.

I handle most of the difficult requests to reset lost passwords, so I 
have to worry about numerous security issues. I also maintain some parts 
of the website, although I have recruited help in this area recently, 
because I don't have as much time to spend on these things as some other 
staffers.


#11 of 16 by dang on Wed Jul 23 15:19:41 1997:

At the moment, my big project is to fix mh so that it understands heirarchical
mail directories.  That's in the works.  I compiled mh on sunday, and it took
almost 12 hours, so if you noticed Grex slow sunday night, that was probably
it.  I'll be testing it over the next week.  

I also do some of those daily things.  I answer questions about Grex, watch
the postmaster list and catch problems that Valerie hasn't already caught,
and I'm the one who usually deletes accounts at the owner's request.  we
usually get 5 - 10 of those a week.  Asside from that, I do whatever catches
my eye that need doing.


#12 of 16 by steve on Thu Jul 31 04:21:50 1997:

   Contrary to the comments of someone else in coop, the Sun-4/670
isn't collecting dust.  I'm able to spend more time on Grex things
in the evening, and am coming up to speed on it.

   Recently we made some new SCSI cables for the internal disk
trays, to test the theory that one of them was damaged, and
affected the disks.  This new cable did not change the problem
we've been seeing, which is that after booting the boot disk
gives errors on a regular basis.

   While disappointing, it tells us something.  In talking
with Jared it may well be the case that under a different
kernel, the 670 did talk to both drives without error.

   Now, I'm still coming up to speed on things and haven't
read the discussions in the garage conference, so I likely
don't have the whole history of trying things correct yet.


#13 of 16 by dpc on Thu Jul 31 19:34:10 1997:

Glad you're working on this, steve!


#14 of 16 by atticus on Mon Jun 15 19:31:48 1998:

(What are the staff activities these days?)


#15 of 16 by janc on Thu Jun 18 20:32:33 1998:

Mostly staff doesn't seem to be too busy with new projects.  Marcus is doing
some work toward a kerberos-type authentication system which will be helpful
to us as we drift further toward using multiple computers to provide Grex
services instead of running everything on one computer.

There is also work being done on getting up a mail forwarding machine and ppp
on the dial up lines.  Progress is kind of slow on these.  It's possible that
Marcus has started thinking of both as being things that would be better after
we have the kerberos authentication going.  If we go that route it could take
a fair amount of time to get there.  I hope we won't.

We have been contemplating doing some reorganization to the staff mailing
lists, to make it easier for us to handle the floods of regular staff mail.
I need to take time to get this done.

Outside of that, I think we are mostly just treading water.  Plenty of work
to do, but nothing very exciting.


#16 of 16 by srw on Mon Jun 22 01:48:44 1998:

We are looking into redesigning Grex's web pages, too. The flood of 
staff mail is a deluge, so investing time in finding ways to reduce it 
will actually make more staff time available.


There are no more items selected.

You have several choices: