response 125 of 181:

Nov 3 07:38 UTC 2001

not 'ir' all goes well....

'rf' all goes well!

response 127 of 181:

Nov 3 20:25 UTC 2001

The university I work for has recently aquired several Sun Enterprise 450
servers from bankrupt dot coms for a mere $10k.  If the discount can be found
at other auctions, perhaps you could pick up a Sun E280R rackmount workgroup
server for around $2k (e450 ~ $100k new, e280 ~ $20k new).  Oops.  This is
in canadian dollars, so multiply all of these figures by 0.62 (even cheaper).
I think it would be much better to take advantage of very high end equipement
being liquidated at rediculously low prices (often the auctioneers don't even
know what it is), than to spend your few dollars on brand new x86 "consumer"
equipment.

response 129 of 181:

Nov 3 21:23 UTC 2001

Regarding #127; I disagree.  x86 equipment is just as reliable as Sun
stuff, performs better, and is a lot cheaper to maintain over the long
run.

Since I can't make it to this meeting, here are my 2c.

That said, I highly recommend that grex go with an x86 machine running
FreeBSD in an SMP configurationm with a lot of RAM, hardware SCSI RAID,
and fast ethernet.  It's the most bang for the buck you can get right
now, it'll be around for a very long time in the future, and can support
a lot of very interesting stuff.  If you can get a used PCI digiboard
serial card, you can hook up all of your modems up to it, too, eliminating
the need for your terminal server.

To address the security issues you'd face, I'd mount everything except
/ nosuid, mkdir -p /suid/{bin,include,lib,sbin} and then get the source
to, and recompile and statically link all the setuid and setgid binaries
you *really* need and put them in /suid/{sbin,bin}.  You could probably
even mount /usr read-only, and possibly do the same with /usr/local if
all the programs that need to write there have some place else to write
(this basically means TeX and maybe emacs; of course, doing this would
make updating things in /usr/local a pain).

For locally written setuid stuff, I'd write a set of highly tested and
vetted libraries that I placed in /suid/lib (headers in /suid/include)
that did things like deal with variable length strings securely, do
IPC and authentication, and so on.

A lot of other security concerns could be mitigated by running recent
versions of such things as OpenSSH, telnetd, ftpd, etc.  The list of
network services grex *really* needs to run is probably pretty small.
I'm guessing:

        finger
        ftp
        http
        ntalk
        smtp
        ssh
        telnet

Servers for these, again, I'd recompile and put into a separate filesystem
(eg, /suid, though at that point, you might want to rename /suid to
/local or something).

Some things I'd rewrite to either use kqueue or sockets (eg, party
would be a good candidate for that; it'd be nice if picospan used
some protocol to talk to its back-end instead of just stashing stuff
into a file structure).  Anyway, by making these things always run as
some user who is dedicated to them (eg, create a user called `party'
to run party), you eliminate the need to make things like part(l) suid.
The party daemon (and others, like the web server) could run in jails
for even more security.

Not many hacks need to be made to the kernel; eg, ipfw already does what
grex needs for firewalling, so the socket based kernel blocks don't need
to be ported.  I question to what extent things like the fork bomb killer
*really* need to be added if the per-user process limit is low enough, and
what facilities to combat this sort of thing does FreeBSD already provide?
At anyrate, it's worth looking into.

Also, using soft metadata updates and aggressive directory caching will
save a lot of I/O hitting grex's disks, which is good, and frees the
staff from having to do hacky things like ordering mkdirs in the home
directory filesystems so that more common directories are searched first.

In userland, If the system is big enough, I don't see a need for
the queuing telnet daemon; one of those other systems (nether.net or
something) doesn't seem to run it, and seems to have plenty of capacity
for letting folks log in at the same time.  SSH can and should be upgraded
to the latest version of OpenSSH, which will fix a lot of the problems
with it on grex.  (Like that weird one-bit-off error message that keeps
popping up when folks log in!).

The functionality of pwd_mkdb can be substituted for what was done with
the shadow password database, which simplifies matters.  I'd set up
Kerberos 5 for authentication (KDC on another machine), and ditch the
grex password algorithm in favor of 3DES.  Failing that, I'd just use
the FreeBSD-esque MD5 support (note: I'd modify the grex software *now*
to do this, whichever way one chooses, so as to be ready when the system
switches over.  Users have to change their passwords once a year anyway,
so this can be done transparantly).

I'd also take the opportunity to ditch sendmail in favor of Postfix,
which has an excellent security track record, is a better performer than
sendmail, and omits a lot of the bells and whistles that grex doesn't
really need.  It can undoubtedly handle hierarchial mail directories.

Also, the BSD ports collection will automate and simplify the installation
of a lot of the services grex wants to offer; for instance, installing
postgres, bash, emacs, etc. is trivial.  A lot of the software that grex
has installed either comes with FreeBSD, or is directly installable via
the ports collection.

The net result would be a machine that was pretty much standard,
yet had most if not all of the functionality of the current grex.
In addition to easing the maintenance hassle for staff, particularly in
the area of upgrades (since their wouldn't be many hacks to worry about
re-implementing), you'd get a much more robust software environment on
some serious capacity hardware.

All in all, with FreeBSD, x86, and Kerberos 5, you can't lose.

response 131 of 181:

Nov 4 04:17 UTC 2001

  Although most of the debate seems to be focused on which processor
  and OS platform to choose, my advice is to concentrate more on I/O
  performance (specifically, disk access.)  But perhaps that's obvious..

response 133 of 181:

Nov 4 05:40 UTC 2001

The short form of the outcome:

  OpenBSD.  This won out over FreeBSD because Marcus and STeve feel that
    the security advantages it offers outweigh the disadvantages in features
    and performance that FreeBSD offers.

  Sparc64 or Pentium.  No decision has been made.  We are going to try to
    get some cheap machines of each type, and start developing the Grex
    OpenBSD configuration on both.  We are going to postpone the purchase
    of the actual Next Grex machine until we have an OpenBSD set up that we
    are ready to put into production.  This way, if it takes us a year again
    to get the new system set up, it won't be a year obsolete before we
    come on line.  Also, the OpenBSD port to sparc64 is only a month old at
    this point, and we need to do some testing on our own to see how stable
    it really is.  If it doesn't meet our requirements, the Pentium (or
    AMD clone) option will likely win out.

  I've sent Email to Dan Cross saying that our previous luke warm interest
    in the two UltraSparc machines he was offering us has suddenly turned
    into rather intense interest.  These would be great development machines
    for the OpenBSD project.

  We are looking for a cheap or free Pentium machine that we can also use
    for the OpenBSD project.  We'd like to find something at least 200 MHz.
    More is better, but this would not be a production machine at this point.

  We want to get newer faster modems and probably a newer better terminal
    server.

I think that's most of it.

response 135 of 181:

Nov 4 08:17 UTC 2001

 ... and teh sun has set?
  
 /// awh, shit!

response 137 of 181:

Nov 4 15:48 UTC 2001

I have an Annex III terminal server sitting around and which Grex is
welcome to have -- if that counts as "newer better".

response 139 of 181:

Nov 4 22:23 UTC 2001

Aww shucks; this means I need to take the train to Bensenhurst and put
those Suns in boxes....  Whadda pain.  :-)  Jan, I'll send the machines
off as soon as I can; probably sometime in the next week.  You just
want the Ultra's, right?  Not the Sun4m machines?

Again, I recommend using x86 hardware due to the upgradability and
performance, and especially if you're going to use OpenBSD; I think
the Ultra port is just too immature at the moment (and besides, it
won't do SMP, so the second processor in the Ultra 2 is going to be
unusable!).

OpenBSD is good stuff, but you'll have to hack it up a bit more than
FreeBSD.  Oh well, c'est la vie.

response 141 of 181:

Nov 4 23:34 UTC 2001

hmmmmmmm, that would not be a desirable outcome.

response 147 of 181:

Nov 5 02:28 UTC 2001

Re #129: even if grex ends up running an os that has kqueue, I suspect
it might be best to avoid kqueue and use sockets, so that if grex changes
again, less porting work well be required.