Grex Agorage Conference

Item 5: OpenBSD: Discussion of appropriateness for grex and technical merits.

Here are some reasons I believe one may consider OpenBSD inappropriate
for grex and/or "not modern."  I divide this into two sections.
The first consists largely of reasons I don't think it's particularly
appropriate for grex.  The second are reasons I wouldn't necessarily
consider it "modern," and is fairly technical.

Note: OpenBSD on the i386 architecture is certainly better for Grex
than was SunOS 4 on sun4m.  Compared to SunOS 4, OpenBSD is a panacea
of modernity.  That doesn't mean it is on an absolute scale, nor
relative to some of its contemporary peers.  In any case, it was
and remains an improvement on what we had before.

OpenBSD was chosen for grex largely because of its reputation as a
"secure" operating system, and largely at the behest of two staff
members.  One of which is no longer active with grex; the other
remains involved and is well known for his care of and ability to
maintain grex.  Despite this, whether by a defect in the operating
system or through improper configuration, we have had security
problems with it.  For instance, the `cat'able tty' hole that existed
under OpenBSD 3.5 on grex.  One naturally questions whether we could
have done better with another operating system?

INAPPROPRIATE FOR GREX:

Most of the problems with OpenBSD result from its relative obscurity.
OpenBSD is far more obscure than some of its better-known relatives;
even those in the BSD family tree.  In particular, FreeBSD has a
much larger userbase and audience.  This leads to several problems:

1) Potential administrators are less familiar with it, and the
   more esoteric features that are used by grex present a learning
   curve for anyone interested in the staff side of grex.  Certainly
   extremely important components of the system differ in non-trivial
   ways from the defacto standards presented by better-used
   alternatives.  For instance, compare the structure of OpenBSD
   ports to FreeBSD ports.

2) There are significantly fewer ported applications on OpenBSD
   than on, say, FreeBSD.  Many useful and desired applications
   don't exist on OpenBSD, or aren't as well supported.

3) Related to #2, there is deficient Java support on OpenBSD.  It
   seems that a JDK port is fairly recent, and there are problems
   with it.

4) Lower overall usage leads to less testing and in a system that
   is, overall, less stable.  This is true both of the core operating
   system and ported applications.  We've had a number of stability
   problems with OpenBSD that have led to crashes (bugs in the
   filesystem code have wedged the machine).

5) The engineering done by the OpenBSD team is less polished than
   that of other projects, and their focus is more on experimentation
   and tinkering.  There seems to be very little in the way of
   process, when compared to say, FreeBSD.

NOT MODERN:

Unix is nearly 40.  Despite continuous evolution over its lifetime,
remarkably little about it has changed: the structure of the current
operating system largely solidified sometime in the mid to late
1970's and then stagnated.  Despite several valiant attempts to
push it in a new direction, not much has happened, and while the
system certainly retains great utility, it can hardly be considered
a pinnacle of the modern state of the art.  Or, rather, it can only
in so much as the "art" has come to mean Unix, which is the textbook
operating system only because the textbook was written about it.
Even in this context, OpenBSD falls flat in several ways:

1) LRU based page replacement policies.  The 4.4BSD Mach-based
   virtual memory subsystem used in OpenBSD was replaced several
   years ago by the "new" UVM virtual memory subsystem to address
   deficiencies in the former that had been idetified over time.
   However, upon closer examination, one seems that UVM isn't all
   that modern: it's based on the same Least Recently Used page
   replacement strategy that Unix has used since paging was introducted
   on the VAX in the late 1970's and early 1980's.  This is despite
   some clear benefits that could be gained by using other strategies,
   such as working set methods.  In fact, the working set model was
   implemented by Charles Forsyth at the University of York in the
   1980's for SunOS 4, demonstrating that method's workability under
   Unix.

2) The antiquated and overly complex VNODE filesystem virtualization
   layer, adopted from 4.4BSD, has yet to be replaced with something
   more flexible.

3) The threading model is weak, lacking good support for kernel-level
   thread scheduling.  Multi-level scheduler activations remain
   non-existant.

4) SMP support remains weak, and most of the kernel is still single
   threaded (relying on interupt priority levels for synchronization
   between processes).

5) Too much emphasis is placed on the "Unix model" and "Unix way
   of doing things."  Hence, despite the widespread availability
   of bitmapped display hardware and networks, OpenBSD remains
   entrenched in the TTY based world for interaction.  The
   pseudo-terminal approach makes most programs think that they're
   talking to a VT100 terminal via a 9600 baud serial line, when
   in reality, they're interacting with a high resolution display
   over a high-speed local area network.  Things like X11 are poorly
   wedged onto this platform.

6) Evolution has led to a messy hodge-podge of interfaces for
   manipulating the system.  Network devices don't look like files
   but TTY's do; shared memory mechanisms are crude and lack higher
   level abstractions (e.g., we still do co-routine style concurrency
   management between processes that share memory, with explicit
   sychronization mechanisms like semaphores and mutexes, instead
   of CSP); we still use ioctl().  There is no push to clean up the
   mess of interfaces and approaches, despite prior examples of
   systems that do so but provide explicit compatability interfaces
   for legacy programs.

These are just a few.  I could come up with more, but I'm tired.

Comments?

   Thanks for starting this -- I'll read all of it when I'm back at home.

I'll take these points on in the order presented by Cross.

INAPPROPRIATE FOR GREX

1. I know little about BSD admin in general, and even less about OpenBSD in
particular, but my impression from moving amonst Linuces is that there really
aren't that many show-stopping differences between them. A decent sysadmin
(and whilst I might not otherwise describe myself as one, let's do that for
the purposes of this post) can easily keep the differences between SuSE and
Gentoo in his head, for example.

2. Lack of wide software support is, surely, an issue if we use a wide variety
of software. I've not seen what I would consider as I wide variety either in
use, or being requested.

3. How much of a problem, realistically, is "lack of decent Java support"?

4. It's true that lack of a user/tester base can make a system less stable,
but I'm sure (given his comments in the past) Cross isn't recommending a move
to Linux, which definitely has the widest userbase of all free Unices and
(arguably) of all Unix systems in general. Even if I were to advocate a move
to Linux, it's clear to me that whilst the system itself tends to be rock
solid, applications such as Firefox (which, admittedly, we wouldn't be running
anyway) and even semi-critical OS components like NFS can be an unstable PITA.

5. I can't vouch for this either way, but I reckon Theo would have words to
say about it; granted, he seems to have words to say about a lot of things.

NOT MODERN:

OK, so OpenBSD isn't modern. FreeBSD (which seems to be the way a lot of
Cross's arguments are heading) might be argued to be more modern, but it's
actually older than OpenBSD itself, and both go back to a 30-year old operating
system (counting from the genesis of BSD specifically). I'm not being facetious
when I say that wheels aren't modern, either, and people still use those :-)

"State of the art" is overrated. Microkernels are arguably more
state-of-the-art than monolithic kernels, but the problem is that they tend
either not to work very well, be even more obscure than OpenBSD, or both. (I
don't consider Darwin, OS X or even mkLinux to be examples of successful
microkernels, since they essentially complicate a microkernel by building a
monolithic kernel over them; this might aid portability, but fulfills none of
the other "requirements" of a "successful" microkernel.)

Unix has the advantage that nobody else bothered to release a capable,
general-purpose, extensible, widely-supported, cheap OS *with source code*
just when operating systems research was arguably at its zenith. Windows is
widely available, but come on, it's not even under discussion for our
purposes. Plan 9 isn't the kind of improvement Cross seems to be after,
either, as all it seems to do is correct several flaws in the implementation
of the Unix networking stack (namely, that it disrupts the principle of
"everything is a file", and that distributed OSes built on Unix are untidy,
at best). It also has several disadvantages which coincide which those Cross
cites as being OpenBSD's (lack of support, lack of admins with sufficient
familiarity, etc).

1. I don't know of any OS that doesn't use LRU scheduling (though I certainly
don't claim to be an authority on the matter). OpenSolaris, maybe? I suspect
that the OpenSolaris community may be dwarfed by even the OpenBSD community.
All other OSes that might qualify are either even more marginal or unfit for
our purposes.

2. & 3. Which OSes with credible replacements for VNODEs and the OpenBSD
threading model are fit for our purposes?

4. IIUC, DragonFlyBSD solves some of the problems of SMP support, but it's
still far from being general purpose. I also understand that many of the
FreeBSD people themselves are unhappy with symmetric multi-threading on that
platform. SMP support on general purpose computers is either marginal (try
getting your hands on a Sequent) or experimental/new (just about everything
else).

5. Many of the problems Cross cites in his 5. could be solved by Plan 9;
unfortunately, even if I could see all Grexers moving to Plan 9 in a dual boot
or spare-machine configuration, current network latencies preclude this from
being a practical proposition. Either everyone has a T1 line (which rules out
most if not all international grexers, and I dare say a fair few domestic (US)
ones, or we put up with a system which *might, on occasion, have about 75%
of its heavy-use nodes online*.

X11 is fit for purpose; if we ever get to the stage where a substantial
proportion of grexers want to run graphical programs, there are X servers for
Windows and a bucket load of other OSes beside. On an aesthetic note, 81/2
and rio are ucking fugly.

6. Very few systems (if any) in as wide use (or greater) than Unix do NOT
suffer from a hodgepodge of interfaces; a great many, great interfaces of
yesteryear have been doomed by making them proprietary, leaving us only with
Beastie and The Beast (and the Penguin).

In short, Cross, you highlight many problems with OpenBSD and Unix, and I
agree with most if not all of them. The problem is, what are the reasonable
alternatives?

Well, there's *VMS, I suppose. VAX/VMS is old (maybe ancient); Alpha/VMS is
moribund and expensive; Itanic/VMS is even more expensive and arguably just
as moribund; FreeVMS doesn't seem to be getting anywhere. There's already at
least one public access VMS system out there, and it's a cluster (the one cool
feature we don't offer but could); VMS is (in the opinion of myself and
several other grexers I know of) ugly; which of our non-techie users wants
to be forced to learn new interfaces for file management, mail, and
conferencing? Which of our sysadmins wants to port Grexsoft?

Regarding what Jeff wrote:

INAPPROPRIATE FOR GREX:

1) It's true that many of the differences are trivial, but some
aren't.  For instance, the whole authentication system is a bit
non-trivial.  Also, as Jan noticed when he put nextgrex together,
many of these things are poorly documented and in some cases just
aren't documented or, even worse, the documentation is wrong.  What
then?  Almost any Unix system is similar to any other, but that
doesn't mean you shouldn't know how one works, does it?

At some level systems are divergent just because they are, with no
better rationale.  Here's a difference between OpenBSD and FreeBSD
that was annoying around the time of the transition from SunOS: The
FreeBSD kernel caches a pointer to the current process's group
credentials in the networking code, which means that the FreeBSD
firewall can check that a user is a member of a specific group,
regardless of whether the saved, effective, or real group ID of the
process talking to the network is in that group.  OpenBSD does not
do this (it only saves the latter).  As a result, to implement the
packet filters previously handled by a custom hack to the (binary)
kernel, it was necessary to write a setgid wrapper and link lots
of things to it.  The wrapper would then look at how it was invoked,
set up the group permissions appropriately, and exec the proper
program.  This is a lot less flexible for our users, who may or may
not understand why the network program they wrote won't work.  It
was also a lot easier for junior sysadmins to install a network
program: under OpenBSD, install the program and THEN find and modify
the wrapper and reinstall that.  Under FreeBSD, they could just
install the program.

2) We might find we were able to provide more things that piqued
the interests of our users if we had the option of installing more,
varied, software.

3) Honestly?  I don't know.  But it *has* been requested, several
times.

4) No, I'm not recommending Linux, because frankly, the rate of
change for Linux is just too high and it's too insecure.  You're
right, it's not just testing, it's a balance of many different
things.  Testing must be compared to security, maintainability,
popularity, etc, etc, etc.  I don't think that Linux would really
win one of those competitions, but I certainly do think that FreeBSD
would beat OpenBSD on a level playing field.

5) I'm not sure Theo would be the one to ask.  He's too volatile.
But this goes along with (4), which really should be about the
stability of the system as a whole.  Testing is one measure of that,
but not the only one.  Also at play are the engineering practices
of the group that produces the OS.  FreeBSD does a bit of a better
job of this than OpenBSD, which seems to be more focused on doing
stuff that's "cool" and just sort of hacking it until it works than
on following the Unix model in the context of a well-defined systems
engineering process.

MODERNITY ISSUE:

I should have been clearer: I'm referring to OpenBSD being not
*modern* in an absolute sense, not with respect to whether or not
it's suitable for grex.  Certainly, a "modern" operating system
wouldn't include an abstraction for emulating a typewriter-based
TTY interface, but that model clearly works well for grex (well,
the next generation of glass text TTY's connected via serial lines
does, at any rate).  And yes, of course people still do use the
wheel.  But we *should* be able to separate what grex does from
what is "modern," and in that sense, very little of what grex does
is particularly modern at all.

As a note, Jeff knows full well that I'm a frequent Plan 9 user,
and that I do consider it quite an improvement over Unix.  It
certainly addresses many of the problems I raised (though it suffers
from others, too).  But, that's just a digression....  Anyway, to
address some of Jeff's specific points:

1) LRU scheduling or page replacement scheduling?  Well, VMS uses
the working set model, as did the EMAS system.  Supposedly Windows
does too, but I've never looked at Windows internals.  I suspect
that might have been part of the original, nice, clean abstract
Windows that didn't make it very far off the drawing board before
it was perverted.  I doubt Solaris does working set, either.

But that's only one problem with the OpenBSD VM system: do they
have a merged filesystem/VM buffer cache, yet?  That makes things
like memory mapped files inconvenient at best (what happens if I
mmap a file, and read from it, and then write to a page that I've
mapped through the filesystem?  Which gets update?  Are the pages
copied from one to the other?  Which way?  Hmm....  FreeBSD solved
this long ago).

2, 3) I'm not sure that I'd consider any credible replacements for
OpenBSD on grex, but Plan 9's model of a central *protocol* for
accessing file (and file-like) resources with the kernel being
something of a protocol multiplexor, is quite nice and much cleaner
than the Berkeley/Sun derived VNODE interface.  8th Edition Unix
had something called the "File System Switch" that consisted of an
array of jump-vectors for filesystem operations, with each "inode"
containing an offset into the array for its filesystem type, as
appropriate.  The VNODE interface tries to be too general, and
that's where it falls down.

4) Dragonfly does a lot of neat things, like fixing the trap problem,
but yes, it is far from perfect.

5) Perhaps.  For grex, it's largely irrelevant since that's the
prevailing and expected mode of operation from the users' stand
point.  If TTY's suddenly disappared, I can think of at least one
grex user who would have a cow.

6) Plan 9.

Regarding #3; VMS is nice, but unlikely to gain traction.  Grexsoft
doesn't matter, it's a dead effort anyway.

I have never been an advocate of OpenBSD.  I think the developers have
an elitist, holier-than-mere-users attitude.  I don't think they really
want the whole world to be running OpenBSD.  They mostly just want to
use it as a vehicle to express their ideology about what Unix systems
should be like.  They are especially ideological about security, and
have introduced and developed many security ideas that I think are very
valuable, some of which have been incorporated in other versions of
Unix. But there are lots of areas where their ideological position is
less developed (like memory management, apparantly) and some areas where
they are so far behind the competition that making any ideological
statement would be hard (like SMP), and those areas of the OS tend to
lag behind.  And things like supporting a huge range of applications
don't really fit into their world view.

For these reasons, not many people run it.  That means less people
looking for bugs and security holes.  Which has a mixture of positive
and negative implications for security, and only negative implications
for stability.

So, I have always thought, and still think, that FreeBSD would have been
a better choice.

However, I am far from convinced that changing Grex over to FreeBSD
would be a good idea.  It would be a lot of work.  And I'd want to know
the answer to this question:

   What substantial advantages would the average user of Grex see
   as a result of a change in the operating system?

Honestly, the users don't care a fig about VNODES, buffer caches and LRU
memory management.  All of these things might improve performance a bit,
but I don't think most users would put "too slow" on the top of their
list of complaints about Grex these days.  Anyway, if you want more
speed, it'd be easier and quicker to buy a faster computer.

Oh, by the way, Plan 9, even more than OpenBSD, is an ideological
statement, not a production operating system.

I have to say I find it rather amusing that a "secure by default" OS suffers
from "not enough eyes to make all bugs shallow".

Another sidenote: I wonder how many of the proponents of systems of yesteryear
spout steam when they hear Unix being held up as the canine's genitals of "a
well-defined systems engineering process"!

re #5
That sounds more like a personality problem than a reasonable assessment of
the technology.  How is security not important?

Not sure what in #5 you're referring to, Todd.

Re: #9. I think the point was that even although OpenBSD is supposed to aim
at being secure, it can't be guaranteed to be more secure than other systems
since it's not as big a target or testbed. Right Jan?

re #11
While its true that advisories and bugtraq reports are demonstrative of
testbeds and white hat hacking, I think its worth noting since most exploits
are roughly based on same.  For the same reasons, a person can brag about lack
of virus problems on OS X.

Security is a measure of configurability and monitoring - we are lacking in
both. 

The ultimate objective with security as it relates to Grex should be to ensure
that resources are managed within an acceptable level of risk.  Some of the
common arguments here are whether maintenance, ease of use, and cost are
aligned with the choice of secure OS.  The question is what does the Board
of Directors deem "unacceptable" and how do we go from there to the
acceptable?  We know what the threats are.  The vulnerabilities are pretty
easy to get a grip on based on OS..but the risks themselves?  Those are the
possibilities.  To me anyway, I think most would agree that the risks are
less with OpenBSD than with most other thing except maybe NetBSD.  Some staff
are squealing that they want an OS with every tweak imaginable which supports
every application imaginable but that contradicts the maintenance and ease
of use argument.  Really, the BoD need to decide whether Grex is going to be
some sort of highly complex monster which requires a few dedicated (read as:
"single point of failure") admins or whether Grex is going to adopt a simpler
platform which offers more sustainability through recruitment and ease of use.

It is not just about ease of use and monitoring.  Anyone who has a 
relatively good command of Unix could do Grex staff duties (if they are 
willing) on just about any Unix or Linux platform there is.  BUT, as I see 
it, there has to be side-benefits/interests to keep technical people 
interested in supporting Grex by offering their time (which is an 
ever-increasing rarity in everyday life).  By this, I refer to an OS that 
is more 'expandable' - i.e. an OS that has architecturally and politically 
positioned itself flexibly to the greater software community.  Probably 
the worst case of such a Unix/Linux-based OS is OpenBSD.  

Hope that sheds some light on the topic -- as I think there are other 
staff members who could immediately relate to this OpenBSD irk. 

re #15
In theory, I agree with you that Grex should want the flexibility of an OS
but in reality its historically significant to point out that the staff is
run tighter than corn rows and the adaptability of its staff to diversity in
the OS is not openly accepted.  OpenBSD presents the easiest out of the box
solution in way of security for this cluster of nepotist marauders.  Nobody
likes to hear the truth but that's been my observation.  Grex can't even get
a grip on email let alone mischevious lil memory leak gremlins so opening the
system up to a flexible OS is just inviting all sorts of available exploits
and hacks to occur under the not-so watchful eyes of this overburdened staff.

I strongly disagree - and, note well, I'm coming from both an inside 
staff perspective and security background. 

For starters, if you call a system which has gone down as much as Grex 
since we have moved to OpenBSD 'secure' you have missed the fundamental 
principle of security - availability of service.  We went down much less 
often when we were on our last operating system - and were also down for 
less time on the odd ocassion we were down.....  OpenBSD is by no means 
necessary for locking down Grex, and is actually counter-productive -- 
worse still is the attitude of those who think it is necessary for our 
perceived security needs!

The biggest, by far, reason why staff has gone missing in action in 
large numbers on Grex is this IGNORANT cowboy (e.g. OpenBSD) mentality.  
I can speak personally that I have lost a lot of interest, and would 
contribute a lot more if we actually were not so close-minded and 
pompous.  I'm sure Dan Cross knows exactly what I am referring to, then 
there's Valerie who left also because of a personal attack.  You can't 
treat good people like crap and expect them to continue volunteering their 
time.  Rob Henderson and Steve Weiss also used to be big contributors, but 
probably also have lost interest (possibly for similar reasons).

re #17
 I strongly disagree - and, note well, I'm coming from both an inside
 staff perspective and security background.
Ok, I'll note your self proclaimed perpsective and background.  Note that I'm
also an oldschool Grexer and have an extensive security background.
(Why credibility is even being approached in your response seems weird.) 

Your claim that Valerie left due to personal attack redacts facts about her
actions which were out of place and unprofessional.  Also keep in mind that
Cross split because he was being micromanaged and treated unfairly by certain
longtime Grex authorities.  Read as: Nepotism ruled both situations 

I don't know where you originated your definition of security principles as
being "availability of service", either.  That may be a security strategy but
certainly not a principle.  Principles would be things like cost
effectiveness, accountability, change control/integration, etc (Mind you, I'm
citing NIST 800-14 off the top of my head as a DoD certified IAM.)

 The biggest, by far, reason why staff has gone missing in action in
 large numbers on Grex is this IGNORANT cowboy (e.g. OpenBSD) mentality.
And here I thought it had to due with the staff physically local to the Grex
machine micromanaging the true troubleshooting and decision making.  Silly
me.  OpenBSD is just a periphery discussion (unless you are close minded.)

Let's address what you really intended to prove:
 We went down much less
 often when we were on our last operating system - and were also down for
 less time on the odd ocassion we were down.....  OpenBSD is by no means
 necessary for locking down Grex, and is actually counter-productive --
 worse still is the attitude of those who think it is necessary for our
 perceived security needs!
1) Same hardware when you had less downtime?
2) SunOS specialist did major bugtracking in each OpenBSD server abend?  Do
you have the utmp and other logs available to show the true causes?
3) Is it "attitude" or "opinion" which shows preference? (Again, you're 
mixing your definitions.)  If you are truly getting upset then perhaps you
should not participate in debates about open systems with open minded people.

You're sooo old-school, tod.

<stands naked behind HP2628A smart termial and waves>

If a system is not available (and their is no contingency), you are 
screwed.  Whether it is OS, hardware, or lack of staff interest being the 
cause (immediate or indirectly) does not matter.  

Security 101 acronymn - CIA: 'Confidentiality, Integrity, Availability'.
The first strategy of attack an adversary will do against you to screw 
your business is a denial-of-service attack against system availability.

---> see http://en.wikipedia.org/wiki/CIA_triad

Tod: I stand 100% behind my comments that you responded to.

 The first strategy of attack an adversary will do against you to screw
 your business is a denial-of-service attack against system availability.
I dunno...bots and local privilege exploits have historically been the
strategy most deployed on Grex.   I haven't seen anything about DDOS beyond
the fubar sendmail processes.  (I agree lack of staff interest is a big piece
of the puzzle and 'reactive' responses or sketchball installs have played a
part rather than proactive design.)  If there's some magic config out there
we don't know about it would've been great to see staff using it.  So far as
I know, OpenBSD is setup as such from the get-go thus quirky guru staff (which
haven't always been available/interested) wouldn't be such a single point of
failure.

Regarding #6; I don't think Plan 9 has ever tried to be a production operating
system.  I'd classify it as a research system, where a lot of proof-of-concept
work has been done.  However, there are those who point out that it has been
used extensively in production systems (phone switches and the like).

Regaring #5; I think FreeBSD would be a better choice, too.  Interestingly,
I think it would be a lot easier to transition to FreeBSD from OpenBSD than
it was to move from SunOS 4 to OpenBSD.  I think the advantages of doing so
would largely be independent of speed or OS efficiency, though.

The problem, and let's be perfectly frank here, is that security on grex is
important, but not the overarching goal.  OpenBSD was chosen by a couple of
the "quirky guru staff" members that Todd mentions because they felt it was
"more secure" than FreeBSD who acted as if security should be the *number one*
priority.  They strong-armed the decision and pushed it through, despite some
misgivings from other of the staff.  One of them is no longer involved with
grex.

And let's be even more perfectly frank: the choice was and is *only* between
OpenBSD and FreeBSD.  Whether this is technical or just plain bigotry is
irrelevant, but it is what it is.  Some people might pretend to make an
"objective assessment" but those are the only two systems that will ever be
considered (unless someone can figure out how to go back to SunOS 4).

But the reality of it, from what I can tell, is that both are about equally
secure, with OpenBSD being more pompous about their "high" level of security.
But in terms of actual levels of exploits, they're about on par, with perhaps
FreeBSD being a little better (and a bit more candid; that comes from being
less egotistical).  However, FreeBSD gives you many more choices in providing
interesting applications and actually building software, is better tested and
better engineered, and has more of a future ahead of it.  OpenBSD talks the
talk, but doesn't really walk the walk any better than anyone else when you
get down to the nuts and bolts.

On grex, there's an infuriating tendency to do things "just because" or based
on some emotional reaction ("I *feel* that this is more secure/better/
whatever...") versus based on technical merit.  There's also a tendency to
defer to the historical guru's on nearly all matters.  That's why OpenBSD was
chosen, not due to its superiority with respect to security or anything else.

Microsoft Windows 2003 Data Center isn't an option for Grex? *duck*

re #24
So am I done with Devil's Advocate here or should I keep rooting OpenBSD? 

What about OpenSolaris? If not, why not?

Ever tried to install it on x86 hardware?

Regarding #26; No, someone should do it.  By all means, go ahead.  If you make
someone scratch their head, that's a good thing.

Regarding #27; Becasue, frankly, no one is interested in OpenSolaris.

re #27 
I had coffee with Peter Gregory this morning so maybe I could talk him into
coming here to tweak it.  I'm sure he'd be glad to do that right after he
reports this zoo to the CDC for its obvious threat to the Internet's mental
health.

Grex is too insignificant for that.

I'm just a bbs user.

i'm a party pooper

Re: #28, 29, correct, despite protestations au the contraire.

   So, I finally have extra free time to respond to this item.

   First, I'll agree that the two only real choices for Grex are
OpenBSD and FreeBSD.  I actually like FreeBSD, and its absolutely
the second best op system I know of.  Had OpenBSD not been around
I'm pretty sure that both Marcus and I would have advocated that.

   The comments Dan has made about the faults in OpenBSD are
interesting, so I'll comment on them:

 - Potential admins less familiar with it.  Thats true to a point,
but the biggest differences in maintaing the system are the lowest
level things, like disk management.  I'm not sure that counts for
much, because anyone who wants to be staff (or a system administrator
in general) should be versed in multiple OS's.

 - Differences between FreeBSD's and OpenBSD's ports system.  Yup,
thats right, and the changes in the ports system by Marc Espie has
been nothing short of incredible.  You can update a bunch of packages
now with really complete checking of dependencies, such that everything
is updated correctly.

   - Significantly fewer ports available.  In terms of raw numbers,
yes, OpenBSD trails, but I think the overall quality of them is very
good, and for Grex, most of what we have been using is already compiled.
The 4.0 package collection consists of 3600+ items for i386.  Are there
other things which Grex might benefit from having that aren't in the
tree right now? Sure!  But we can port them, and its a whole lot easier
to get stuff to OpenBSD than SunOS was becoming.  So not perfect but 
very good.

 - Deficient Java.  That is just flat out WRONG.  Kurt Miller has done
a great job of keeping Java 1.3, 1.4 and 1.5 working.  Java and OpenBSD
is an excellent combination.

 - A less stable system because of overall less testing.  Again, I think
that is just plain wrong.  A lot of testing goes on for OpenBSD by a lot
of people, and when you don't have hardware problems, OpenBSD is just
rock rock solid.  My main webserver at work will be seven years old in
a couple of months with zero problems, and less than 4 hours of downtime
in that period.  My own testing of OpenBSD, keeping more than 100 processes
running continuously for more than 3 months last year proved to be more
reliable than MSUs power. Are there bugs? Again, sure.

 - The engineering is less polished.   I'm not sure how to respond to that.
I certainly don't think it is, but I'l quite willing to talk of specifics.


   One of the reasons Marcus and I pushed for OpenBSD was to be in an
environment of trying to do things "right" over speed and flashy items
was important.  Security comes from a lot of things, but attention to
little details, like string manipulation is important and provides
benefits that aren't readily forseen, ie fixing problems just because
something was done in a bad way.  This means that various items in OpenBSD
aren't as advanced as in FreeBSD or others.  Dan's examples of 1, 2 and 3
are correct in that there is room for improvement.  I can't agree that
SMP support is "weak": I've seen multi processor systems and they
scream.  They'd scream even more with kernel threads, yes indeed.  Marcus
has bemoaned the lack of this for a long time. 6 starts to get into
areas where I can't comment that much on; I'm learning about more
kernel stuff.  Suffice it to say that I'll agree that there is room
for improvement.

   Commenting on this comment:

>#13 of 34: by Michelangelo Giansiracusa (spooked) on Sun, Sep  3, 2006
(07:18): > If Grex moved to a more modern, flexible operating system you may
see > more staff members interested.  There are no guarantees, however! > > The
fact that OpenBSD was pushed for its 'security hardness' has strained > the
interest of at least a few current and former staff members.  And, > given
that, we have seen it has hardly proven a fortress either!  Being > a security
expert, I can tell you for a fact that most security issues > are the result of
poor configuration and lack of diligent > maintenance/monitoring --- something
we have been lacking.  OpenBSD (and > its push by a couple of former/current
staff memebers) has lost not only > the interest of others in terms of
participation, but also limited our > software base and chance of
modernisation.

I am utterly perplexed as to this statement.  I don't see how OpenBSD could
be much more "modern"--hardware support in 4.0 has increased dramatically
with lots of new ethernet cards, Wireless cards, SATA and IDE componenets/
controllers supported, including some "1 Wire" devices, SD cards, audio
and GPS devices.  There was a Slashdot article recently where some Linux
people were decrying the fact that OpenBSD better supported wifi than
they did.

So although a lot of the above isn't relevant to Grex, a whole lot of
modern hardware is supported.  POSIX complance is very good.  The "man"
pages are at the very top if not the top in terms of quality and accuracy.
I've given OpenBSD to many people and have gotten at least 10 comments
about how good the documentation is.

In terms of expandability, I don't see how Grex would EVER touch the
limits that it has.  SCSI U320 controllers are supported, which is why
I spec'd out U320 disks -- we can have better disk throughput for less
than $300 if we wanted.  PAE mode is now supported, so how much memory
would you like on Grex?  8G? 16G?  We could do that (which would be nuts).
AMD64 support is excellent as well, meaning Grex could jump to that
platform far easier than our jump from SunOS to i386 OpenBSD.

I suppose you can fault the 1TB disk limit that OpenBSD's ffs has, but
again, Grex doesn't need that kind of storage.

So in terms of hardware flexability, there aren't any issues for Grex.

Software is of course another item, and here we've crashed into some
problems.  There is a bug in the soft update code.  *I* can't duplicate
it on some test systems I have, but Grex has bumped into it.  Around
the time that we had some crashes relating to this I went looking in
the FreeBSD CVS repository and found that they've made changes to
the code, and I've found a person who has seen some of the problems
we've had with an earlier version of FreeBSD.  Thats an issue, but
after turning it off we didn't notice a difference.  The other issue
may be quotas.  I'm not sure about that and need to investigate that
more.

pulling a paragraph from the quote:
> given that, we have seen it has hardly proven a fortress either!  Being
> a security expert, I can tell you for a fact that most security issues
> are the result of poor configuration and lack of diligent
> maintenance/monitoring --- something we have been lacking.

This is only partly true.  If the OS has problems, then no amount of
diligence is going to keep you safe.  My example there is Windows.  It's
definitely true however that if you don't do things right you're going
to have problems.  OpenBSD isn't and can't be any different in that
regard. I applied a patch to the kernel in Janurary; there are 
possibly others that should be, but on the other hand there are no
actual exploits for any of the problems listed in the securtiy updates
page, and if anything exists which can screw up OpenBSD, it gets 
plastered very loudly: witness the OpenSSH bug in version 3.1.  This
is NOT to say that we shouldn't be better than we have been about things
like updates.  We need to upgrade to 4.0.

Something to remember is that the vast (better than 90%) Grex's crashes
have been due to faulty hardware, ram.  I take a large amount of the
blame of that because I didn't correctly sniff the crashing to be ram
and I should have, as I've dealt with it before.  Lately, Grex has had
uptimes of 72 days and currently 40 days.  We didn't have enough process
table space which has since been corrected.  Now that this has been
changed, Grex is stable.  I note that some of the people who've complained
about OpenBSD have been strangely silent on this issue.

 I believe what mic mostly means by "modern" is "good Java support."

   Well, it does have that.

It's the nature of grex that you can't easily see what you're responding to as
you respond.  So, I'll respond to Steve's posts in parts, so that I can
actually see what I'm replying to.

I'll first thank Steve for responding.  I always find these technical
discussions (really, they're more like debates) on grex fun.  But that's just
me....  However, I do find that his post reads more like an advertisement for
OpenBSD and less like a seriously thought out technical argument.  But more on
that as I go on.

The problems for potential administrators isn't just confined to low-level
interfaces; in fact, I find that the from the sysadmin point of view, the disk
structures are nearly the same (except that FreeBSD got rid of block I/O
devices a while back).  While it is indeed good for a sysadmin to be familiar
with more than one system, as I think I acknowledged when Jeff posted it, it's
also useful for a system to conform to standard interfaces to minimize
differences.  For instance, on grex, the BSD login.conf mechanism is used to
deal with grex's weird password hashing algorithm.  But this is different
from, say, the PAM system used on FreeBSD and almost every other Unix-like
system out there.  So administrators who want to deal with Grex's
authentication system have yet another - and in this case, not so trivial -
interface to master.  Is this bad?  Maybe not; but in a disaster, it might be
a real achilles heal.

OpenBSD's ports system is nice when it comes to the updating stuff, but I'll
say that (a) this is relatively recent (like, OpenBSD 3.6 or  3.7 or
something) and certainly wasn't a consideration when grex was switching to
OpenBSD several years ago.  However, lack of port coverage, or outdated ports
certainly was, and certain applications (like aspell) were ported to grex by
hand instead of using the ports collection.  Consequently, the process for
putting grex together required many manual steps that, on FreeBSD, could have
been automated.  Why expend precious staff time and energy porting
applications to OpenBSD when they're already ported to FreeBSD?  And the
OpenBSD 4.0 packages collection isn't particularly useful to look at, since
OpenBSD 4.0 hasn't been released yet (though, if I'm not mistaken, it should
be out soon).

If Java support is so good, why hasn't it been in the ports collection until
recently?  Users have asked for Java since this version of grex went online,
which has been over a year (i *think* nearly two now).  Attempts to install it
from ports haven't met with much success - because it hasn't been in ports.
Indeed, it's not in /usr/ports on grex right now.

And of course OpenBSD is less tested.  Quite simply, FreeBSD goes through more
cycles in a day than OpenBSD probably does in a week or maybe even a month.
That's just more testing, whether formal or not.  And experience with grex has
shown that OpenBSD is anything but "rock rock solid."  Indeed, the softupdates
related crashes weren't due to hardware by any accounts, and yet the system
crashed more than once from it (sometimes staying down for very long times
indeed).  Since moving to OpenBSD, we've seen several extended outages that
were not related to hardware bugs: this is not the hallmark of a reliable
operating system.  Now, it's true that grex could have benefited from more in
the way of server class hardware (and I never understood why, after all the
noise that was made by Marcus about ECC memory, non-parity memory was
purchased instead), but it's also impossible to deny that the software has
been unreliable at times.  Empirically, Steve's claim that the software is
reliable enough for grex have already been shown to be false - on grex itself.

As for the engineering being less polished, this is my sense rather than
something I can formally define.  My impression is that there's a small group
of individuals who guard the source base diligently.  They are quite "good" at
what they do in some senses, but leave out things like formalized process,
setting project milestones and tracking them, etc.  On the other hand, FreeBSD
actually has something resembling a process and a design guiding their
implementation.  OpenBSD is more, "hey, I thought this was cool, so I made it
happen...."  My experience is that that can work, but only to a point, and
after a while, you hit a critical mass point where the model breaks down and
you need to impose some sort of order onto the process.  OpenBSD just isn't
there.

With respect to SMP being deficient, I was specifically referring to the lack
of fine-grained locking in the kernel and the fact that most of the kernel can
only run on one processor at a time.  This inherently limits scalability in an
SMP setting.  This is what Steve refers to as, "kernel threads": a multi-
threaded kernel.  Unfortunately, problems in Unix's basic design are so
deep it requires massive effort to get a truly multithreaded kernel going
without starting from scratch.

Given the above, I fail to see how OpenBSD is doing things "right."  So they
audit their source base looking for string problems and using some sophmoric
routines they wrote to fix them: so does the FreeBSD team (indeed, members of
the FreeBSD team read the OpenBSD commit logs and track those changes, too,
applying them to their own source base!).  Some of the responses Steve makes
to my issues with OpenBSD weren't valid at the time OpenBSD was chosen for
grex, and have only recently become factors: ie, OpenBSD is only now catching
up.  Hmm.

Yes, OpenBSD leaves lots of room for improvement, but what's the point of
sticking it and waiting for them when FreeBSD already gives you all the
benefits today?  And why, *really* was it chosen over three years ago to
run grex?

STeve mentions that "FreeBSD is the second best [OS]" he knows of. What I'd
like to know is, what's wrong with NetBSD, either as GrexOS or in general?

   Why was it chosen?  In just a few words, paying attention to security
issues, starting with strcat().  I'm in a meeting right now. More later.

But FreeBSD does the exact same thing.  Strlcpy() and strlcat() are certaily
in libc on FreeBSD, and as I said, the FreeBSD team tracks the OpenBSD source
changes - as well as doing their own auditing.

How about a comparison of exploitable bugs between the two since the time grex
has been online running OpenBSD?

   Right, but where did they come from?  OpenBSD.  FreeBSD *does* do a whole
lot right.  I have said before and will say again that its a great operating
system, and absolutely the second best choice for Grex.  You might be surprised
to learn that I steered a friend from OpenBSD to FreeBSD, given the nature and
load of a web site being spec'd out.  But for Grex, I think--I know--that we
need every bit of help we can get.

   A comparison might be interesting, but there is more to things than
a numeric count.  I'm running out of time right now, sigh.  A lot of
the external things (sendmail, bind, openssl) are going to be very
similar, so I'll bet that the majority of interesting differences is
going to be kernel/library things in each os.

What Dan says a couple of posts back I have said previously in a shortened 
version, and I agree 125%

Grex does not need OpenBSD (even for security)... and OpenBSD has proven 
to be horrendously unreliable.

Oh, I only have to laugh (because it beats crying) your comments of 'full 
Java support'.

And, I'm rather peeved - let it be said Steve that you promised me when I 
first joined Grex staff to show me the reeping process (and despite many 
reminders) you never did.  That is not even mentioning your bias and 
zealous towards OpenBSD...

Regarding #43; You totally discount FreeBSD's own auditing efforts.  They
don't *just* "get it from OpenBSD."  And I'd say that their auditing effort
is roughly contemporaneous with OpenBSD's and has been going on for years.
They even audit many of their ported applications, and release security
advisories for insecure ports.  Can you point to any independently verifiable
data that says that OpenBSD is more secure (by any metric) than FreeBSD?  Or,
again, is this your intuition?  I'm not saying that intuition is bad, but
given OpenBSD's less than stellar track record on grex, I think some
justification beyond, "they do it better" (which has empirically been shown
to be false) is warranted.

Regarding #36; Jan Wolter, who did much of the early work preparing grex to
move to OpenBSD, found that the documentation was actually quite poor.  Either
things weren't documented or, worse, were documented incorrectly in many of
the programming interfaces that mattered to us.

With respect to stability now: Sure, now that the bugs in the OS have been
worked around, it's fairly stable.  Now that we've upgraded, we've fixed the
issue where users could cat /dev/tty* and see passwords being typed in.  Since
turning off softupdates (a very useful feature for a system like grex) the
filesystems hasn't caused us any crashes.  Since increasing the sizes of
various tables in the kernel, the system hasn't paniced from having too many
processes running (why would that cause a panic, anyway?).  It's clear that
we can *make* OpenBSD relatively stable.

But why doesn't anyone ever ask the question of WHY we have to do that in the
first place?  The other operating system under review does't seem to have
these same issues, so why patch OpenBSD when we can just run something that
doesn't crash?  I would *really* like to see some sort of sound technically
oriented answer to this question.  If you want to say that FreeBSD would crash
in the same way, I'd like to see proof: my contention is that it would not.

As for the bad RAM....  Originally, when the question was what platform to
move to, one of the major objections that Marcus had to i386 was lack of ECC
memory.  I continually responded that one could get ECC memory for the i386,
one just had to choose the right motherboard.  Regardless, I think it was you,
Steve, who suggested non-parity memory.  In retrospect, this seemed like a
very bad idea.

Mic, how the in the world is OpenBSD "horrendously unreliable"?  We had 
hardware problems earlier.  NO OPERATING SYSTEM IS GOING TO WORK WELL WITH
BAD MEMORY.  Say whatever else you want about OpenBSD, but to so completely
misrepresent the stability problems we've had as endemic to the operating
system itself shows that you either don't understand the issues here, or
you are really blinded by your dislike of OpenBSD.

We've had crashes due to hardware problems, a failed disk, an obscure bug
in the soft dependency code and kern.maxproc not set high enough.  I still
can't reproduce the softdep bug myself, and that I know of no one else has,
else it would have very likely been in the OpenBSD bug tracking system.

Every OS needs to be tuned to some extent.  Now that we've done that we've
had uptimes of 72 days, and currently 42 days.

That is "horrendously unreliable"?  Give me a break.

   As for my not showing you the reaping process, I guess I never did.
Thats my fault, though I'll point out that crabbing at me more might
have done something (maybe).  Still, I should have done that, and you
are right.  As for Java, take a look at the current state of Java and
tell me what is missing.  Yes, its 4.0 and not what we're running
right now, but its there.  Yes, we need to upgrade.  The open source
world is a moving target.

  Re #45: I do not totally discount FreeBSD's auditng efforts.  I keep on
saying--and you don't seem to hear me--that FreeBSD is a very good operating
system, second only to OpenBSD.  I'm getting tired of having to say this.
If Grex magically appeared on a FreeBSD system in the next hour, it would
work.  I hope I'm getting through.  It's a matter of degrees.

   Crap, I'm out of time at the moment.

Regarding #47; Okay, tuning: but it only took us two years to get there.  We
still don't have enough TTY's, and I still don't understand why a system would
crash just because the proc table filled up.  Then there's the tty security
problem, the system logger security problem, and while it may be difficult to
reproduce, there's no denying that the softupdates bug is an actual operating
system problem (and has caused multi-day downtimes).  But no one's ever seen
it in FreeBSD, at least not during the time that grex has been running
OpenBSD.  Some of this is configuration (which just goes to show that even
talented system administrators working with supposedly secure software can
make mistakes or overlook things).  As for bad memory, yes, what you say about
it is true - if it was really the problem - but then the decision to buy
non-parity memory after so much stink was made about ECC memory during the
hardware debates was clearly a bad one.  And, after all this time, I'm still
not completely convinced that the memory module is really "bad": it may have
just been poorly seated, and it may have been a bug in the OS, which was
upgraded at around the time the memory module was removed - perhaps the bug
was fixed by the upgrade.  Or perhaps it was something totally different.

For the record, the decision to buy a mid-tower case instead of a rack mount
case when grex moved into colo shortly after moving to the present hardware
was also bad.

I'll note that with the exception of the hardware related problems, the rest
*may* have been completely avoided by running FreeBSD from the start instead
of OpenBSD.  We'll never really know.  Would I describe grex as "horrendously
unreliable"?  Hmm, that'd be stretching it, but it was pretty shakey there for
a while, and at least some of that was OpenBSD - supposedly chosen for its
reliability and security.

Regarding #48; OpenBSD 4.0 hasn't been released yet.  Java has been running
under FreeBSD for years.  What's missing?  Any sort of semi-modern JDK in
/usr/ports on grex right now, in September, 2006.  Java is older than OpenBSD,
OpenBSD is something like 9 or 10 years old, and they're just now getting it?
Linux has had it for ages, as has FreeBSD.  I don't see how you can point to
what they're just doing, now, and use it as justification for a decision made
three years ago.

Regarding #49; That's an opinion that you posit as fact, and then ignore the
parity between their ongoing security efforts when you make statements to
support your opinion.  I'm refuting that, and asking you to give me a
substantive argument that justifies your opinion.  Repeating, "FreeBSD is very
good, but not as good as OpenBSD..." over and over again will not convince
me.  This is not the Bush administration, after all....  :-)

The check is in the mail with your reaping process and FreeBSD, m'kay?

Heh.

  Dan, take a look at the cvs logs for FreeBSD.  They've had some problems,
I know that.  I'm not sure that it has the same problems as we've had, but
I did briefly talk to someone who had a problem with soft deps on a large
database system.  So I don't think its fair to say that FreeBSD's
implementation is perfect.  I haven't seen the problem on any of my machines
either, so it isn't something easily produced.

   The logger problem is a policy issue; syslog from users is normally
trusted.  Grex can't do that.

With all due respect, STeve, if someone reminds you more than a few times 
(and is promised that he will get it in a short period more than a few 
times, and never got it) they just give up and do it for themselves 
because of a lack of respect.

When I do stuff, like programming, build scripts etc through my work -- 
I always document as I'm doing the activity.  Any responsible IT 
professional does this - if not, they are hardly worthy of being in a 
team in my opinion.  

   True enough, documentation is a good thing.  The reaping process,
like many of the other things done to maintain Grex was done by
various folks and it kind of evolved over time.

And making use of it without documentation is considered to be statutory reap.

   Re 50: Grex was shakey there, and we warned people there would be a 
transiton period before things were rock solid.  You should have seen
Grex when we got on the Sun-3 and then with our new 2G disk: we crashed
about every other day, taking files with it. Ick.  The worst problem we
had after moving to OpenBSD was the memory problem.

   As for Java, had we been good about upgrades we'd have been in a far
better position re Java.  This isn't to say that OpenBSD support isn't
more latent than FreeBSD's, because its true that such support came late
for OpenBSD.  However, I can't recall people asking for Java 'till after
we were here, ie it wasn't one of the things that we were worried about
when considering how to leave SunOS.

Regarding #54; I never said FreeBSD was perfect.  But I am implying that these
things are more mature on that platform.  Part of that is that it has a larger
user base, so snarky problems (that are difficult to reproduce, like grex's
issues with the softupdates code) get exposed more rapidly - and thus fixed.
Further, since more businesses depend on FreeBSD, there are more loud voices
(and willing eyes) joining the chorus to have these problems fixed.  But
that's what you get when you have a more mature and larger installed base in a
server oriented system.  Linux could have this, too, had they not adopted the
hacker of the month model for development (I remain largely unconvinced that
Eric Raymond's "bazaar" model really works beyond a certain threshold of
complexity and size).

Regarding #58; Two years is quite a transition period.  But then, had we run a
system that more sysadmin types were familiar with, would it have taken two
years?  And that's part of my point.  Perhaps the *worst* problem we had was
the memory problem, but it certainly wasn't the *only* one, and that it has
taken two years to get to the point where problems with the systems software
have been masked speaks to the unsuitability of that software for the task at
hand.  I earnestly believe that we could have done better.  Whether that is
true or not, the data shows that we didn't do all that well (at the least,
grex should have gotten ECC memory - about which all the stink was made).

I'm not sure how being better about upgrades would put us in a better position
with respect to Java.  On *my* OpenBSD system - running the latest stable code
(that is, 3.9-stable from about two weeks ago), Java doesn't exist in the
ports collection, either.  Surely you aren't suggest that we run the
unsupported, unstable developer -current tree, which is what OpenBSD 4.0 will
be forked from once it stabilizes?  This cannot be what you mean by, "being
better about upgrades."  What's more, I cannot find credible your (implied)
claim that Java support on OpenBSD will be as mature as on FreeBSD once
OpenBSD 4.0 is released: Java has run on FreeBSD for years.  It is quite new
on OpenBSD, and simply cannot be as well tested.

What's more, as you know, many Java programs rely heavily on threading for
concurrency.  The Java threading implementation, in turn, exists on top of the
POSIX threads suite.  That suite under OpenBSD exists entirely in a user-land
library, with no support for thread concurrency from the kernel.  This means
that, if a process should call some sort of blocking system call, every thread
running in that process will hang until the operating system returns from that
system call.  On FreeBSD, threads are (if I'm reading the man pages right)
scheduled by the kernel (they also have a user-level threads library - great
for applications that do lots of thread switching with little chance of
blocking).

Having programmed in Java, I can say that I don't care for their threading
model: it's an architectural throwback to the 1960's, critical section
protected, cooperative coroutine model that's too low level and not as
expressive as, say, CSP-based concurrency systems.  But, the fact is that Java
threading just won't work as well on OpenBSD.

As for the logger problem, OpenBSD touts itself as "secure out of the box" and
takes a stance of not really trusting the local user.  Why, then, not protect
the system logger, which could be used for a denial of service attack, at the
very least?  But I see your point that it's a policy issue rather than an
actual software flaw (I'd say it's more of a configuration issue).  MY point
is that we chose this system for security, and then failed to adequately
secure it.  No system, no matter how good or bad its security, is going to
remain secure if its configured insecurely (how's that for a tautalogical
statement?).  Since we didn't properly secure grex on OpenBSD, what then, was
the point of choosing OpenBSD based on its security record?  I'm just not sure
I buy the security argument, and I remain unconvinced.

Steve, if there are specific points on which OpenBSD does demonstrably better
than FreeBSD (with respect to security or anything else), I'd really like to
hear them.  So far, all I've heard is impressions and opinions, with little
recourse to facts of measurable data.  While those things are fine for
starting to form an opinion, major decisions should not be based on them, and
opinions and intuitions should be subject to change based on observed
behavior.  In the absense of sound technical argument in favor of OpenBSD, I
simply don't see how the decision to go with that operating system can be
objectively justified.

With respect to Java, I specifically remember Mic asking for it when the
discussions of what operating system to go with post SunOS 4 were being held.
Item 123 in the old Garage conference has postings from October, 2003 where
it's mentioned.

And for the record, lest it be said otherwise: I'm not "against" OpenBSD.  I
run it on several of my own systems.  But I really want to understand why it's
appropriate for grex, and I really don't understand that and don't believe
that it is.

   Java will never be in the package repository, unless Sun changes their 
license.  So you'd have to build it yourself.  As for the reliability of
Java on OpenBSD, go into the misc@ archives and see comments from Henning
Brauer, one of the developers, for his confidence level in Java.  Others
have made the same comments as well.  Will it be as fast as on FreeBSD?
I don't know, but likely not.  OpenBSD has never been about speed, so I'm
not concerned about that.

   I'm puzzled why you don't see policy issues being something that can't
reasonably be set for everyone.  syslogd.conf is something that darn well
ought to be looked at and adjusted accordingly.  I don't think anyone on
staff had thought about the current toxic users who'd exploit this, I
know I certainly didn't.  I might also point out that the systems security
remains unchanged by that--messages to the screen and into a log file
might be annoying, but not threatening to the system.

   Yes, we took too long to get Grex stable, I certainly agree.  But I'm
not believing for a minute that FreeBSD would have magically made anything
better.  I heard the same exact thing years ago when we were on SunOS.
What Grex needs is more local staff who is able to deal with problems
as they arrise.  We also need 24 hour access to the system.  There were
several times when I would have dealt with problems at night when I'd
gotten back to Ann Arbor, but I couldn't do to the 10pm curfew that is
imposed on us.  Grex has never been exactly efficient in certain things.
I also flatly disbelieve that there are many people on Grex who'd be
willing to do systems work on Grex if we didn't run OpenBSD.  The two
most vociferous opponents to our current op system don't live here.

More when I get back from a work thing.

I'm not talking about the package collection, I'm talking about the ports
collection.  The former is pre-built packages, the latter are built from
source (and he mechanism by which the former are created).  The threading
issue isn't just about speed, it's also about reliability and expected
behavior: if I have an application that utilizes n threads, and I expect that
n - 1 of them will ccontinue running even if 1 of them blocks on a system
call, and then I try to run it on OpenBSD and it doesn't work right, I won't
be happy (particularly if the Java spec or something says that it should keep
running).  Surely things like this *are* cause for concern?

The problem with the logger had nothing to do with syslog.conf, it had to do
with the fact that people could write to the log socket.  It could be masked
by judicious tweaking of syslog.conf (which, you are right, surely should be
modified to be brought into accord with local system policies), but that's
just treating the symptom, not the disease.  Personally, I don't think that
ordinary users should be able to write to the logging socket *by default*:
If I want to enable it, that's another matter.  And I would further suggest
that it *is* a systems security issue in that it can be used as the basis for
a denial of service attack.  Random junk spewed into the logs can also mask
actual system attacks, and greatly reduce the value of information in log
files.  If the system log rotation policy favors size over time when deciding
to rotate a file, real information may be lost.

Perhaps FreeBSD wouldn't magically have fixed all of grex's problems, but
we'll never know since we didn't try it.  I submit that several of the
problems we experienced with OpenBSD wouldn't have shown up, and if they had,
there would have been a larger user community to draw from to get them sorted
out.  As it is, we still have OS related problems: Grex isn't configured to
support enough terminal devices for the number of users who want to use it.
No one seems to have figured out how to get OpenBSD to allow more.  Hmm.

I think that it's telling that Jan Wolter, who did the majority of work in
getting grex running under OpenBSD, ended up not liking it very much that
experience.  See, for instance, item 170 in the old garage conference.

A correction to my earlier posts: it appears that Java *is* in OpenBSD's ports
collection, even on grex right now: /usr/ports/devel/jdk.

I think Grex needs less local staff considering the pool of volunteers is
likely to come from abroad.  Spooked and cross are on opposite sides of the
planet but are well equipped to patch and lock down FreeBSD.

Hmm. How likely is it that volunteers are more likely to come from abroad than
the US, really?

Given that the pool of local volunteers is approaching 0, the chances of
finding volunteers from other places seems to be more likely each year.

I'm not sure volunteers need to come from abroad (I mean, not that it's a
problem if they do - I'm sure they're certainly welcome and I'm not trying
to imply otherwise), but it's clear that those that are coming from Ann Arbor
itself are getting fewer and fewer.

Todd is right.

re #66
Silly man.  Abroad=anywhere outside the Planet of Ann Arbor (aka perpetual
high school mentality land)

My bad.

Mine too.

Dan Cross "cross": 

--> "reasons I believe one may consider OpenBSD inappropriate
for grex and/or "not modern."...(blah-blah-blah)..."deficient Java
support on OpenBSD"

--> "STeve andre" in #35: 
"Kurt Miller has done a great job of keeping Java 1.3, 1.4 and 1.5
working.  Java and OpenBSD is an excellent combination."

--> cross #39: "Indeed, it's not in /usr/ports on grex right now." 

--> cross #50: "What's missing?  Any sort of semi-modern JDK in
/usr/ports on grex right now"

--> cross #59: "On *my* OpenBSD system - running the latest stable
code(that is, 3.9-stable from about two weeks ago), Java doesn't exist
in the ports collection, either." 

But
--> cross #62: "A correction to my earlier posts: it appears that Java
*is* in OpenBSD's ports collection, even on grex right now:
/usr/ports/devel/jdk." 

Err, that was why I made the correction...?  How does this address my other
points?

(For instance, grex was off the network for about 8 or 9 hours last week. 
We never saw any sort of explanation as to why....)

People talk about security in the abstract when discussing projects like
 OpenBSD without asking the two critical questions:

    1) what resource or value are you trying to secure?
    2) What threat are you trying to secure it against?

If Cyberspace Communications BOD is interested in, for example, 
protecting availability of the forum and the near-real-time chat-room, 
there are radically different concerns than if they were interested in 
protecting the confidentiality of messages, either at-rest or in-flight.
 Does the BOD give guidance on what to protect, and from whom? Without 
answering those two simple questions, discussion about security is at 
best supposition and partisan shouting.  -DTK 

Short answer: no.

But Grex is going to be moving away from OpenBSD and to FreeBSD on the next
revision of hardware.

I thought FreeBSD (or anything that does not have the look-and-feel of 
pre-SystemV SunOS/OpenBSD) was anathema here. Huzzah! Sounds like a 
porting nightmare for all of the weird custom codes that are run here. 
- DTK 

Yeah.  We're moving to a model where it's kind of understand that if you want
to force policy, you have to be involved.  Porting will be a pain, but at
least we'll correct the mistake of using OpenBSD in the first place

So that future ports and upgrades are less painful, would it make sense
to  put at least the customization into CFEngine or the like? This would
also  make the system more self-documenting. I know it makes more work
up-front,  but after the first upgrade, it has paid dividends.  -DTK 

I think if we were running across multiple machines, something like cfengine
or puppet would make a lot of sense.  But given that we're just on one
computer, and likely will be forever, I'm not sure it's worth the hassle. 
We have a set of home-grown scripts for doing things like that (basically,
a big Makefile) and all of the customized files either live in a dedicated
directory hierarchy, or are in a Subversion repository.

That makes sense (although CFEngine runs easily as single, stand-alone 
node). 

Yeah.  I need to give cfengine another look.

• Read the next item
• Go to the menu of items for the Agorage conference
• Go to the list of conferences
• Go to the opening screen