response 1 of 24:

Feb 17 08:16 UTC 2003

I have another proposal: keep the system source online, and rebuild the
system from source periodically.  If you track one of OpenBSD's stable
release branches, this is more like throwing in a vendor patch than doing
a brand-new-system-installation.

response 3 of 24:

Feb 26 00:50 UTC 2003

Now linked to Coop as Item 175; Garage 146

response 5 of 24:

Feb 26 06:36 UTC 2003

The reliable way to do removable drives would be hot swappable drives.

From a system security standpoint, it's much better to do system builds
and updates on a 2nd system, then ship built binaries over to the main
system.  Building binaries on the main system creates an opportunity for
vandals to sneak in their own local hacks.  Recording a snapshot of what
was built also makes it possible to more quickly restore the system.

In general, openbsd updates come in 2 forms, "compatible" upgrades, and
"incompatible" upgrades.  Compatible upgrades can be done relatively
transparently.  New stuff fixes bugs or enhances existing APIs.  Old
binaries continue to run.  Incompatible upgrades break everything -
anything that is a binary has to be rebuilt, and the question becomes
not what changed, but what remained the same.  Just as a "for instance",
the current i386 openbsd is still a.out based, but there's a chance it
could become elf within the next few years.  The use of "propolice" is
another good example of why "incompatible" upgrades happen.  "Compatible
upgrades" are obviously easier than "incompatible" ones.

There are going to be 3 necessary steps for doing updates:
 (1) dealing with etc
 (2) dealing with /usr/local
 (3) dealing with any system mods we make that aren't just etc hacks.

Of the 3, dealing with etc is the most obvious one.  "etc" is the
package that contains most of what lives in /etc and /var.  These are
mostly configuration files, some of which are relatively canned things
we don't want to change, and some of which are explicitly hooks for us
to customize, and some of which contain both.  /etc/master.passwd is the
outstanding example - the bulk of this will obviously consist of our own
entries, but the start contains system IDs.

Dealing with /usr/local - this is where things get really messy.
OpenBSD packages build to go here.  We will likely also want to build
our own things to go here.  There can be complicated interdependencies.
Packages can and often are updated independently of the core OS.  Some
of those updates may be important to us.

Local system mods may not be hardest to handle, but they'll be hardest
to anticipate.  One sample we might have: safefork, which is a
modification to fork to deal with fork bombs.  This could go any of 3
ways: a loadable kernel module - potentially the most modular of the 3,
but lkms aren't well supported in obsd.  A patch that we apply at build
time to the kernel source.  Obviously this precludes using precompiled
kernel binaries, but gives us the most flexibility.  Convince the obsd
developers to adapt "safefork" as a standard feature.  In this case it
may even be likely, but it's not certain.

response 7 of 24:

Feb 26 23:41 UTC 2003

Tell us about encap, in 25 words or less.

response 9 of 24:

Feb 27 05:36 UTC 2003

(twenty-five even.  nice job!)

response 11 of 24:

Mar 4 04:04 UTC 2003

Before the development machine croaked, I tried the "build on another
system and move the binaries over" thing a couple times with a couple
different packages.  I didn't like it one bit.  One of the packages was
mysql.  It was a "configure; make; make install" kind of thing (except
for all the SunOS incompatibilities I had to fix).  "Make install" runs
a long time, creating all sorts of library files, directories, config
files, header files, you name it, all with appropriate permissions and
ownerships, in many different directories.  The output from "make
install" is rarely totally illuminating, so figuring out exactly what it
installs where is not simple.  So how do I "move the binary over"?  That
may work for Picospan, but few modern packages are quite that simple. 
Backtalk I can mostly manage, but only because I know it so well.

In the end, with mysql, I tar'ed up the entire build directory after
doing the 'make', ftp'ed it over, unpacked it, and run "make install" on
Grex.  Simple enough, but hardly nice - that build directory is gigantic.

response 13 of 24:

Mar 7 22:57 UTC 2003

a
Most modern packages (including mysql) have various options for where
to install, and where the installed copy finds bits of itself.  Most
commonly, there is a
        configure --prefix /run-time/path
option to configure to tell it where it lives once installed, and at
install time you can say
        "make DESTDIR=/readwrite/path install"
to tell it to to prefix all the runtime pathnames with
"/readwrite/path".  This is commonly done for packages installed in
AFS, for instance, where the run-time path may point to a read-only
path that can't be written directly.  This is how most gnu utilities
work, if they're properly configured (not always the case), and how
many other things can be made to work (depending on how faithfully they
follow the model.)  (Gnu configure scripts also allow you to segment
and additionally customize prefixes for various parts of the package,
but that's an unimportant point of confusion.)  Mysql is packaged like
most gnu software, so knows about both
--prefix (mentioned in the documentation) and DESTDIR (which they
somehow omitted.)  Even the documented --prefix alone can be easily
used to identify which bits are supposed to be installed.  For most
generic distributions, this can certainly be be made to work, modulo
some work which is often slight.

For packages that are complicated but somehow managed to omit the
customary install options, a simple alternative on a guinea pig system
would be to just install it, then use some tool like tripwire or aide
to find out what changed.

If you just install everything in "/usr/local" (which is how most of
these things work) you can end up with filename collision problems, and
package A breaking package B.  This is especially problemmatical when
installing multiple versions of the same package.  There are lots of
ways to solve this, but they generally involve installing the thing in
its own per-package directory, then making symlinks to some more
central place to publish its existance.  In solaris 2, sun has
installed many of its things in /opt .

For OpenBSD, many things, including mysql, are available as precompiled
packages.  Source is available via the "ports" stuff.  The nice thing
about the precompiled packages is that somebody else has worried about
filename collisions, obsd configuration and patching, and also
installing and removing the thing.  This can greatly simplify life,
unless you *really* need bleeding edge stuff or some arcane option that
isn't available via the packages, and are willing to endure the pain.
The source that's available via the ports includes the necesary bits to
compile it locally and turn it into a package that can then be
installed on a different system.

It should be fairly easy to take random source and turn it into a
package.  I haven't tried this so can't speak from personal experience,
but the package management bits mostly seem to include local patches
and some wrapper stuff to decribe which files get installed & such.
This doesn't look hard to reproduce.

Backtalk is one of the few things where this would not be sufficient,
mostly because of anything associated with authentication that
might need to get done in httpd.conf .

response 15 of 24:

Mar 18 21:41 UTC 2003

Dan probably meant my 3rd paragraph ("/usr/local") not my 2nd ("how to
figure out what components get installed by package X".)

response 17 of 24:

Jun 24 15:41 UTC 2003

This topic is still one that I feel Grex staff lacks a solution for.  This
is a major problem.

response 19 of 24:

Jul 19 16:11 UTC 2003

I could write a php interface that would be twice as good as backtalk. I could
also built a java client for party. I could bring Grex into the 21st century.
This interface is from the 90's
 See an example of my work here:
http://www/antionline.com

You guys have no vision.

response 21 of 24:

Jul 20 01:44 UTC 2003

re #19 - if you can't even format a link properly, I'd be a little 
scared to try any software you write.

response 23 of 24:

Nov 14 20:09 UTC 2006

Having looked at the http://www.antionline.com/ webpage, I will say that
I respect the poster's technical abilities, but would not want to see
that as the default interface. I like the simple, uncluttered look of
backtalk. Having to deal with a metric assload of information in a day,
it is a nice rest for the eyes to see something straight-forward, in
which information flows linearly, rather than coming at me from all
sides. 

And yes, a metric assload is roughly 1.0296 imperial assloads. You can
do the conversion into absolute terms yourself.