|
Grex > Coop12 > #12: brainstorming solutions for the full disk problem |  |
|
| Author |
Message |
| 25 new of 203 responses total. |
jp2
|
|
response 148 of 203:
|
Apr 16 13:31 UTC 2001 |
This response has been erased.
|
carson
|
|
response 149 of 203:
|
Apr 18 22:15 UTC 2001 |
resp:148
(very much so.) :^)
|
valerie
|
|
response 150 of 203:
|
Apr 20 00:08 UTC 2001 |
This response has been erased.
|
mdw
|
|
response 151 of 203:
|
Apr 20 02:28 UTC 2001 |
A fair number of people drag stuff over using Lynx or other web
browsers. This is particularly popular for graphics, but it is also
common for software (it's easier to setup apache than ftpd), and of
course for vandal tools (httpd logs are generally not as useful.) I'd be
very surprised if even 50% of the bloat on grex comes via ftpd these
days.
|
gull
|
|
response 152 of 203:
|
Apr 20 02:48 UTC 2001 |
<grin> I can't help but think of this bit from "Know Your System
Administrator -- A Field Guide":
--SITUATION: Low disk space. --
TECHNICAL THUG: Writes a suite of scripts to monitor disk usage,
maintain a database of historic disk usage, predict
future disk usage via least squares regression analysis, identify users
who are more than a standard deviation over the mean,
and send mail to the offending parties. Places script in cron. Disk
usage does not change, since disk-hogs, by nature, either
ignore script-generated mail, or file it away in triplicate.
ADMINISTRATIVE FASCIST: Puts disk usage policy in motd. Uses disk
quotas. Allows no exceptions, thus crippling
development work. Locks accounts that go over quota.
MANIAC:
# cd /home
# rm -rf `du -s * | sort -rn | head -1 | awk '{print $2}'`;
IDIOT:
# cd /home
# cat `du -s * | sort -rn | head -1 | awk '{ printf "%s/*\n", $2}'` |
compress
(Courtesy of http://www.cs.bgu.ac.il/~omri/Humor/sysadmins.html)
|
gelinas
|
|
response 153 of 203:
|
Apr 20 04:47 UTC 2001 |
I use ftp to get files OFF grex; I'd not appreciate losing access to it.
|
pfv
|
|
response 154 of 203:
|
Apr 20 13:09 UTC 2001 |
Val -
I just went back about 100 posts, and I can't find anything about
stats.. (I admit I'm having the first cup of coffee)..
The last thing I found was the idea of a "logout cleaner".
|
gull
|
|
response 155 of 203:
|
Apr 20 15:01 UTC 2001 |
I'd suggested making incoming FTP operations drop altogether after a
certain number of kilobytes, instead of just slowing down. If most
files are now coming in via lynx, that wouldn't help, though. (And I
suppose if we did limit FTP PUTs, lynx use would probably jump.)
I really think the only permanent, reliable solution to this would be
disk quotas. It's a classic disk quota situation. That may need to
wait until we get newer hardware and/or a newer OS, though.
|
pfv
|
|
response 156 of 203:
|
Apr 20 16:27 UTC 2001 |
The idea of "quota", (on a system that can't run "quota"), is why I still
suggest a root-managed global .logout script..
As they leave, it could run a nohup program, (owned by root), that simply
scans their space and proceeds to thermonuke oversize files or subdirs with
bot/irc/bnc crap.
I can't imagine that a background job like that is going to slow/inhibit
the disconnect of a user. And, the program itself can 'nice' itself.
|
jp2
|
|
response 157 of 203:
|
Apr 20 16:56 UTC 2001 |
This response has been erased.
|
gull
|
|
response 158 of 203:
|
Apr 21 03:34 UTC 2001 |
Consider the limited time the Grex staff has. Now consider the hours
(nay, days) of labor it'd take to move all of Grex's custom software and
custom network patches over to another platform. It took months to move
from the Sun 3 to the Sun 4, IIRC.
|
mdw
|
|
response 159 of 203:
|
Apr 26 16:51 UTC 2001 |
NetBSD's kernel architecture is less advanced than SunOS: it doesn't yet
support even asymmetric multi-processing. That means it won't make as
good a use of our current very cheap hardware.
|
jp2
|
|
response 160 of 203:
|
Apr 26 17:43 UTC 2001 |
This response has been erased.
|
mdw
|
|
response 161 of 203:
|
Apr 26 19:07 UTC 2001 |
Well, considering this decade is only about 4 months old, that means
expensive *new* hardware at close to list prices. To get something
roughly equivalent to the current hardware, we'd be looking at SCSI disk
and ECC (or at least parity) RAM, and a non-intel instruction set CPU.
There are a number of fine makers of such hardware, but it would be a
stretch to make our budget cover any of them.
Keep in mind when comparing to the 3AM "act now" TV offer computer that
what you see on TV probably comes with one spindle of IDE disk, no tape
backup, virtual parity RAM, intel instruction set CPU, a lot of
software, video hardware, etc., that we don't need, and is generally
optimized to perform well as a slightly flakey computer for *one* person
who is probaby not doing a lot of multi-tasking. Grex probably *could*
afford that system, but spares would be another question.
IDE disk is cheap, but doesn't do overlapped seeks - not a big issue for
IDE since it only supports 2 disks at most. Grex has a history of
expanding to fill its disks - no upgrade path seems pretty dubious to
me, especially given that the excuse for this whole project is so we can
store an infinite number of copies of eggbots and graphics.
Backups - I think it goes without saying that these are important. We
currently have a very nice SCSI tape drive, less than 1 year old. We'd
certainly want to continue to use that if at all possible.
"no parity" ram works fine when it works. Of course, you'll never know
when it doesn't. We've already lived through flakey memory on grex, but
at least we *knew* it was causing kernel panics. Memory that was
flakey, but *doesn't* crash the system. Do we really need it?
"Intel instruction set". Don't get me wrong. I have a lot of respect
for Intel. Nevertheless, the pragmatic fact is (1) intel does not have
a pristine bug record, and (2) *every* vandal under the sun is trying
exploits using intel machine language. Using almost anything else will
buy us at least 2 months of time to patch holes, and completely dissuade
80% of the vandals out there. Also, if you price out intel hardware
with SCSI disks, sufficient ECC RAM, etc., you start looking at prices
that are pretty comparable with the non-intel server market.
|
jp2
|
|
response 162 of 203:
|
Apr 27 02:09 UTC 2001 |
This response has been erased.
|
tenser
|
|
response 163 of 203:
|
Apr 27 03:44 UTC 2001 |
Actually, I offered to donate some Sun hardware to grex about a
month back, but, alas, I never heard back from anyone, and the
hardware is no longer available. Specifically, I was offering a
Sun Ultra 2 with dual 170 MHz processors, 640 MB of RAM, and 40
GB of SCSI disk spread across two controllers. I also offered
to throw in an Ultra 1 with 128MB of RAM and 22GB of disk, plus
a SPARCstation 5 and SPARCstation 2. That would certainly have
mitigated some of these problems, and would have also provided
an upgrade path. Plus, grex could have continued to use its
current tape drive.
I'm not sure why no one got back to me about any of this. All
I asked was that grex pay for shipping from New York City. Of
course, to effectively use the dual processor Ultra 2, grex
would have had to move to Solaris. Perhaps that had something
to do with it.
Marcus, I'm not sure how you can rate SunOS 4 as more advanced
than, say, NetBSD, just because it lacks the hack multiprocessor
support of the former. Consider that NetBSD has many features
that SunOS doesn't, such as large file and filesystem support, a
better VM system, soft metadata updates for UFS file systems
(which, IMHO, would be of huge benefit on grex), file flags, kernel
security levels, and more. Not to mention active maintenance. In
fact, I'm willing to bet that a single Sun Ultra 1/170 running
NetBSD would blow the doors off of the current Sun running SunOS
in all respects, would use less power, and only cost a few
hundred dollars. I'll even help grex finance some or all of it,
if desired.
|
jp2
|
|
response 164 of 203:
|
Apr 27 13:54 UTC 2001 |
This response has been erased.
|
i
|
|
response 165 of 203:
|
Apr 28 01:25 UTC 2001 |
Continued very interesting technical discussions.
New/fast/sexy hardware for grex would be nice to talk about.
It would take at least hundreds of hours of staff time - time that
doesn't seem to be available - to get grex working on any new hardware.
Ergo, i don't think that new hardware is a viable strategy for dealing
with the disks-perpetually-filling-with-crud problem.
|
tenser
|
|
response 166 of 203:
|
Apr 28 04:14 UTC 2001 |
Regarding #165, it remains that grex is sitting on ``dead end'' hardware
and software. Those hundreds of hours of staff time required to move to a
new system are partially a result of having to deal with that. Having an
expandable hardware and software platform with which to work alleviates
a significant number of those problems. For example, if running Solaris,
then moving from a Ultra 1 to an Ultra 2 is relatively easy.
What's more, I read statements in coop seemingly quite frequently wherein
staff members make mention of how they simply don't have hardware to
do this or that. That's just not true. The fact that I was prepared
to donate ~US$4,000 worth of gear to grex for the price of shipping
demonstrates otherwise. Alas, this cannot happen now, since the hardware
just isn't available anymore. :-( (Though I'd still like to donate
something to grex!)
One of the problems with having such a ``customized'' system is just this;
it's impossible (or at least *really* hard) to upgrade. The most common
accepted solution to that problem is to simply not customize the system
so much; that is, find ``standard'' solutions to problems and use them,
thereby shifting the burden onto someone else whose generally willing to
accept it and thus leverage economies of scale in the maintenance arena.
What's more, going with a newer system eliminates the need to do many
of the things which grex has to do (for instance, installing GNU versions
of most of the common utilities in order to work around SunOS
deficiencies).
It's a different way of thinking about the system maintenance problem.
|
gelinas
|
|
response 167 of 203:
|
Apr 28 04:22 UTC 2001 |
(Right; they just have to install GNU versions because that's what most people
use.)
|
tenser
|
|
response 168 of 203:
|
Apr 28 04:26 UTC 2001 |
Regarding #167; Oh really? I've run a lot of Unix systems over the
years; I haven't installed GNU versions of df or ls in, oh, 6 years
maybe. Then again, I haven't installed SunOS 4 in about that long,
either.
I don't know many users who ``expect'' GNU versions of most of the
basic utilities. Now things like gcc on the other hand....
|
gelinas
|
|
response 169 of 203:
|
Apr 28 04:31 UTC 2001 |
I don't remember using GNU versions of df or ls on SunOS 4 systems, either.
|
tenser
|
|
response 170 of 203:
|
Apr 28 04:43 UTC 2001 |
Regarding #169; then you must have a very short memory:
: grex 48; ls --version
ls (GNU fileutils) 3.16
: grex 49; which ls
/usr/local/bin/ls
: grex 50;
I suspect that if you were to type, ``which ls'' you'd see the
same thing. I suppose I could be wrong, though.
/usr/local/bin/df seems to be a symbolic link to /usr/bin/df. I
remember absolutely needing GNU df on AIX 4.1.3 in order to avoid
going crazy looking at poorly formatted information about NFS
mounted filesystems. But things have progressed quite a bit in
the last 6 years. Anyway, the fact that /usr/local/bin/df exists
is probably an indication that the GNU version was probably on
the disk at one point.
|
jp2
|
|
response 171 of 203:
|
Apr 28 14:21 UTC 2001 |
This response has been erased.
|
tenser
|
|
response 172 of 203:
|
Apr 28 16:14 UTC 2001 |
Regarind #171; Errm, yeah, that to. My point is that with a more
modern version of Unix, most of the functionality of the ``GNU
enhanced'' tools already exists in the base system; there's no need
to install (and maintain) a bunch of other stuff.
|