|
Grex > Coop11 > #36: Increasing User Disk Space Maximum | |
|
| Author |
Message |
| 25 new of 111 responses total. |
rcurl
|
|
response 50 of 111:
| 
Nov 4 07:13 UTC 1998 |
I did? I started this item to argue for a larger disk quota. I have since
asked why not implement e-mail (or link use) quotas too, to balance the
"freedoms" that users have. I have been under the impression that disk
quotas are on now - implemented by hand.
Why not buy more disk space rather than freeing up just a little from
some users? Then everyone can have more.
|
scott
|
|
response 51 of 111:
|
Nov 4 11:44 UTC 1998 |
(Psst...Rane... If you download your accumulated mail onto one of your own
machines, you can access it without having to connect to Grex)
|
valerie
|
|
response 52 of 111:
|
Nov 4 13:18 UTC 1998 |
This response has been erased.
|
keesan
|
|
response 53 of 111:
|
Nov 4 15:37 UTC 1998 |
I will download half the files in my home directory to my own machine, now
that I figured out where the empty space is on my hard disk. Thanks Scott.
|
rcurl
|
|
response 54 of 111:
|
Nov 4 20:16 UTC 1998 |
(Re #51: psst...that is much less convenient for retrieving, editing and
forwarding pieces of past mail, which is my main use for the mail I
keep on Grex - I do not just hoard it, I use it. Also, I have offered to
pay for another MB of disk space...I'll even offer to pay 1000% of the
cost (that would be, oh, about 75 cents)).
How about just doubling the allowed disk space to 2 MB, and see what happens
(and buy more disk space). At the same time, implement the automatic
limit on disk use (and for link use too). Then, people would know what the
automatic consequences would be when theyt 'grow' into the new space, and
I expect there would be less grumbling about having the limits enforced
automatically.
|
steve
|
|
response 55 of 111:
|
Nov 4 23:24 UTC 1998 |
The 1M limit was arbitrary, but an incredibly generous one, even
by todays standards. I don't think there is another shell account
system on the net today that is as free with disk as we are. No,
we're not really a "shell account" system, but still close enough.
1G divided by 1M is only 1000. Right now, Grex has about 23,000
accounts. By "normal" system admin figures, we're doing it all
wrong already and not conserving disk enough. Thats OK with me
however, as its the Grex way.
Yes, we will get more disk--I don't think there is any way we
can avoid that. And, with prices making their ever downward
spiral, we shouldn't have to spend a large amount for it. The
part about this that worries me is the extra staff impact that
more disk has, namely backups. It costs money for the tapes to
back disk up, and costs the more valuable comodity of staff time
to do this. One of the things we might think about in the not
too distant future is the position of "operator", someone who is
1) trusted, 2) able to work correctly on their own, 3) knows when
to ask for help when "weirdness" arrises, 4) is able to do things
on a periodic basis. That should largely solve the backup issue
in terms of time.
The other issue, one that I'm trying to get data on, is how
big a disk we can put on Grex. Today we can get a large SCSI
disk easily, but I want to know exactly what others have done
with this first, before we spend any money on a disk and find
out that it won't work here.
|
janc
|
|
response 56 of 111:
|
Nov 6 02:53 UTC 1998 |
Hmm...I don't know enough about SCSI, but I seem to recall that our SCSI
controller is unable to handle some newer disks. Something about not
being "Fast" or "Wide" or "Ultra" or something. If this is true, then
we are limited to using older disks, whose price doesn't drop as fast,
so disk space isn't as cheap as all that. There must be a way around
this - an alternate SCSI controller on the Sbus maybe?
|
mdw
|
|
response 57 of 111:
|
Nov 6 04:01 UTC 1998 |
There are several kinds of scsi. Some are compatible with others,
others not. I don't think, for our purposes, that makes sense to get
anything that won't plug in and work with the existing cpu scsi
controller; the difference in cost is unlikely to pay for the different
controller.
|
steve
|
|
response 58 of 111:
|
Nov 6 13:14 UTC 1998 |
We're compatible with standard SCSI-2 hardware--that isn't the issue.
There are lots of fast SCSI devices that can work in their backwards
compatible mode, as well. What I'm concered with is the size of the
disk. I believe there is a 2G per partition limit on this version of
SunOS, and there is a limit on how big a disk SunOS can talk to, to
make those 2G partitions. If we can use a 9G SCSI-2 disk and break it
into several 2G partitions that would be great. I haven't talked to
anyone who's done that, yet. So thats my question. Sure, we could go
out tomorrow and get a 2G SCSI disk, but I'd like to get much bigger
ones if we can. I think this is possible but I want confirmation first.
|
janc
|
|
response 59 of 111:
|
Nov 10 05:55 UTC 1998 |
Lots of people are still running 4.1.4 BSD or older. If it is possible
to use bigger disks, lots of people must be doing it. Maybe I'll ask on
the Suns@Home mailing list.
|
valerie
|
|
response 60 of 111:
|
Nov 12 17:28 UTC 1998 |
This response has been erased.
|
rcurl
|
|
response 61 of 111:
|
Nov 12 22:43 UTC 1998 |
What alerts the staff to excessive disk? Can't that be automated (with
suitable warnings)?
I never expected a computer guru to say "there is no way....". ;)
|
davel
|
|
response 62 of 111:
|
Nov 13 02:25 UTC 1998 |
Well, she *said* that, but if you read closely, aside for the "strange bugs"
side remark, the accurate analysis is "will bring Grex to its knees". I for
one think that's a good reason for not doing it. (And I believe Valerie's
analysis is correct, but don't have any direct experience.)
|
scg
|
|
response 63 of 111:
|
Nov 13 08:33 UTC 1998 |
The SunOS quota software has to check how much disk space a user is using
before every time it writes something to their home directory. That will slow
disk writes way down, which in turn will slow everything else way down as
things wait for the disk writes. We don't want to do that.
The processes staff members use to find excessive disk hogs could be
automated. We couldn't run them all the time, since it would slow things way
down, but we could have it run automatically once a week. There is then the
question of what to do when it discovers that a user is over their quota.
Having it go in and delete random stuff would be a bad idea. Having it send
them mail might do something, but would probably be somewhat ineffective.
What might make more sense would be for it to change the permissions on their
home directory such that they couldn't save stuff to it, but that runs into
problems too. Staff has discussed all of those options. It's possible that
we might do something like this in the future, but none of those are easy
solutions.
|
rcurl
|
|
response 64 of 111:
|
Nov 13 17:53 UTC 1998 |
I don't see why a disk-space check has to be run freqhently. Disk space
use grows slowly, usually without the user really being aware of it. I
don't think we need to be aggressive or pushy about this, either. Weekly
or biweekly checks in the course of other things (backups, for example)
would be fine. I would also suggest not being aggressive about doing
something about it. Send an automated e-mail message. Call it "first
warning". State a deadline for action. Send a second warning before
the deadline is reached, as a reminder. I'm not sure what the best
"sanction" would be when the deadline is reached, but certainly the user
should still be able to access files to rm some. Perhaps shutting off
e-mail and some other resources would be sufficient.
I still think, though, that this is the wrong step at this point. First,
disk space allocation should be increased to 2 MB, and then it be made
widely know that this *will* be enforced.
|
janc
|
|
response 65 of 111:
|
Nov 15 05:23 UTC 1998 |
I've always heard that turning on quotas on SunOS slows the system down
badly. I do wonder about that. It seems like it shouldn't really cost
*that* much overhead. I'm no expert, but I think it's just one extra
disk read/write every time you add a block to a file, create a file, or
truncate a file. (There might be worse issues depending on how locking
is handled with the quota files, but I'd like to think that Sun did this
intelligently.) That might have been enough to bring our previous
servers to their knees (frankly, they never got far off their knees at
the best of times), but I think it is quite possible that the Sun 4/670
would work OK with quotas on. It might at least be worth a test.
I'm not sure that we really *want* quotas on, even if we could handle
them though. The less rigid enforcement is kind of more friendly.
One possibility: set hard quotas for newusers, but take them off once a
user has been on Grex for 6 months or they become a member. Note that
the amount of disk they are allowed to use doesn't change, we just go
from software enforcement to the honor system. This should slow down
hackers trying to crash the system by filling up the disks, but it
wouldn't have as big a performance penalty as having all users on
quotas. It also allows old users a bit more flexibility, so they can go
over quota for short periods, and things like that.
|
remmers
|
|
response 66 of 111:
|
Nov 15 12:21 UTC 1998 |
Interesting idea. Technically how would it work? Does SunOS support
setting quotas on a per-user basis?
|
steve
|
|
response 67 of 111:
|
Nov 15 21:50 UTC 1998 |
quotas do slow the system down. Trust me. But worse, are the
problems SunOS has with quotas on a highly active filesystem. We
do *not* want to run Sun's quota code. Integrating some other
quota system into SunOS isn't a simple task, but could be done.
Unforunately Rane, disk usage (in terms of Grex--not an
individual user) can go up *very* quickly, like 60M in less
than an hour.
I'm pondering the idea of how to enforce disk limits in a
reasonable and efficiant way. I think we can do this, but we'll
spend more time looking at disk usage than I'd like. However,
I think it's come to the point where we're going to have to
do this, regardless of the details.
|
scott
|
|
response 68 of 111:
|
Nov 15 22:39 UTC 1998 |
Summary of disk use issues:
1. Most users don't ever get near 1Mb.
2. Some long-time users accumulate email (or whatever) and gradually get
over 1Mb.
3. Some users create an account, subscribe to a mailing list, then never
come back. Mail spool space fills up, we have a quota there as part of the
mail software.
4. Some very few vandals deliberately fill up file systems (disks) as a
denial-of-service attack.
5. A number of one-time users transfer over huge packages like BitchX,
eggdrop, etc. and try to run. Once they give up, accounts are eventually
reaped.
6. Disk quota software penalizes all, for the transgressions of a few,
but being slow.
7. Storing data on Grex long-term is a bad idea. We've had disk bugs in
the past, and our current backup system is atrociously bad (so says backup
czar scott).
Rane, I'd be happy if you'd make some non-token donation for disk use. You'd
be free of worrying about not compensating Grex for your use, and the
Treasurer wouldn't have to process a $0.07 donation. I really doubt, though,
that anybody really cares about your specific use of space, except perhaps
in principle. I know I don't worry about your disk use.
|
rcurl
|
|
response 69 of 111:
|
Nov 15 22:59 UTC 1998 |
I do get 'pestered' now and then to cut back, though I am always cutting
back (and building up). "non-token donation for disk use"...sounds like
a bribe? Since grex doesn't sell disk space, and dues are not in exchange
for any service, and donations must be given without any expectation of
anything in return....I'm stuck.
I thought that maybe your summary was working toward cracking down on
the thoughtless and accomodating the occasional over-run - but it didn't
quite get there.
|
scott
|
|
response 70 of 111:
|
Nov 15 23:07 UTC 1998 |
Yeah, well, I don't have a really strong opinion. :)
What my summary does say (not very clearly) is that abusers are few, and it's
hard to have to put bars on all the windows when most of the people are not
causing any problems. :(
|
aruba
|
|
response 71 of 111:
|
Nov 15 23:44 UTC 1998 |
(The treasurer is happy to process all donations, however small.)
|
steve
|
|
response 72 of 111:
|
Nov 16 04:20 UTC 1998 |
There are a total of 171 accounts on Grex that are over 1M of disk;
8 of those are staff who are always going to be over given the things
they're doing. That makes for 0.64% of the accounts on Grex being over
the limit. So, there aren't many accounts over the limit, but the
problem is wading through that other 99.36% to find the hogs. As I
said before, I think we might be able to a better job of finding
the disk hogs, and should.
|
janc
|
|
response 73 of 111:
|
Nov 19 19:46 UTC 1998 |
Ways increasing Grex's disks:
(1) Move the 1.7 Gig Fujitsu drive from the old machine to the new
machine. The staff is pretty much ready to completely abandon
the old machine. We can't imagine going back to it, even if the
new machine failed. We wish we had spare parts for the new
machine though. This is probably going to happen as soon as
someone takes the time.
(2) Buy another disk. We've confirmed that we aren't limited to 2 Gig
disks. We can handle disks up to 32 Gig. Currently, it looks
like SCSI disk prices are in the following neighborhood:
2 Gig $200
4 Gig $300
9 Gig $500
|
lilmo
|
|
response 74 of 111:
|
Feb 6 19:42 UTC 1999 |
Re #73: (2) was done, right? Does that make (1) obselete? Are we still
concerned about the issues brought up in this item?
|