Grex Oldcoop Conference

Item 62: Disk Quotas on the New Grex Machine

Entered by gelinas on Tue Dec 30 06:41:52 2003:

On the new grex machine, we will have larger disks for the "user"
partitions.  The OS will also be able to enforce quotas directly, without
staff intervention.  Right now, the plans are to use the quota system to
share the available disk-space equitably.

The quota system has three limits: a 'soft' limit, which generates a
warning, a 'hard' limit which cannot be exceeded, and a 'grace period'
after which the 'soft' limit becomes a 'hard' limit.

When the 'hard' limit is reached, no new files can be created.  (If I
remember correctly, existing files cannot be written to, either.)

The 'grace period' takes effect when the 'soft' limit is exceeded.
If files are not removed before the grace period expires, the soft limit
becomes the hard limit, and no new files can be created.

The questions to be answered are:

        What should the soft limit be?
        What should the hard limit be?
        What should the grace period be?

Another question might be:

        Should the hard limit and soft limit be the same, 
                eliminating the grace period?
73 responses total.

#1 of 73 by gelinas on Tue Dec 30 06:49:36 2003:

Here are my answsers to the questions, as a starting point for discussion:

        2 megabytes
        3 megabytes
        7 days

Obviously, I don't think the soft and hard limits should be the same.

I recommend upping the quota from one megabyte to two simply because
we'll have the space.

The three-megabyte hard limit will eliminate most of the current lot of
space-wasters on grex.

The grace period will allow people the time to figure out what to do with
their excess files before they can't do anything.  Seven days should be
enough time, especially since they will be warned as soon as they exceed
the soft limit.


#2 of 73 by willcome on Tue Dec 30 07:58:40 2003:

Does it really matter?  The way information is these days, two megabytes isn't
really any more substantial than one megabyte; and, the way time flows these
days, seven days isn't any different in substrate than six.


#3 of 73 by sholmes on Tue Dec 30 08:10:17 2003:

7 days grace period looks short , like when you are travelling and not
accessing grex. And hwo do we get notified that we have crossed the soft limit
? mail ?


#4 of 73 by mary on Tue Dec 30 12:20:53 2003:

By increasing disk space over what's allowed now, will we be
encouraging storage of graphic files?


#5 of 73 by gelinas on Tue Dec 30 13:35:45 2003:

Re #3:  You exceed quota by creating a file, writing to a file, or moving
a file onto grex; it is an _interactive_ process, with immediate feedback.
The systems I've used with quotas enabled also informed me of the "problem"
each time I logged in.

Re #4:  Most of what I see is many (relatively) little text files.  And tar
files moved in from outside.  Others may see something different.


#6 of 73 by gull on Tue Dec 30 15:11:39 2003:

Re resp:4: I think the question of storing graphics files can be dealt
with seperately.

The proposed limits look okay to me.  I suggest we start fairly small. 
If the limits turn out to be conservative, we can raise them and no one
will complain.  Lowering them after the fact is more difficult.


#7 of 73 by keesan on Tue Dec 30 15:39:52 2003:

I sometimes use grex to download a large file to my home directory and thence
to my computer.  Should I be downloading to /tmp instead?  By large I mean
as high as 1MB. 


#8 of 73 by slynne on Tue Dec 30 15:49:31 2003:

I dont have any strong opinions about this. I think your recomendations 
are fine, gelinas.


#9 of 73 by jp2 on Tue Dec 30 15:50:22 2003:

This response has been erased.



#10 of 73 by jp2 on Tue Dec 30 15:52:41 2003:

This response has been erased.



#11 of 73 by gelinas on Tue Dec 30 15:56:17 2003:

Re 7:  We should put quotas on /tmp as well, I think.  Do you often (or ever)
have more than one of those 1MB files, Sindi?


#12 of 73 by other on Tue Dec 30 16:16:07 2003:

I have two distinct notions here.

One is "Why should we bother with the soft limit and grace period at 
all?  User will be informed either during newuser or in a welcome 
email of the quota limits, and presumably those limits will be 
referenced in viewable documents such as the statement of principles 
or user guidelines."

The other is "The soft limit should only be a way to remind users of 
the limit before they actually hit it.  Effectively, this means an 
infinite grace period so that the soft limit never becomes a hard 
limit, but does trigger a notice when the hard limit is approached."


#13 of 73 by aruba on Tue Dec 30 16:35:32 2003:

I don't have a strong feeling here, but I'm confused about what happens to
the users files when the grace period expires, and he's still somewhere
between the soft and hard limits.  Do the files get deleted?


#14 of 73 by jp2 on Tue Dec 30 16:52:27 2003:

This response has been erased.



#15 of 73 by keesan on Tue Dec 30 16:54:12 2003:

Re the 1M files in /tmp, most recently I actually downloaded a 3.2M file via
grex when none of my DOS browsers could handle my webmail site.  I did
remember to delete it immediately after getting it to my computer.  
Would a 5M limit on files in /tmp be reasonable?  Do they go away at the end
of each day?  Someone once posted something in /tmp for me to download.
I rarely use grex to download anything over about 250K from the web but would
do larger files if we ever got faster modems.  I like using lynx at grex
because it often goes much faster than my own copy via my ISP and then to
download something would mean hanging up on grex and dialing the ISP if I did
not do it this way.  Lynx no longer works at Driverguide.


#16 of 73 by jep on Tue Dec 30 17:02:06 2003:

Will the quotas apply to e-mail?

I don't store files on Grex, so don't have any personal concerns about 
quotas.  I'd say this is the sort of thing where you try something, 
then if it doesn't work, you know more and you adjust accordingly.


#17 of 73 by aruba on Tue Dec 30 17:35:16 2003:

Yeah, that attitude sounds right to me, too.


#18 of 73 by gull on Tue Dec 30 17:56:33 2003:

I think the soft limit is important to avoid people's conference
participation files from getting trashed without warning if they
accidentally bump above the limit.


#19 of 73 by naftee on Tue Dec 30 18:54:24 2003:

What, exactly, is wrong with the storage of graphics?  Certainly it can't be
bandwidth, because with the faster NextGreX machine, it should take that in
stride, no?


#20 of 73 by gelinas on Tue Dec 30 19:31:08 2003:

Yes, it's bandwidth, the number of bits that can pass in and out on the
network connection.  A faster CPU, or a bigger disk, does not improve
the network bandwidth.


#21 of 73 by mary on Tue Dec 30 19:45:26 2003:

If we increased the quotas enough to allow folks to put
web-accessible pictures on Grex, would that traffic 
noticeably slow most user's sessions?

I'm *really* hoping that Grex can be significantly faster
on the new machine.  Unless Grex is faster I expect fewer
and fewer users will stick around long enough to get hooked.

Don't know if that helps but it is where my concerns rest.


#22 of 73 by cross on Tue Dec 30 20:07:24 2003:

Regarding #20; That's never been measured, and is only a conjecture.


#23 of 73 by willcome on Tue Dec 30 20:32:04 2003:

You're all silly-asses.


#24 of 73 by gull on Tue Dec 30 21:05:18 2003:

I think the ban on graphics was also intended to keep people from
putting up porno sites.  The target audience for ASCII porn is pretty small.


#25 of 73 by gelinas on Tue Dec 30 21:28:39 2003:

Which is conjecture, Dan?  That a faster CPU and more disk space won't
improve network bandwidth?  Or that the effect of allowing graphics files
on grex will be a worse experience for interactive users?

Either way, I've started a new item, #63, for the discussion of grex's
policy on multi-media files.


#26 of 73 by naftee on Tue Dec 30 22:08:33 2003:

gelinas, you messed me up now :-0


#27 of 73 by cross on Wed Dec 31 00:56:22 2003:

Regarding #25; The conjecture is that grex's network bandwidth is
at the point of being exhausted.


#28 of 73 by gelinas on Wed Dec 31 01:02:13 2003:

The bandwidth does not have to be at the point of exhausation for the
additional network load of graphic files to be noticible.

But let's take it to the next item. ;)


#29 of 73 by bhoward on Wed Dec 31 02:13:38 2003:

These numbers seem as fine as any to start with, Joe.  I would like
to keep an open mind about raising the numbers if we find folks are 
bumping into the limits too often in legitimate uses of the system.


#30 of 73 by gelinas on Wed Dec 31 04:04:56 2003:

Daniel Gryniewicz commented on this subject in e-mail:

        The problem with 2, or 3, or 5, is that you can download and
        build eggbot in that much space...


#31 of 73 by ryan on Wed Dec 31 04:59:13 2003:

This response has been erased.



#32 of 73 by gelinas on Wed Dec 31 05:00:33 2003:

Is that supposed to be 2 and 3 MB, Ryan?


#33 of 73 by ryan on Wed Dec 31 05:00:44 2003:

This response has been erased.



#34 of 73 by ryan on Wed Dec 31 05:01:31 2003:

This response has been erased.



#35 of 73 by gelinas on Wed Dec 31 05:07:28 2003:

Timing. ;)


#36 of 73 by jp2 on Wed Dec 31 13:12:00 2003:

This response has been erased.



#37 of 73 by other on Wed Dec 31 15:27:14 2003:

Jamie, I thought EVERYTHING you posted was ACSII PORN.


#38 of 73 by jp2 on Wed Dec 31 16:13:32 2003:

This response has been erased.



#39 of 73 by naftee on Wed Dec 31 17:49:23 2003:

re 37 Actually, that's twinkie.


#40 of 73 by malymi on Wed Jan 7 10:23:16 2004:

quotas and e-mail need special consideration.  without quotas on the
mail spool abusers will just e-mail themselves things, perhaps even work
within the spool if it's user writable (i forget if obsd uses a mode
1777 spool).  but if the spool has a quota then there is a way for a
user's quota to be exceeded which is not interactive and thus invisible
to the user, and in fact happens as a result of forces typically outside
their control, i.e., spam, worms and abusive action can disable e-mail
reception -- staff could not even deliver a warning without an enhanced
remote access mechanism or some fancy footwork quota-wise.

as it happens i favor having quotas over not having them, but you need
sufficient status visibility in all reading modes.  unfortunately such
visibility is not available by default in popular and free tools, thus
would require some custom patches or spending money.  these days spam/
worm containment with a much higher (perhaps shared) quota is also
necessary, but again care is required otherwise abusers will try to
store things in the quarantine.


#41 of 73 by gelinas on Sat Jan 24 04:26:19 2004:

Another question has come up:  Should we put a limit on the number of 
files a user can create?  If so, what should it be set to?


#42 of 73 by keesan on Sat Jan 24 15:03:35 2004:

Can you put a limit on the number of new items a user can create in one day,
for instance 3 or 5?  I don't see why you need to limit the number of files
if you are limiting disk usage already.


#43 of 73 by ryan on Sat Jan 24 16:37:04 2004:

This response has been erased.



#44 of 73 by remmers on Sat Jan 24 17:23:52 2004:

Ryan is correct - under Unix, each disk partition has a set maximum
number of files, equal to the number of slots in the "inode table".
So a possible denial-of-service attack would be for a user to fill
up the inode table.  Then no other user on that disk partition
could create new files, even if there were plenty of free space
on the disk.

So it sounds like we should set a maximum number of files per user.
I'm assuming the quota system lets us do that.  On NextGrex as
currently configured, the user partitions /a and /c combined have
over 5 million inodes, so even being generous and assuming that
we grow to 20000 users with an average of 50 legitimate files
apiece, that would take up less than 20% of the inode space.
So the limit could be pretty generous and still avoid a problem.
If we set the maximum at, say, 5000, a twit would have to create
a few hundred accounts to run the system out of inodes.  That's
a pretty good deterrent, and even if they persisted and tried,
the activity would be noticed and stopped long before the
limit was reached.


#45 of 73 by gelinas on Sat Jan 24 17:47:06 2004:

The few places I've seen quotas, the file limit was 1,000.  Would that
be a reasonable place to start, bumping it up later if necessary?


#46 of 73 by keesan on Sat Jan 24 20:29:31 2004:

I think I have between 50 and 100 files which I thought was a lot.
2M disk space and 1000 files would be 2K average per file - do people have
that many small files?


#47 of 73 by ryan on Sat Jan 24 20:42:35 2004:

This response has been erased.



#48 of 73 by styles on Sun Jan 25 02:59:02 2004:

compilations can create a "lot" of files, but any compilation that would
create that many files would surely hit a 2MB limit before an inode limit.
someone might have a one-file-per-entry type of webboard, which could create
lots of small "ROTFL!!!" and "LOL OMG WOT U SAY?" response files.  there's
probably a few other practical-ish cases where this would happen, but not that
many.


#49 of 73 by bhoward on Sun Jan 25 03:21:42 2004:

I support the higher limit proposed by remmers.


#50 of 73 by gelinas on Sun Jan 25 03:34:44 2004:

Like disk space, inodes have both soft and hard limits.  How about 4000 soft
and 5000 hard?


#51 of 73 by bhoward on Sun Jan 25 05:53:05 2004:

I don't normal users of the system will generally approach either of
those limits, so no strong opinion either way.  


#52 of 73 by scott on Sun Jan 25 14:15:57 2004:

Big packages like eggdrop tend to use up a bit more than 1Mb, and have
hundreds of files even before compilation.


#53 of 73 by ryan on Sun Jan 25 15:03:48 2004:

This response has been erased.



#54 of 73 by remmers on Sun Jan 25 17:21:27 2004:

I support the higher limits proposed by me.  :)

My philosophy in setting limits is to set them so as to avoid system
problems but to make them as generous as possible within that constraint.
What do we gain by making them any smaller?


#55 of 73 by gelinas on Sun Jan 25 18:25:30 2004:

I like soft limits for the warnings they give.  That's the only thing gained
by 4000/5000.


#56 of 73 by remmers on Sun Jan 25 21:55:10 2004:

Sounds reasonable.


#57 of 73 by bhoward on Sun Jan 25 23:25:52 2004:

Yeah, and we can always up it later on if we find this is too low.


#58 of 73 by slynne on Mon Jan 26 15:30:03 2004:

Makes sense to me. 


#59 of 73 by twenex on Thu Oct 14 01:58:54 2004:

Perhaps we could set the hard limit to double the soft limit, to accommodate
users like Sindi (not crackers, script-kiddies, or graphics-file uploaders.)
Or bump up the hard limit for members?

2, 3, or maybe 4 for a bunch of text files and small executables sounds fine
to me.


#60 of 73 by keesan on Thu Oct 14 03:47:16 2004:

If grex would just install links (or links2) I could delete the copy I
compiled and use much less disk space.  (Twenex, how much am I over quota?).
Is there going to be some way for people to contribute programs that are kept
in shared disk space?  (Oops, I also have a large pdf file here temporarily
because I use grex to transfer files between my three residences).  


#61 of 73 by gelinas on Thu Oct 14 04:28:47 2004:

Any one can ask that a program be installed at any time.  Then it's just a
matter of staff getting the time to install it.

FWIW, links is installed on the new grex machine right now:

} grex:gelinas {110} links -version
} Links 2.1pre14
} grex:gelinas {111} 


#62 of 73 by twenex on Thu Oct 14 10:04:04 2004:

Re: #60. To find out the amount of disk space you are using, type "du -h ~" 
(Disk Usage [in] -human-readable format [shows in kilobytes, or megabytes 
instead of bytes], [directory [folder]:] ~ [shorthand for your homedirectory.])
If you are in, or cd to, your home directory you can leave out the ~. "Quotas"
on nowGrex are managed manually by staff; on NextGREX you will be notified
by the system when and by how much you are over quota.


#63 of 73 by janc on Thu Oct 14 13:23:15 2004:

Yeah, Cindi requested links months and months ago, so I installed the copy
from the ports tree.  However there are about a zillion variations on links,
so I have no idea if the one in the ports tree is the one Cindi wants.

I don't understand wny setting the soft limit so much higher than the hard
limit makes sense.

Since we already increase the limit for any user who gives us a vaguely
sensible explanation for why they should have a higher limit, I don't see a
need to give members a higher limit.

For the time being, Grex staff has no problem with Cindi using lots of disk
space.  Keeping a copy of links2 around is a more than vaguely sensible
explanation.


#64 of 73 by gelinas on Fri Oct 15 00:05:03 2004:

The reason I suggested a meg between the soft and hard limit was that I find
it fairly easy to fill up a meg.  If we've the disk space to give each user
five megabytes, we aren't hurt by giving them plenty of warning as they
approach that limit.

FWIW, I don't see a reason to set the soft limit at half the hard limit.


#65 of 73 by keesan on Fri Oct 15 02:16:35 2004:

Thanks Joe.  I think while you were not looking links may have evolved to
version 15 and it will probably keep evolving.  It handles some but not all
javascript now.  Jan, what do you call the links that you installed and how
would I access it?  I  can't use it by typing 'links' so I still use my own
from my own subdirectory (links-0.96 not elinks or links hacked or links2).
Are all the program files in one directory somewhere?


#66 of 73 by gelinas on Fri Oct 15 02:19:47 2004:

It's on the new machine, not the current one, Sindi.


#67 of 73 by janc on Fri Oct 15 13:19:57 2004:

I'm not installing anything on the old machine these days.


#68 of 73 by scott on Fri Oct 15 14:59:58 2004:

I think Sindi would be a great beta tester for NextGrex, at least the
text-based-over-dialup stuff.


#69 of 73 by gregb on Fri Oct 15 18:29:39 2004:

I second that.


#70 of 73 by janc on Sat Oct 16 13:19:56 2004:

Yeah, she's on my list.


#71 of 73 by keesan on Sun Oct 17 04:35:00 2004:

I did a whole year of beta testing for Newdeal.  I found lots of small bugs
in keyboard navigation, which nobody else was using, and in text-based
printing, and in the older GUI.  I was the resident ignoramus who had not used
any GUIs before.

Is it possible to let everyone be a beta tester for NextGrex?  I. e., can one
phone line be hooked to NextGrex for dialin use only, without any web
connection?  Just to test the bbs, for instance.  Or could people choose to
telnet to NextGrex?


#72 of 73 by janc on Sun Oct 17 21:23:16 2004:

Well, last time I tried to dial into Grex, it took me an hour to find a
computer with a working modem, install comm software, and figure out how
to make it work.  Very soon I plan to start letting some people onto the
system to start banging on things, but I want to limit the set of
people.  I'm pretty sure security is good, but if there are security
holes to be discovered, I'd rather they were discovered by someone I
trust.  And I'd rather keep the dialins hooked to Grex.


#73 of 73 by jesuit on Wed May 17 02:14:31 2006:

TROGG IS DAVID BLAINE


There are no more items selected.

You have several choices: