On the new grex machine, we will have larger disks for the "user"
partitions. The OS will also be able to enforce quotas directly, without
staff intervention. Right now, the plans are to use the quota system to
share the available disk-space equitably.
The quota system has three limits: a 'soft' limit, which generates a
warning, a 'hard' limit which cannot be exceeded, and a 'grace period'
after which the 'soft' limit becomes a 'hard' limit.
When the 'hard' limit is reached, no new files can be created. (If I
remember correctly, existing files cannot be written to, either.)
The 'grace period' takes effect when the 'soft' limit is exceeded.
If files are not removed before the grace period expires, the soft limit
becomes the hard limit, and no new files can be created.
The questions to be answered are:
What should the soft limit be?
What should the hard limit be?
What should the grace period be?
Another question might be:
Should the hard limit and soft limit be the same,
eliminating the grace period?
73 responses total.
Here are my answsers to the questions, as a starting point for discussion:
2 megabytes
3 megabytes
7 days
Obviously, I don't think the soft and hard limits should be the same.
I recommend upping the quota from one megabyte to two simply because
we'll have the space.
The three-megabyte hard limit will eliminate most of the current lot of
space-wasters on grex.
The grace period will allow people the time to figure out what to do with
their excess files before they can't do anything. Seven days should be
enough time, especially since they will be warned as soon as they exceed
the soft limit.
Does it really matter? The way information is these days, two megabytes isn't really any more substantial than one megabyte; and, the way time flows these days, seven days isn't any different in substrate than six.
7 days grace period looks short , like when you are travelling and not accessing grex. And hwo do we get notified that we have crossed the soft limit ? mail ?
By increasing disk space over what's allowed now, will we be encouraging storage of graphic files?
Re #3: You exceed quota by creating a file, writing to a file, or moving a file onto grex; it is an _interactive_ process, with immediate feedback. The systems I've used with quotas enabled also informed me of the "problem" each time I logged in. Re #4: Most of what I see is many (relatively) little text files. And tar files moved in from outside. Others may see something different.
Re resp:4: I think the question of storing graphics files can be dealt with seperately. The proposed limits look okay to me. I suggest we start fairly small. If the limits turn out to be conservative, we can raise them and no one will complain. Lowering them after the fact is more difficult.
I sometimes use grex to download a large file to my home directory and thence to my computer. Should I be downloading to /tmp instead? By large I mean as high as 1MB.
I dont have any strong opinions about this. I think your recomendations are fine, gelinas.
This response has been erased.
This response has been erased.
Re 7: We should put quotas on /tmp as well, I think. Do you often (or ever) have more than one of those 1MB files, Sindi?
I have two distinct notions here. One is "Why should we bother with the soft limit and grace period at all? User will be informed either during newuser or in a welcome email of the quota limits, and presumably those limits will be referenced in viewable documents such as the statement of principles or user guidelines." The other is "The soft limit should only be a way to remind users of the limit before they actually hit it. Effectively, this means an infinite grace period so that the soft limit never becomes a hard limit, but does trigger a notice when the hard limit is approached."
I don't have a strong feeling here, but I'm confused about what happens to the users files when the grace period expires, and he's still somewhere between the soft and hard limits. Do the files get deleted?
This response has been erased.
Re the 1M files in /tmp, most recently I actually downloaded a 3.2M file via grex when none of my DOS browsers could handle my webmail site. I did remember to delete it immediately after getting it to my computer. Would a 5M limit on files in /tmp be reasonable? Do they go away at the end of each day? Someone once posted something in /tmp for me to download. I rarely use grex to download anything over about 250K from the web but would do larger files if we ever got faster modems. I like using lynx at grex because it often goes much faster than my own copy via my ISP and then to download something would mean hanging up on grex and dialing the ISP if I did not do it this way. Lynx no longer works at Driverguide.
Will the quotas apply to e-mail? I don't store files on Grex, so don't have any personal concerns about quotas. I'd say this is the sort of thing where you try something, then if it doesn't work, you know more and you adjust accordingly.
Yeah, that attitude sounds right to me, too.
I think the soft limit is important to avoid people's conference participation files from getting trashed without warning if they accidentally bump above the limit.
What, exactly, is wrong with the storage of graphics? Certainly it can't be bandwidth, because with the faster NextGreX machine, it should take that in stride, no?
Yes, it's bandwidth, the number of bits that can pass in and out on the network connection. A faster CPU, or a bigger disk, does not improve the network bandwidth.
If we increased the quotas enough to allow folks to put web-accessible pictures on Grex, would that traffic noticeably slow most user's sessions? I'm *really* hoping that Grex can be significantly faster on the new machine. Unless Grex is faster I expect fewer and fewer users will stick around long enough to get hooked. Don't know if that helps but it is where my concerns rest.
Regarding #20; That's never been measured, and is only a conjecture.
You're all silly-asses.
I think the ban on graphics was also intended to keep people from putting up porno sites. The target audience for ASCII porn is pretty small.
Which is conjecture, Dan? That a faster CPU and more disk space won't improve network bandwidth? Or that the effect of allowing graphics files on grex will be a worse experience for interactive users? Either way, I've started a new item, #63, for the discussion of grex's policy on multi-media files.
gelinas, you messed me up now :-0
Regarding #25; The conjecture is that grex's network bandwidth is at the point of being exhausted.
The bandwidth does not have to be at the point of exhausation for the additional network load of graphic files to be noticible. But let's take it to the next item. ;)
These numbers seem as fine as any to start with, Joe. I would like to keep an open mind about raising the numbers if we find folks are bumping into the limits too often in legitimate uses of the system.
Daniel Gryniewicz commented on this subject in e-mail:
The problem with 2, or 3, or 5, is that you can download and
build eggbot in that much space...
This response has been erased.
Is that supposed to be 2 and 3 MB, Ryan?
This response has been erased.
This response has been erased.
Timing. ;)
This response has been erased.
Jamie, I thought EVERYTHING you posted was ACSII PORN.
This response has been erased.
re 37 Actually, that's twinkie.
quotas and e-mail need special consideration. without quotas on the mail spool abusers will just e-mail themselves things, perhaps even work within the spool if it's user writable (i forget if obsd uses a mode 1777 spool). but if the spool has a quota then there is a way for a user's quota to be exceeded which is not interactive and thus invisible to the user, and in fact happens as a result of forces typically outside their control, i.e., spam, worms and abusive action can disable e-mail reception -- staff could not even deliver a warning without an enhanced remote access mechanism or some fancy footwork quota-wise. as it happens i favor having quotas over not having them, but you need sufficient status visibility in all reading modes. unfortunately such visibility is not available by default in popular and free tools, thus would require some custom patches or spending money. these days spam/ worm containment with a much higher (perhaps shared) quota is also necessary, but again care is required otherwise abusers will try to store things in the quarantine.
Another question has come up: Should we put a limit on the number of files a user can create? If so, what should it be set to?
Can you put a limit on the number of new items a user can create in one day, for instance 3 or 5? I don't see why you need to limit the number of files if you are limiting disk usage already.
This response has been erased.
Ryan is correct - under Unix, each disk partition has a set maximum number of files, equal to the number of slots in the "inode table". So a possible denial-of-service attack would be for a user to fill up the inode table. Then no other user on that disk partition could create new files, even if there were plenty of free space on the disk. So it sounds like we should set a maximum number of files per user. I'm assuming the quota system lets us do that. On NextGrex as currently configured, the user partitions /a and /c combined have over 5 million inodes, so even being generous and assuming that we grow to 20000 users with an average of 50 legitimate files apiece, that would take up less than 20% of the inode space. So the limit could be pretty generous and still avoid a problem. If we set the maximum at, say, 5000, a twit would have to create a few hundred accounts to run the system out of inodes. That's a pretty good deterrent, and even if they persisted and tried, the activity would be noticed and stopped long before the limit was reached.
The few places I've seen quotas, the file limit was 1,000. Would that be a reasonable place to start, bumping it up later if necessary?
I think I have between 50 and 100 files which I thought was a lot. 2M disk space and 1000 files would be 2K average per file - do people have that many small files?
This response has been erased.
compilations can create a "lot" of files, but any compilation that would create that many files would surely hit a 2MB limit before an inode limit. someone might have a one-file-per-entry type of webboard, which could create lots of small "ROTFL!!!" and "LOL OMG WOT U SAY?" response files. there's probably a few other practical-ish cases where this would happen, but not that many.
I support the higher limit proposed by remmers.
Like disk space, inodes have both soft and hard limits. How about 4000 soft and 5000 hard?
I don't normal users of the system will generally approach either of those limits, so no strong opinion either way.
Big packages like eggdrop tend to use up a bit more than 1Mb, and have hundreds of files even before compilation.
This response has been erased.
I support the higher limits proposed by me. :) My philosophy in setting limits is to set them so as to avoid system problems but to make them as generous as possible within that constraint. What do we gain by making them any smaller?
I like soft limits for the warnings they give. That's the only thing gained by 4000/5000.
Sounds reasonable.
Yeah, and we can always up it later on if we find this is too low.
Makes sense to me.
Perhaps we could set the hard limit to double the soft limit, to accommodate users like Sindi (not crackers, script-kiddies, or graphics-file uploaders.) Or bump up the hard limit for members? 2, 3, or maybe 4 for a bunch of text files and small executables sounds fine to me.
If grex would just install links (or links2) I could delete the copy I compiled and use much less disk space. (Twenex, how much am I over quota?). Is there going to be some way for people to contribute programs that are kept in shared disk space? (Oops, I also have a large pdf file here temporarily because I use grex to transfer files between my three residences).
Any one can ask that a program be installed at any time. Then it's just a
matter of staff getting the time to install it.
FWIW, links is installed on the new grex machine right now:
} grex:gelinas {110} links -version
} Links 2.1pre14
} grex:gelinas {111}
Re: #60. To find out the amount of disk space you are using, type "du -h ~" (Disk Usage [in] -human-readable format [shows in kilobytes, or megabytes instead of bytes], [directory [folder]:] ~ [shorthand for your homedirectory.]) If you are in, or cd to, your home directory you can leave out the ~. "Quotas" on nowGrex are managed manually by staff; on NextGREX you will be notified by the system when and by how much you are over quota.
Yeah, Cindi requested links months and months ago, so I installed the copy from the ports tree. However there are about a zillion variations on links, so I have no idea if the one in the ports tree is the one Cindi wants. I don't understand wny setting the soft limit so much higher than the hard limit makes sense. Since we already increase the limit for any user who gives us a vaguely sensible explanation for why they should have a higher limit, I don't see a need to give members a higher limit. For the time being, Grex staff has no problem with Cindi using lots of disk space. Keeping a copy of links2 around is a more than vaguely sensible explanation.
The reason I suggested a meg between the soft and hard limit was that I find it fairly easy to fill up a meg. If we've the disk space to give each user five megabytes, we aren't hurt by giving them plenty of warning as they approach that limit. FWIW, I don't see a reason to set the soft limit at half the hard limit.
Thanks Joe. I think while you were not looking links may have evolved to version 15 and it will probably keep evolving. It handles some but not all javascript now. Jan, what do you call the links that you installed and how would I access it? I can't use it by typing 'links' so I still use my own from my own subdirectory (links-0.96 not elinks or links hacked or links2). Are all the program files in one directory somewhere?
It's on the new machine, not the current one, Sindi.
I'm not installing anything on the old machine these days.
I think Sindi would be a great beta tester for NextGrex, at least the text-based-over-dialup stuff.
I second that.
Yeah, she's on my list.
I did a whole year of beta testing for Newdeal. I found lots of small bugs in keyboard navigation, which nobody else was using, and in text-based printing, and in the older GUI. I was the resident ignoramus who had not used any GUIs before. Is it possible to let everyone be a beta tester for NextGrex? I. e., can one phone line be hooked to NextGrex for dialin use only, without any web connection? Just to test the bbs, for instance. Or could people choose to telnet to NextGrex?
Well, last time I tried to dial into Grex, it took me an hour to find a computer with a working modem, install comm software, and figure out how to make it work. Very soon I plan to start letting some people onto the system to start banging on things, but I want to limit the set of people. I'm pretty sure security is good, but if there are security holes to be discovered, I'd rather they were discovered by someone I trust. And I'd rather keep the dialins hooked to Grex.
TROGG IS DAVID BLAINE
You have several choices: