|
|
| Author |
Message |
| 25 new of 467 responses total. |
janc
|
|
response 175 of 467:
| 
Oct 13 16:26 UTC 2004 |
Yes, I've read that.
Yes, I agree that we should set a softlimit just a little smaller than the
hard limit - maybe softlimit 1000K, hardlimit 1050K. This gets you a warning
message 50K before you run out disk and things stop working.
I was baffled by that comment on /tmp too.
I've read the edquota docs and also the setquota docs (it's available from
the source tree) and I think they are too clumsy to use directly for quota
administration. My first thought was a wrapper, something like 'lockuser'
or 'zapuser' that would log the date a user's quota was changed and the
reason. Instead of specifying disk usage on /a, /c, /d, /e, /tmp, etc,
you'd specify usage on 'home' (the partition there home directory is on)
and 'nonhome' (all other home partitions - this would virtually always be
zero), and 'tmp'. You wouldn't have to set both soft and hard limts -
config file would give difference.
I'm not sure, but I think it might be good to have a quota database other
than the raw quota file that would keep track of what user's quotas are
supposed to be, and from which the system's quota database could be generated.
The more I thought about this, the more I though what we really need is a
comprehensive user database, in SQL. As each new account is created, an
initial record would be created for it, including all information about the
user other than passwords. Current quota settings would be there. We'd
also log status changes to the database - creation, deletion, locking,
group change, disk quota change, etc. Maybe monthly we'd log usage info
for each user - time spent logged on, amount of use of ftp, backtalk, etc.
This is all information we already keep, but it is difficult for us to
pull it together now.
We've long wanted something like this so we could do things like send mail
to users who had been here for three months asking them to consider membership.
Things like the project Joe mentions (figuring which IRCers had not been
locked but had been reaped) would be much easier with this. We'd be able to
get a better picture of how people were using Grex.
However, this is not a project to put on the straight line to getting NextGrex
up. It'll keep for later.
Here's my plan for what I'm going to do now:
- Make /var/tmp a symbolic link to /tmp
- Modify newuser so the config file can specify which disks have quotas,
with syntax something like:
qdisk home /a
qdisk home /c
qdisk tmp /tmp
and specify initial settings for soft/hard block/node for new users:
quota home 1000K 1050K 500 510
quota nonhome 0 0 0 0
quota tmp 50K 55K 50 55
I'll spend some time researching sensible values for inode and tmp quotas
but not too much - we can fiddle with that later.
I might also do a simple wrapper for setquota that lets you change a user's
home quota easily and log the reason.
|
gelinas
|
|
response 176 of 467:
|
Oct 13 21:37 UTC 2004 |
So I investigated. From the edquota manpage:
Setting a soft limit to one with a hard limit of zero indicates
that allocations should be permitted on only a temporary basis
(see -t below).
and
The -t flag can be used to change the grace period.
So *if* this works as described, the files would be removed at the
end of the grace period. 'Twould be worthwhile to test, I think.
Right now, the grace period on the new grex machine is one week.
See /usr/include/ufs/ufs/quota.h
|
mfp
|
|
response 177 of 467:
|
Oct 13 22:05 UTC 2004 |
I happen to have used nedquota manpage on vario'puters, and *have* found it
to be less than unsatisfactory.
Schwa.
|
twenex
|
|
response 178 of 467:
|
Oct 14 00:07 UTC 2004 |
Speaking of "home", why are the different user filesystems in the root instead
of in /home? (Yes, I know it's so we can split them up so that the system
doesn't have to search through thousands of directories corresponding to
logins - but why not put ./a, ./b, etc. under /home? I admit it's an asthetic
issue - imho, root file systems shouldn't be crowded with non-standard stuff,
especially since nowadays we have /root to serve as "root"'s homedir instead
of / - but it would also mean that we could also put all the home directories
on one disk in one fell swoop if we ever ran out of space on them, just by
moving /home.
Are there any plans to increase the maximum available amount of space each
user gets?
|
gelinas
|
|
response 179 of 467:
|
Oct 14 00:32 UTC 2004 |
Take a look at Item 62 in coop, twenex. I'd be interested in seeing any new
thoughts on the subject.
|
twenex
|
|
response 180 of 467:
|
Oct 14 01:49 UTC 2004 |
Thanks.
|
janc
|
|
response 181 of 467:
|
Oct 15 14:41 UTC 2004 |
I went and read the kernel source, and I'm pretty sure I understand all this
now. The operating system never automatically removes overquota files. If
you perform an operation that hits a quota, then one of two things happens.
Either you get a warning message printed to your terminal and the operation
continues, or you get an error message and the operation fails.
If go over the soft limit, you get the warning message, unless you have been
over the soft limit for more than a week, in which case you get the fail.
If you attempt to go over the hard limit, you always get a fail, so you
can't go over the hard limit.
So the "soft limit" is the limit we want to advertize to the users. All
users should always be below the soft limit. The higher hard limit provides
just a little niceness. If you are doing some complex operation then it
has a bit of a better chance of running to the end instead of failing in
the middle, but we still expect people to clean up their disk usage after
the end of that operation, and we don't really expect ordinary Grex users
to be doing lots of operations on really large files. Also nothing is going
to automatically bump people down below the soft limit. So I feel we should
keep the hard limit only a little higher than the soft limit - maybe an
extra 50K or 100K.
Joe's notes suggest a block limit of 2000 and a inode limit of 4000. I
don't think that makes sense. Every non-empty file and every directory is
at least one block. So the only way you could hit that inode limit is to
create at least 2000 empty files. No grex user needs 2000 empty files.
My thought was to set the inode limit to about 2/3 the block limit. I
feel that anyone hitting that is doing something sufficiently weird that
maybe they should be talking to staff. But I suppose there would be
no particular harm in just setting the inode quota equal to the block
quota. Inodes aren't really a scarce resource.
Historically, home directories were put under / and given short names to keep
the /etc/passwd file smaller. That's certainly ancient history now, but one
name is as good as another, and I like having some continuity with Grex
history.
By the way, the reason there is no /b was that /b was a link to the bbs shell.
Writing /b instead of the full path was another way to keep /etc/passwd small.
|
janc
|
|
response 182 of 467:
|
Oct 15 17:26 UTC 2004 |
So I've been working on quota stuff.
I've got quota code patched into newuser and web newuser. Marcus had
some quota code in there, but it was inadequate for our needs (supported
only one filesystem with quotas), and wasn't entirely correct anyway.
So you can now edit the initial quotas from the config file and they all
get set right. I should note that the OpenBSD manual page for quotactl
is completely inadequate. As usual, I had to read the kernel source to
figure out the correct way of calling this thing.
I've turned up some additional problems with newuser. There was some
bad pointer arithmatic that used to squeek by but broke when OpenBSD
changed around it's memory mapping for security reasons. It's not safe
to assume you can add together two pointer addresses without an
overflow, even if you plan to divide the sum by two to get the middle of
a region of memory.
Another bit of fun occurs because of redundant attempts to disable
coredumps. I had modified newuser to do its passwd file editing via the
standard OpenBSD routines for this. Those routines start out by setting
the hard limit on the coredumpsize to zero. This would be important to
do on systems where coredumps are readable, because the coredump could
contain fragements of the password file, but OpenBSD does not have
readable core dumps, so it's just some redundant paranoia. This starts
to cause problems because newuser finishes by running the 'login'
program to log the user in. Because of this, the user's shell, on the
first login only, inherits a hard coredump limit of zero. Now the
default .login file for the menu shell does a 'limit coredumpsize 0'
because most menu shell users don't need core dumps. This command sets
the soft coredump limit to zero. It ought to be legal, but the OpenBSD
setrlimit() call seems to generate an error if you try to set the soft
limit to zero when the hard limit is zero. This causes the limit
command to fail in a way that terminates the execution of the login
script, so the 'menu' shell itself doesn't get run, leaving the
bewildered newuser in the csh shell, with an incompletely configured
environment. So we've got some stupidity in the kernel and some
stupidity in csh (it shouldn't stop execution of the .login file), but I
gotta fix it in newuser by resetting the hard limit to infinity before
giving up root status so that it can be set back to zero.
So I'm making progress, but I'm not exactly willing to give any
estimates about when this is all going to be done. It's hard to
anticipate problems like these. Oldgrex is the result of thirteen years
of tuning by dozens of staff people. I've only got a couple of hours a
day I can put into this stuff.
|
janc
|
|
response 183 of 467:
|
Oct 15 17:29 UTC 2004 |
I also wrote up a simple little wrapper script for the 'setquota'
utility that lets staff people easily change user's disk quotas and logs
what they did. I should look for a tool to list users who are over
quota. We'll need that when we move users over from oldGrex. We'll
need to manually go through all over-quota users and decide either the
up their quota or take other action.
|
albaugh
|
|
response 184 of 467:
|
Oct 15 17:39 UTC 2004 |
As always, continued thanks to janc for taking this on himself.
|
aruba
|
|
response 185 of 467:
|
Oct 15 18:50 UTC 2004 |
Likewise. It's like sweet cognac every time I read a report of progress.
|
janc
|
|
response 186 of 467:
|
Oct 15 19:13 UTC 2004 |
OK, ran into another annoyance.
The standard .login file issued to all new users contains this line:
tset -m 'dialup:?xterm' -m 'network:?xterm' "$TERM"
This does various terminal initialization. If your terminal is "network"
or "dialup" when you login in, it will change it to whatever kind of
terminal you said you had when you registered, in my case 'xterm'. Notice
that the current value of the TERM environment variable is passed in on the
command line. In OpenBSD it isn't necessary to pass it in on the command
line. If you leave it off, it fetches it from the TERM environment variable
all by itself. The problem comes in because when you pass the terminal
type in from the command line, it does not do the mappings. So to make the
mappings work, I've dropped the last argument from the prototype .login and
.profile files on NextGrex. However once we move over the grex users, we are
going to have to change some 10,000 user's dot files. Ugh. I suppose we'll
write a sed script to do this.
Actually, on most telnet and ssh connections these days, the correct terminal
type is passed across, so it won't usually say "network" or "dialup" to start
with and the mapping won't be needed. However this is never the case with
dial-in connections (which actualy probably arrive saying "network" not
"dialup" because they all are telnets from the terminal server.
|
marcvh
|
|
response 187 of 467:
|
Oct 15 20:29 UTC 2004 |
(is Cognac supposed to be sweet? I guess if you mix it with something like
Chambord..)
|
janc
|
|
response 188 of 467:
|
Oct 15 20:44 UTC 2004 |
I'm seeing various apparant inconsistancies in the quota stuff. I think
it all comes down to the fact that the block size on OpenBSD is really
512 bytes, not 1024 bytes, as I read somewhere. I'm going to have to
revise my newuser hacks to suck the correct block size out of param.h.
Need to investigate this further.
There's also an issue with MANPATH. The one set in most user's dot
files on Grex doesn't work on nextGrex. With luck, I'll be able to fix
this with a few symlinks. Or another sed script.
I don't have my to do list in front of me, but I other tasks that I can
think of that still need to be done are updating the log rolling scripts
and fixing several recently reported bugs in fronttalk.
Sometime soon I want to try to recruit more people to try testing
things, and try to run some of that in parallel with some of the
pre-move tasks I have to do.
Pre-move tasks include copying over all of Grex's web pages, board
minute archives and such slowly changing stuff. I want to do this while
Grex is still up. I'll be advising the people who routinely maintain
such things that they should suspend changes that they don't want lost
until we are on the new system. Mostly those only get changed by a
small set of people, so this should be OK.
The other pre-move tasks are preparing for the actual move. We will
need to renumber groupid. I think Dan Cross once built some tools for
that that I need to review. I'll need scripts to modify dot files as
discussed above.
Then it's time to shut down Grex and start the change over, moving bbs,
user files, etc. I think we're within a week or two, but it's hard to
be sure. I've spent most of the day today dealing with unanticipated
problems, who knows how many more await.
|
aruba
|
|
response 189 of 467:
|
Oct 15 22:33 UTC 2004 |
I was bluffing - I don't actually know what cognac tastes like.
Jan, could we write a wrapper for the tset function which would act like the
SunOs 4 function?
|
janc
|
|
response 190 of 467:
|
Oct 15 23:09 UTC 2004 |
Yes, but I'd rather not. I don't want to try to turn a 2004 edition of
OpenBSD into an emulation of SunOS. I'd rather take the one time hit of
editing 10,000 .login files than forever kludge up tset to behave like
an older and more obsolete version. Creating symlinks to make the old
MANPATH is different, because it doesn't prevent the new MANPATH from
working.
|
janc
|
|
response 191 of 467:
|
Oct 16 02:06 UTC 2004 |
I think I figured out the source of my confusion about block size and quotas.
The 'quota' command says says:
Filesystem blocks quota limit grace files quota limit grace
/a 7 1074 1024 6 1953 1862
The 'edquota' command says
/a: blocks in use: 7, limits (soft = 1074, hard = 1024)
inodes in use: 6, limits (soft = 1953, hard = 1862)
They both say "blocks" but they both lie. The source code for 'edquota' for
example looks like this:
fprintf(fd, "%s: %s %d, limits (soft = %d, hard = %d)\n",
qup->fsname, "blocks in use:",
(int)(dbtob((u_quad_t)qup->dqblk.dqb_curblocks) / 1024),
(int)(dbtob((u_quad_t)qup->dqblk.dqb_bsoftlimit) / 1024),
(int)(dbtob((u_quad_t)qup->dqblk.dqb_bhardlimit) / 1024));
So it prints "blocks in use" then takes the numbers of blocks, feeds them
through dbtob(), which converts "disk blocks to bytes" and divides by 1024
to get kilobytes.
So both 'quota' and 'edquota' say "blocks" but output "kilobytes". Since
on nextGrex a block is half a kilobyte, this makes a big difference.
I've sent a bug report to OpenBSD.
But once again, the only way to figure out what OpenBSD is doing is to
read the source code. I'm begining to miss working on exim. At least on
exim the documentation said what the code did, and the code did what the
documetation said. The OpenBSD folks write very nice code, often with
clear comments, so reading the source isn't as bad as it sounds. But I
wish they wouldn't keep getting sloppy and vague whenever they are
communicating to a user.
|
janc
|
|
response 192 of 467:
|
Oct 16 02:12 UTC 2004 |
Note that this changes my previous notions about inodes. Since blocks are
smaller than I thought, you can fit twice as many files into a 2 Meg limit,
but one block files are going to be less common than I previously thought.
I think I'll start out setting inode limits to 2/3 of the block limit, or
4/3 of the kilobyte limit. If that doesn't work out well, we can always raise
it.
|
janc
|
|
response 193 of 467:
|
Oct 16 15:36 UTC 2004 |
Nobody showed for the Grexwalk, so I came home and started fiddling some more.
Finished tests of quota code in newuser and finished testing and debugging
the new 'upquota' admin command.
Need to do more testing on webnewuser. Think I'm going to move over the Grex
web pages first.
|
aruba
|
|
response 194 of 467:
|
Oct 16 15:49 UTC 2004 |
Sorry we didn't make it for the walk - the weather discouraged us.
|
cross
|
|
response 195 of 467:
|
Oct 16 17:42 UTC 2004 |
Why edit the .login and other files? I would rewrite them (and the
common, global) login files to reflect the new reality, and then wholesale
replace the old ones with the new ones (backing up the old ones, of course)
and advise users who've made customizations to merge them into the new
files. That'd also give an opportunity to have a single place to make
environment updates (as opposed to hacking login and sshd).
|
janc
|
|
response 196 of 467:
|
Oct 16 21:02 UTC 2004 |
I think many users would have difficulty with this.
I would like to make more use of the global files. The default dot files
we've given out for years mostly seem to source a global file, with a comment
that you shouldn't delete that. But this is redundant because most shells
now source a global file automatically. Except I think the "rc" shell doesn't
but newuser doesn't offer that one anyway. It could all stand some cleaning
up and redesign. Dunno where that falls on the priority list.
|
janc
|
|
response 197 of 467:
|
Oct 16 21:09 UTC 2004 |
I could use a volunteer to update the copy of the Grex web pages on nextGrex.
Changes to be made:
- Talk about 'ssh' as well as telnet. Need to update the instructions
on how to connect to grex to include ssh.
- Deletion of explanation of things like the telnet queue which no longer
exist. Replacement of reference to SunOS with references to OpenBSD.
Checking all commands given to make sure syntax is still the same, etc.
- Addition of discussion of accessing Grex vi https.
- Probably other things I haven't thought of.
You'd need to know a lot about Grex and a bit about HTML to do this. You'd
need to have time to work on it SOON.
I started updating the pages, but felt like this was really a job I could
maybe find someone else to work on. Some changes have been made. Some
pages just have comments in red saying what needs to be change. Some pages
haven't even been looked at.
|
gelinas
|
|
response 198 of 467:
|
Oct 17 03:25 UTC 2004 |
I'll take a look at them tomorrow.
|
keesan
|
|
response 199 of 467:
|
Oct 17 04:39 UTC 2004 |
Thanks Joe and Jan. Can we please have at least 3 days notice when grex will
be moving. People have been complaining about a lot of bounced mail when grex
was out for longer periods and I want to warn them this time. I use grex as
a business address. I also want to warn a few other grexers whom we signed
up to tell their friends not to send mail for a few days.
Amazing that you have accomplished so much in the past few weeks.
|
