78 new of 80 responses total.
The next Grex Board of Directors meeting is Monday, June 28 at 7:00 p.m. in the Michigan Union Food Court. See item 107 in the Coop conference (item:coop,107) for the agenda. Note the change in location. Our regular meeting place was not available this month.
Grex took longer to reboot than normal Friday night, due to a problem with a disk. We have some bad sectors on /a, which I don't think present an immediate problem. We're going to have to deal with this in the near term however, so more downtime will result. More once we know more about it all.
"No space left on device...file system is full."
The Grex Board of Directors will meet this Monday, July 26, 7 p.m. upstairs at Zingerman's Next Door, 422 Detroit Street, Ann Arbor. The public is invited. See Item 112 in the Coop conference (item:coop,112) for the agenda.
Grex may be shutting down as of Sunday, unless the lawsuit mentioned in
#2 results in a quick injunction. From the minutes of Monday's Board
meeting:
AGENDA ITEM 6: ACLU Suit
- A hearing was held in which the ACLU asked for a temporary
injunction.
If granted this would prevent Michigan's new Internet Censorship Act
(Michigan Public Act 33 of 1999) from going into effect on the
scheduled
date of August 1, 1999. Jan Wolter was called as a witness. John
Remmers, Steve Gibbard, Mark Conger, and STeve Andre attended. It
appears to have gone extremely well, and everyone is confident that
the injunction will be granted. The ruling will be announced before
August 1.
- There still needs to be a trial to determine the constitutionality
of
the law. This is likely to happen 6 to 9 months from now.
- Although we think it almost a certainty that the injunction will
be granted, the board felt it would be prudent to have a plan of
action in place in case it was not. Figuring out whether Grex can
continue to operate in any way under this law is going to be
extremely
difficult and will probably require getting legal advice on a number
of points. We don't want to go to the trouble of formulating this
plan unless we need it. Mary Remmers proposed that if this law
comes into force, Grex should temporarily shut down while a policy
is worked out. This lead to the following motion by Jan Wolter:
In the event that Michigan Public Act 33 of 1999 goes into
effect,
all public access to Grex shall be suspended, with the exception
of an informational web page, pending the formulation of new
policies.
Seconded by Dan Gryniewicz.
Passed 7-0-0.
Again, we do not think there is any large chance of this happening,
and
we think that it may be possible to bring at least a few services
(like Email) back on line pretty soon.
Will this shutdown permit email forwarding?
I don't know. There was very little advance discussion, and there is no information available other than the minutes of the most recent Board meeting.
I dunno, shutting down seems pretty drastic.
A partial shutdown is drastic but probably the best choice of action until we can get some good legal advice on where the board and users would stand in terms of liablity should this law go into effect. This is being discussed in Co-op, items #114 and #113. Check it out.
Yes, it would be drastic. Also, I think it's unlikely to happen. The possibility is being discussed in item 114 of Coop (item:coop,114). I'm the board chair, so I posted a longish response there (#8) to attempt to explain where the board is coming from.
(Mary slipped in.)
Such an amusing slip :)
(We're a 2-computer, 2-modem household, so these things happen...)
I am HAPPY to announce that Judge Arthur J. Tarnow of the United States District Court, Eastern District of Michigan, Southern Division, has GRANTED Grex's and other plaintiffs' request for an injunction preventing enforcement of 1999 Public Act 33. In plain language, that means we won. The internet censorship law will not go into effect on August 1. There will be no disruption of Grex's services. I think we owe many thanks to the ACLU attorneys who developed and presented the case, and the Grexers who put a lot of work into it, especially Jan Wolter (our declarant and witness), Mark Conger (our contact person with the ACLU), and Mary Remmers (our press contact). The judge has issued a 30-page opinion, which I'll post online as soon as possible.
Temporarily, the judge's opinion is available from my personal
web directory, in Adobe PDF format. See
http://www.cyberspace.org/~remmers/opinion.pdf
Not everyone can view PDF files, so I'm looking into getting it
converted to something else, like plain text or HTML. I don't
seem to have software myself that will do this.
Thanks, John! Interesting document.
This response has been erased.
Well then we'd be faced with converting ps to html, which if anything is harder. But you knew that. :) I think someone is doing the conversion. Hopefully a link to the result will find its way onto the Grex 'lawsuit' web page before too long.
An HTML version of the ruling is at http://www.cyberspace.org/lawsuit/injunction.html
Congratulations to Grex - and all those that put in the effort on this action - for a successful outcome.
Thanks, Jan. I'll let Mr. Steinberg know it's up.
the idle buster doesn't seem to be working...
I have a MS Word version of the opinion at http://www.cyberspace.org/~dang/opinion.doc
Is it really necessary for 8 pages of mostly doublespaced text to take up 97K?
Out of curiosity, am I the only one having problems getting on with dial-in? I've been dialing in, connecting, and then just hangig for 5-10 minutes before I give up and try several hours later.
It is if you have all kinds of typesetting information involved. Think of it as a stored picture, and you won't be too far off.
HUH?????
Uh, eeyore, resp:28 was in response to drew's resp:26, not your res p:27
Okey....I'm happy then. :)
(And my resp:30 showed up all on one line when I entered it...)
An online vote to rescind the board action referenced in resp:7 is now underway. See Item 114 in the Coop conference for discussion. To cast a ballot, telnet or dial direct to Grex and type 'vote' at a Unix shell prompt or '!vote' at any other prompt. The polls are open through the end of the day (EST) on Sunday, August 22.
Grex was down for several hours (about 5pm to 11:30pm) Friday. A system file had its contents changed, and was writable to the world. This is of course Not Good and for a little bit I thought we'd had a real security breach, and took Grex down. This is of course the thing that all staff fear the most, that someone has figured out some new way of getting into the system and becoming root. There haven't been many times that I've thought that this might have happened, but this was one of those times... As it turns out, the file in question had the wrong permissions because of the way the system booted up, and although we specified a certain mode for the permissions (read only to the world) dear old SunOS had a different idea. We took out the code that caused this to happen, and all is well now. Also tonight was the testing of a new method of dealing with fork bombs, which is faster than previous forkbomb control--this one should kill forkbombs nearly instantly. We tested it a bit and its now running.
text help a: a text
Re #34: What the heck are fork bombs?
Forking is the term used when a running program splits itself into two running programs. This is often done when a program wants to run another, for example. Forking is a good thing. However, a runaway forking program is a monster, creating endless copies of itself and ultimately clogging the operating system by hundreds of copies of itself, to the detrement of the rest of the system. A forkbomb is a very small program which does nothing but fork itself so very quickly the compter is doing nothing but dealing with all these tiny little programs whose idea of a god time is to replicate. Think of them as software tribbles and you have a good model. The new anti fork-bomb code deals with this kind of problem very quickly.
I prefer to think of them as process-table cancer, though the "software tribbles" analogy works, too..
You can look at "insan3" for a typical example of a fork bomber.
PTC! I love it! Thanks, Mike...
Thanks for the explanation Steve.
then why isn't he kicked out?
also, could you hack GCC to refuse compiling programs that are plainly fork
bombs?
for those of you wondering about the bombs, here's one courtsy of insan3
main()
{
while (1)
fork() ;
}
simple as that.
btw, i was wondering if the fork-bomb killer is available to download in
source? i might be interested to run one on my comp...
Always admired Berkeley for developing an elegant solution to
fork-bombs.
"safefork" doesn't come from Berkeley. It will probably appear in the grex staff notes in due course. We'd like to make sure it's toilet trained before releasing it into the wild. "Disabling" a fork bomber doesn't work because it's probably a throwaway account, and because we don't do any form of account validation to enforce "uniqueness". For many of our users (indians, for instance) proving "uniqueness" would be difficult. What we've historically done instead, is to complain to the ISP. This is actually generally fairly effective, because running a fork bomb is in fact illegal and nobody on the internet wants to harbour criminals. On the other hand, it's a terribly time consuming process for us, because we have to comb through our records and construct a detailed report on the vandal, send e-mail out, &etc.
No, I'm assuming you folks are using tools of the homegrown variety.
I was thinking of Berkeley's process accounting and control.
under what governing body is fork-bombing illegal? if it is the US, then how does it apply to a foreign user? I'm curious about this...
It certainly is in the US, as crashing a computer falls under various Federal computer vandalism statutes (the one a lawyer told me about a few years ago was, "unauthorized destruction of data," which included data held in a computer's memory). The situation in other countries presumably depends on the other country, although I would assume that damaging a computer in the US from another country is probably illegal under US law.
MIchigan has a law about interfering with the proper operation of a computer. Thus I would imagine that the fork bomb would fall under this law. Certainly they're disrupting enough. We've had one vandal try this, twice. I guess the first time was enough of a shock that it wanted to see if it was really real. ;-)
Today, Sunday August 23, is the last day to vote on the proposal described in resp:33 . The polls will close at midnight (EDT).
Oops, slight correction - make that Sunday August 22 that the polls close. (It's still today, but I had the date wrong.)
The proposal was defeated: 16 yes votes, 27 no. (The totals are slightly unofficial, pending any changes in the membership roles in the last month, but there wouldn't have been enough changes to affect the outcome.) Unofficial non-member votes: 67 yes, 74 no. Same outcome, although the vote was closer.
Grex Board of Directors meeting tonight, Monday August 23, at Zingermans Next Door, 422 Detroit Street, Ann Arbor. See item 119 in the Coop conference for the agenda.
is there a time?
Oh, whenever everybody gets there... :) Oops, forgot to list the time. 7:00 p.m. Thanks for noticing.
Grex was down today for a reboot, which turned out to be more compilcated than usual. For the last two days the system has been running slower than normal, because of a program we ran to get rid of some vandal droppings, namely deeply nested directories. Due to various problems deep inside the kernel the program couldn't finish and we had a couple of them hanging in this strange state, using resources. Seeing that the problem was still here today, I rebooted the system. Unlike most occaisons where I've rebooted remotely, this one couldn't be done without human intervention. Thanks to Marcus for getting over and finishing the cleaning job from the vandal's droppings.
What gives with people creating recursively nested directories,
anyways? I've never seen them try that trick on any other system I've been
on.
i think that newer OSes have a limit on depth of nested directories, but i'm not sure. if they do and the other systems you have been on are newer oses then parhaps it simply isn't possible there. note this is just a hunch - i don't have the time to look in the linux source code (which is several million lines long) to check if i'm right about this.
I've seen people create deeply nested directories on m-net. This was on the altos. I remember spending some time teaching the OS how to limit directory nesting. It's not an entirely trivial problem to fix.
You start out with a deeply nested directory, like /a/u/s/user. Recursively nested - that would be like having a symbolic link to a few levels up? I guess there's no harm in it, unless you run something that recursively follows all subdirectories and treats links as actual directories.
/a/u/s/user is only 4 levels down, which hardly qualifies as deeply nested. I have pathnames at work like /afs/umich.edu/group/itd/build/mdw/krb5.cast/krb5/src/lib/crypto/des/afsstring2k ey.c and even that's not too hairy; it does fit on one line, and is only 13 levels deep, at least. The directories created by the last vandal were over 3000 levels deep, and this was deep enough to cause problems for the "SYSAUDIT" feature which we had inadvertently built into the grex kernel; this feature collects path name information for C2 security logging purposes, and apparently the feature didn't like pathnames over 64k in size. It seems, in some respects, we operate in a more hostile environment than the pentagon.
Why dontya just modify the sysaudit program to spit out a message into the sys log when there's a huge directory and skip it?
Because it's not a program, we don't have source to it, we don't need it, and taking it out is effectively the same as having it skip every directory.
Actually, taking it out was a win. IN talking with someone at MSU today who has dealt with C2 security stuff on SunOS, his comment was along the lines that we ought to have crashed into C2 weirdness before now. So just getting rid of it was a win.
I get cognitive dissonance when someone talks about crash-preventing measures as a "Win".. Too much Microsoft influence, I guess..
Mike, when you administer a system like Grex, anything you can do to stop the vandal from doing things is a "win". It's just one more item that they can't use to hurt you. Now, granted this is similar to the Microsoft style of software creation, but in Grex's case I think its pretty reasonable.
I think Mike was making a pun. Win as in Windows.
Heh.
Recursive-directory spam seems to be of the sort of low-brow attempted
hacking that'd be stopped with a few changes to mkdir.
The changes need to be in the kernel, not mkdir. A vandal could circumvent a protected mkdir program by simply calling mkdir in some code. It can be done in the kernel, but it isn't a trivial task.
To truly prevent it, yes. I'm just saying that that sort of exploit
is in most cases used by someone who wouldn't *think* nor have the resources
to write even a simple chdir() mkdir() recursive program. Come to think of
it, most programmers I know aren't capable of really recursive programming,
heh. :P
This particular vandal used a perl script.
At least 50% of the recursive dir bombs have been C code. The nastyist one creates random strings of files and dirs, then dives another level deep and starts over again. Changing mkdir itself isn't worth it.
Yup, most of the people who are stupid enough to want to do something like this aren't smart enough to write even a shell script - they use programs written by other people that they found on the web. Half of them barely know what the programs they run do - except that it is somethink K00l and hackerish. The programs themselves aren't typically very good either, since they were mostly written by people who just barely knew what they were doing, but they are very diverse. Building good defenses against them is non-trivial. We finally have good defenses against fork bombs, at least.
If there's something running around the script kids that does mkdir()
calls, then I retract my statement. I haven't seen one on any of the ususal
suspects.
If you were writing a program whose only objective was to make really deep directories, why would you recurse? I would think looping would be far more effective.
By recursing, you can make 'em deep and wide.
How would recursing help to make them wide? Are you assuming that the
recursion is finite? I mean, if you have something like (ignoring the
finer points of syntax)
foo()
{
mkdir("something");
chdir("something");
foo();
/* never reached */
mkdir("something else");
chdir(something else");
foo();
chdir("..");
}
you still won't get a tree-like expansion, you'll get a single line of
deeper and deeper nested children, one for each recursion, until
someone kills the program. So how would you use recursion to make 'em
wide, too? And why does it matter if they're wide, if the objective is
to fill up a filesystem? Won't deep do it just as well?
Maybe this question should be taken elsewhere? :)
You need to spawn() or something like that, to get wide. And do it more than once in each instance of the program. Deep gets you a linear growth, while wide gets you an exponential expansion.
It doesn't really matter. We have a nice tool that digs through and gets them all no matter if they're wide or deep. Usually, I think these people are trying to create something that's deep and "annoying" (weird filenames); probably they're hoping to core dump things like "find", "restore", and "rm -rf". Filling up the filesystem appears to be of 2ndary interest to the vandals that make deeply nested directories.
Here's how I think they do it (ignoring syntax):
foo()
for (n=1,n<1000/*Or something equally large*/,n++)
{
mkdir("%d",n);
chdir("%d",n);
spawn(foo.c);
chdir("..");
}
This would create a tree with 1000 branches at each level, and infinitely deep
(ie, until stopped by staff or some sort of filesystem limitation).
At least, this is how I'd do it.
Yup, except on Grex it would probably create maybe 50 directories before all your processes, including your login shell, suddenly and mysteriously died. These days our fork bomb defenses are very effective, and that little program is a fork bomb.
You have several choices: