No Next Item No Next Conference Can't Favor Can't Forget Item List Conference Home Entrance    Help
View Responses


Grex Mnet Item 69: M-Net Downtime status, June 5th, 2002
Entered by hash on Wed Jun 5 16:22:43 UTC 2002:

This morning I tried to log on to m-net and found that my password 
didn't work.   It looks like something corrupted the password file.  I figure
one of two things happened.   either 1) we crashed while the passwd file was
rebuilding from a newuser or some other event that rebuilds the password file,
and it got corrupted.  or 2) the xreap process that ran last night was the
first one in a week or so, due to a problem with the cron daemon, and
corrupted the password file.  or maybe the crash happened when the password
file was rebuilding from xreap.  either way, the password file is corrupt.

a side note that complicates matters:  early yesterday when I realized cron
hadn't been running for a while, I ran my script that backs up the password
file and it removed all of the oldest backups, because it does that every time
it runs.  So, basically, we have no good backups of the file on the machine.
our only hope is the last tape that was in the drive.  and that was the last
time I went to wwnet... a couple weeks ago :-/

anyhow, when I got to wwnet this morning I figured all of this out.  I noticed
that wwnet has a nice bug problem in the server room there, they have a nice
little nest of these little flying ants.  I also noticed the side fan on our
machine was dead.  so, after getting cleared by the intense security at wwnet,
I took the machine and I have it here at wcc.  I'm going to put in our two
new donated drives, piece together the password file the best I can, and
possibly do some other upgrades.   

I'm betting we'll come back up on friday morning.

120 responses total.



#1 of 120 by edina on Wed Jun 5 16:57:23 2002:

Thank you for the update, Rex.


#2 of 120 by jerryr on Wed Jun 5 17:09:39 2002:

ditto.


#3 of 120 by jor on Wed Jun 5 18:26:09 2002:

        Damnation.

        Thanks hash



#4 of 120 by styles on Wed Jun 5 18:51:51 2002:

yeah, thanks rex.


when's the board meeting?  tomorrow?


#5 of 120 by void on Wed Jun 5 19:13:14 2002:

Thanks, Rex.

Yep, the BoD meeting is tomorrow at WCC.


#6 of 120 by ric on Wed Jun 5 20:46:24 2002:

/cues Michael Bolton "How am I supposed to live without you?"


#7 of 120 by tonster on Wed Jun 5 21:32:21 2002:

That's too funny about WWNet.  They actually ICQ'd me this morning 
saying 'uhh...there someone here claiming to be from Arbornet...Rex?  
He's trying to take the box.'

I was tempted to have them call the police, but I let it pass.


#8 of 120 by hash on Wed Jun 5 21:33:42 2002:

heheh.  
yeah, the board meeting is tomorrow in BE 172 on WCC's main campus.


#9 of 120 by ryan on Wed Jun 5 22:00:32 2002:

So it looks like literal bugs caused mnet to be hosed...
... guess it's better than a script kiddie with the rootkit du jour.


#10 of 120 by hash on Wed Jun 5 22:09:06 2002:

I don't think the insects caused m-net problems.  but, they are a little
disconcerting.


#11 of 120 by krokus on Wed Jun 5 23:38:05 2002:

Fun stuff.  Thanks for the work.


#12 of 120 by iggy on Wed Jun 5 23:39:33 2002:

yeah.. hope to see you all over there soon.


#13 of 120 by cyklone on Thu Jun 6 00:43:24 2002:

<wonders what chamberl and juno do when m-net goes down>


#14 of 120 by polytarp on Thu Jun 6 00:44:33 2002:

It'd be funny is hash was in jail now!
Well, not really funny, but iteresting.


#15 of 120 by jor on Thu Jun 6 01:53:23 2002:

        It'd be serious if M-Net didn't have security.
        I'm curious about the non-insect bugs.


#16 of 120 by tod on Thu Jun 6 02:08:50 2002:

This response has been erased.



#17 of 120 by jmsaul on Thu Jun 6 02:14:41 2002:

Re #13:  Each other.  (Have fun sleeping tonight.)

Thanks, Rex!


#18 of 120 by tod on Thu Jun 6 02:24:45 2002:

This response has been erased.



#19 of 120 by cyklone on Thu Jun 6 02:29:24 2002:

Now I'm *really* not sleeping tonite!


#20 of 120 by jp2 on Thu Jun 6 02:48:59 2002:

This response has been erased.



#21 of 120 by md on Thu Jun 6 11:40:24 2002:

I hope it was just the insects, and not some hacker, like the kid last 
time whose online attack was so viciously devastating that it 
physically destroyed the entire system, which then had to be replaced 
at a cost of thousands of dollars which the court ordered the kid to 
pay back to arbornet in monthly installments.  Is he still sending his 
little checks, btw, or has someone from arbornet finally told the court 
the truth?


#22 of 120 by russ on Thu Jun 6 11:56:08 2002:

Good work, Rex.  Let's hope it's simple and easily fixed, or that you
can recover the inode of one of the backup password files.


#23 of 120 by slynne on Thu Jun 6 17:38:35 2002:

Rex, what would Mnet do without you?


#24 of 120 by jor on Thu Jun 6 17:46:11 2002:

        when it's down it's M-Not


#25 of 120 by tonster on Thu Jun 6 18:51:36 2002:

#21: It's been said numerous times.  Noone ever told them he physically 
destroyed the machine, and several people specifically told them that 
we wanted no money from him.  We'll take the checks and we'll cash 
them.  We didn't ask for the money, and we didn't mislead or lie to 
anyone.


#26 of 120 by tod on Thu Jun 6 19:25:17 2002:

This response has been erased.



#27 of 120 by md on Thu Jun 6 19:27:21 2002:

I beg your pardon, *someone* told them he physically destroyed the 
machine.  Read the press release from the AG's office.  Youse guys got 
selective amnesia real bad.


#28 of 120 by jmsaul on Thu Jun 6 21:36:55 2002:

Nobody said that he physically destroyed the machine.  I, at least, explained
what actually happened to them in great detail.  A couple times.

Why, exactly, does it surprise you that the facts could be wrong in a press
release from the AG's office?


#29 of 120 by md on Thu Jun 6 23:20:48 2002:

It doesn't surprise me at all.  Someone gives them bad info, they base 
a press release on it.  Easy, no?


#30 of 120 by styles on Thu Jun 6 23:25:02 2002:

or, they kinda misinform themselves.
"Arbor Net, an internet service provider"?  yeah, someone musta told them
that, too.


#31 of 120 by cyklone on Thu Jun 6 23:25:50 2002:

Another possibility is that someone explained what happened in terms they
thought were understood, but actually weren't.


#32 of 120 by hash on Thu Jun 6 23:59:31 2002:

m-net's problem was, and had been, a flakey hard drive.  I'd been meaning to
replace it with the two donated drives we recently received.  we'd been
rebooting every day or so for the last couple months.  no one really notices
because it comes right back up.  

I'm sorry to say that I got very busy at work today and spent hardly no time
working on m-net.  it's current status is that I've got the new drives in it,
and I'm planning on reinstalling FreeBSD from scratch and then copying the
data over to the new drives.  (data that I've already copied off of m-net's
old disk on to a temporary IDE drive.)  We've also done considerable work
inproving m-net's air flow inside the case.  making sure all cables are tucked
away and installing two new fans in the case.  Todd has graciously donated
monies for a new power supply and I have an order in for a new drive bay fan.

all in all this should be a very good upgrade for m-net.  and hopefully it'll
be back up and running sometime this weekend.


#33 of 120 by jp2 on Fri Jun 7 00:26:37 2002:

This response has been erased.



#34 of 120 by tod on Fri Jun 7 01:32:24 2002:

This response has been erased.



#35 of 120 by hash on Fri Jun 7 03:40:03 2002:

vinum.  it's arguable wether any OS solution will be a great advantage.  but
I'm thinking it's the best thing to do with the hardware we've got.


#36 of 120 by twinkie on Fri Jun 7 13:12:22 2002:

What size power supply does the system need? I have some 300W ones lying
around, IIRC. (I'd check, but I'm at work right now)

If that would work, I'd be happy to donate one, and we could put whatever
money tod donated to something else.



#37 of 120 by hash on Fri Jun 7 13:17:43 2002:

we've got two scsi hard drives, an ide drive, a tape drive, and two extra case
fans.  I was just going to get a 500 watt one.   todd told me to buy one and
he'll pay me back.


#38 of 120 by jmsaul on Fri Jun 7 13:50:05 2002:

Re #31:  No way.  You're telling me it's possible for a prosecutor to make
         a mistake about something technical?


#39 of 120 by sq2 on Fri Jun 7 15:43:16 2002:

a 500 watt ps would rock



Next 40 Responses.
Last 40 Responses and Response Form.
No Next Item No Next Conference Can't Favor Can't Forget Item List Conference Home Entrance    Help

- Backtalk version 1.3.30 - Copyright 1996-2006, Jan Wolter and Steve Weiss