response 3 of 24:

Aug 2 21:50 UTC 2002

(the Pumpkin is cool.  I know that doesn't really add anything to the 
discussion that's likely to follow, but in my book, having a place that
Grex can call its own says to me that the system [and the organization
behind it] has grown up and become a responsible member of the community.
that, to me, is cool.  but I grant that the status symbol of having a
place of one's own may not be worth the expense, whatever that expense
is.)

(I'd also like to point out that the entire block lost power.  I don't
think co-location would make Grex more immune to that sort of thing.  it
certainly wouldn't have made it more immune to that DoS-like incident
earlier this week.  plus, it seems to me that not having physical access
to Grex wouldn't have brought it back online any sooner.)

response 5 of 24:

Aug 3 00:28 UTC 2002

Colo spaces tend to have backup power, both batteries and generators.  They
also have network connections that are a lot harder to saturate than a 160K
DSL circuit, and which if they do saturate tend to slow down the whole
operation, meaning the colo provider will be highly motivated to correct the
sitaution.

That wouldn't have helped with the immediate problem of the fried router,
though.  The fastest thing to do in that situation was to replace the router,
which was done.

response 7 of 24:

Aug 3 02:47 UTC 2002

UPS systems are not a cure-all; I have seen the results of what happens
when you don't maintain a large rack of lead acid batteries; and before
that, I for one was fond of calling it the "interruptable power supply".
Since the UPS is a significant expense, some colo locations may make it
an "extra cost" option (and if they don't today, increasing financial
pressure may change this.) Regarding "harder to saturate"--problem there
is that the colo provider might be highly motivated to correct the
situation by unplugging grex.  Some colo locations (perhaps even many)
have the ability to rate limit machines; and may vary network access
charges based on data rate.  Anyways, none of these are show-stoppers or
even necessarily likely; and certainly the air conditioning can't help
but be better.  But certainly power & DOS attack handling are things
we'd want to investigate.  Another thing we'd have to investigate is
access policy; that's been an issue for grex in the past, but I don't
know how the current colo market addresses this.

I think what I would identify as the 2 biggest challenges would be "loss
of space", and "phone lines".  I think it's unlikely we'd want or could
carry our current phone number, modem pool, & terminal server into this
new environment.  Presumably, instead, we'd either buy a share of
someone else's pool, or possibly we would consider dropping dial-ins
entirely.  Right now, we enjoy the "luxury" of having a common space in
which to store spares (to the despair of our treasurer) and to work on
things, including most especially, sitting in a real chair before an
actual desk to get things going after a disaster.  The "spares" storage
would disappear (not clear where).  The chair would be replaced by
either staff members bringing in personal laptops and hooking in through
that, or by staff members actually removing the machine, taking it to
their private houses and fixing it there.  There are advantages to
working in the privacy of one's home, but group collaboration is not
usually one of them.

response 9 of 24:

Aug 3 03:47 UTC 2002

Is that a reference to Dan's login?

response 11 of 24:

Aug 3 12:34 UTC 2002

Most good colo facilities do have generator backup and UPS backup power.  Even
smaller facilities tend to have some kind of generator backup.  My friend runs
a datacenter that is about 8' wide by 20' deep, and he's got a propane
generator outside to run power in the event of a power outage.  

Most good datacenters also have redundant internet connections.. so if one
backbone is down, you're not offline.

I also disagree with Marcus about his dot-com comments.  ISP's are not
generally dot coms (though some of them are).  The ISP industry is
considerably more stable than the "dot com" industry.

I did have an ISP go bankrupt but their assets were sold in bankruptcy court
and the service was never disrupted.

Of course, M-Net gets none of these advantages.  Last I knew, WWNet didn't
have generator backup, nor did they have redundant connections.  

They're still more reliable than a DSL link to a closet.  If WWNet has an
internet problem, they've got people 24/7 working on it who are dedicated to
it because they lose money if they're down.  If they have a power problem..
well..

response 13 of 24:

Aug 3 14:38 UTC 2002

Knowing Jim, Grex has at least five years before they have to think about that
option.

response 15 of 24:

Aug 4 12:05 UTC 2002

How would colocation impact our direct-dial services?

response 17 of 24:

Aug 4 14:19 UTC 2002

Regarding #9; No, my login name is cross because my last name is Cross
(blame the English for what they did to the Scots for that one; what can
I say?), and Global Crossing was a company that went out of business,
but as SCG points out, they had long before sold their colo business.

I don't think looking for a locking 8x10 cube (which wouldn't be cubic,
oddly enough) is realistic, nor is putting grex in Keesan's boyfriend's
basement.  But there's a middle ground that's probably a lot more
reliable than what grex is doing right now, and cheaper, and with better
net access.

Perhaps the bottom line is that more research is waranted, but I think
it's useful to do that research, and not just dismiss the idea because
of preconceived notions.

response 19 of 24:

Aug 5 07:04 UTC 2002

re 11:
        WWNet certainly has redundant connectivity.  They're AS 11386, if you
want to look at a BGP table.  I don't know what the redundant power situation
is there now, but when I worked there (almost 2.5 years ago) there was battery
backup for everything, and generators that had to be cut over to manually.
It wasn't the sort of operation I've since grown acustomed to, but they
certainly weren't helpless in the face of power problems.

Everything Grex is running now would easily fit in a 19" rack.  Whether
storage for large amounts of useless junk and kindling, I mean loose scrap
paper, small amounts of spare parts, and a large collection of empty coke
bottles, is worth the financial and system performance cost appears to be an
open question.  I can say that having done a lot of work on both low and high
budget datacenter environments, I find the conditions in the Pumpkin rather
horrifying.

response 21 of 24:

Aug 7 05:58 UTC 2002

I doubt it -- or at least, not unless we offered them a *lot* more rent.
Probably *their* insurance says that we're supposed to be insured, and
it would likely cost them a bunch more to get insurance to cover claims
we might make then than it would cost us to get the same coverage.

response 23 of 24:

Aug 8 12:07 UTC 2002

A rider is what Mr. Wells suggested yesterday and I'm going
to call the lease agent today and let them know of our problem.
I'll ask about a rider.  Mr. Wells hasn't heard back from his
contacts, as of yesterday afternoon.  He's working on it.

State Farm hasn't called back.  Talking with Barb White would
be a waste of time, I'm afraid.

I assume Aprill is still waiting to hear from Hastings Mutual
and if I haven't heard anything by today, twoish, I'll give
them another call.