Dropping the ICNET Link

Oct 1 17:07 UTC 1998

At last week's board meeting the board voted to stop paying for the ICNET
link that used to be Grex's link to the Internet.  STeve Andre wasn't at
the meeting and has asked that we delay implementation of the decision
until after we have discussed the matter for a while longer in coop.

response 2 of 125:

Oct 1 17:49 UTC 1998

With the non-mission-critical nature of grex :-) the risk of the ISDN
connection (literally & figuratively) going away is less important than the
tangible benefits from doing away with the ICNET link, IMO.

response 4 of 125:

Oct 1 18:56 UTC 1998

I don't even think we will get into the difficulties the Ancient Mariner did.

response 6 of 125:

Oct 1 20:31 UTC 1998

I think it was a fine decision to drop the ICNet service.  Not only
will it save us money, but we free up an extra line in the MOTD!   8-)
        Since the Board decided to drop it, this decision should be
carried out immediately.

response 8 of 125:

Oct 1 23:52 UTC 1998

I'm disappointed that the ICNet lines are being dropped before the 
use of them for mail has been tried.  Is there some alternative 
plan for throttling mail's consumption of resources?

response 10 of 125:

Oct 2 16:53 UTC 1998

   I can see I'm just about alone in this.  Herewith is a long
explaination of why I say this.

   I'm going to try to explain why this is not a good idea.  The idea
of doing this, as far as I understand, is to save money.  I know
that some people are worried about the money inflow and have been
looking at all the expenses that we have, trying to trim some costs.
I can accept that.

   As I understand it, cutting two phone lines would trim enough
from our monthly expenses that we'd (hopefully) be at an even state
monitarily.  Given that, there are two possibilities here:

  1) Cut the IC-Net link.  It uses two phone lines.

  2) Cut two dialin lines.

   If we must cut some phones, I favor cutting two dialin lines.

   Why?  Because the IC-Net link is something we need.  Trust me on this.
I'll explain more in a bit.  Part of the attraction in cutting something
like a phone line is that it is a reversable decision--at such point
as one is needed again, it can be turned back on and is once again
running.  That is true with a random dialin line; all we have to do is
scream at Ameritech when they don't do it right the first time, and as
usual they'll get it right and we'll have the line back.

   But this is not the case with the IC-Net link.  The donor who arranged
for the link isn't likely to go back and re-establish it for us.  It isn't
a simple case of just ordering a line and having it come back.  Likely it
will remain gone unless we pay for such a link.

   A dedicated PPP link isn't cheap, either.  You cannot compare the $20
a month cost of someone getting a PPP link for their computer to a dedicated
link.  Such a link is expected to be up nearly all the time, which
a consumer link isn't, and a dedicated link has a "static IP address",
which means that it doesn't change.  The cost of such a link in the real
world, today, is about $125 a month, plus a phone line, so figure $145 a
month.  Right now we're paying $40 (or just under) for the IC-Net link.
This means we're paying approximately one third of the going rate for such
a line at the moment.

   But we have an ISDN link -- why not cut the slower IC-Net link?

Simply stated, there are three reasons why we need to keep it.

   + We have a small Pentium machine running Open BSD unix that we need
to get sendmail on, such that it can handle the stream of mail coming
into Grex.  Now, this is not dependant on the IC-Net link, but just as
the concept of the mail machine is to free Grex from some of the burden
of mail processing, the IC-Net link can free the ISDN link from the
burden of mail traffic.  Right now, the ISDN link is doing pretty well
in terms of saturation--I don't have hard numbers, but the link is able
to handle all our traffic, currently.

   That will not be the case indefinitely.  As the person does 99% of the
account reaping, I keep track of the growth of Grex in ways that likely
no one else does.  If you look at ~steve/GrexStuff/daily_newuser_registrations
you will see the number of newuser runs on Grex for all history.  Grex is
growing at a rate a little slower than I talked about in a board meeting
a couple of months ago, but I project that we will be using all available
UIDs (user ID numbers) that SunOS has (65,536 possible accounts) in about
two years.  As Grex grows all the various uses of it grow as well.  I
cannot tell you when moving mail to a seperate circuit will be needed, but
I will state now that Grex's growth is not stopping, and we're going to
be looking at ways of increasing our connectivity in various ways, because
we're going to have to.

   But can a PPP link keep up with Grex's mail?

A 33K PPP link running on modei that do data compression can be thought
of as a 50K link, if you don't count the latency that a modem creates as
it moves data across a phone line.  This is not as fast as an ISDN link
but is still a major chunk of the ISDN links speed and would be devoted
to mail.  Remember, back on the Sun-3/260 all of Grex (telnet, lynxm mail,
etc) lived on that 28K link.  Yes, we have more mail now then we did back
then but we're talking of using the entire link for this purpose.  Note
that at some point, the PPP link will not be large enough--that day will
come, and at that point we'll have to do something else.  I'm not going
to say that the IC-Net link will be useful to Grex forever, because it
won't--but right now, it is.

   + The second reason to keep the link is that by routing mail traffic
over this, we avoid the problem of mail clogging the ISDN link when Grex
is off the net for several days at a time.  Since we have gotten the ISDN
link, we've been off the net three times now, each for three days, because
of Ameritech problems.  This isn't good.  When Grex got back on the net
after our four day outage in early August, the Sun-4/670 was running flat
out trying to catch all the mail.  It did finally do that (and a LOT faster
than the previous Sun-4/260 did) but there was still a period of several
hours when the system wasn't usable by people, because of this.

   If a real phone snafu hit the Pumpkin we could move the mail machine
to another site for the duration such that Grex could still collect mail.
Inconveinent for the person who'd have to give up their phone line for
this, but possible.

   + A third reason to keep the IC-Net link is for security reasons.  Grex
has not (yet) had a massive Denial of Service attack on it, rendering the
ISDN link useless with garbage traffic on it, but I'm afraid we will have
that at some point.  Several staff people work/live remotely from Ann Arbor
and an Internet link is vital to be able to do anything on the system
remotely.  We have not hit an occaison for this yet, but I am afraid we
will.  An alternate route into Grex in times of emergency would be more
than useful.

   + A forth reason which relates to the second is that in the case of a
major foul up with the ISDN link, Grex could, however horrid it would be,
drop back to the link (with a drastic cut in ptys) and be on the net once
again only tons slower.  This involves coordination with other people, but
could be done.  Could we do this with Dorian directly?  Yes, we could and
that would be the better solution technically, but Grex doesn't move
quickly and having the IC-Net link already running would make it that
much simpler to get Grex back on the net.


    The alternative: cutting two phone lines

   For the time being, this seems to be the clearly better alternative,
to me.  Back several board meetings ago, we were talking about ways to
cut things and Scott (I beleive) mentioned cutting some dialin lines.
My initial reaction was negative, as I was sure that all the dialins
were needed and I was squirming at the idea of cutting access.  Scott
said that he had either never seen them all in use or only very rarely
(I forget which he said then).  Since then I have looked at line usage
some, and I have come to the conclusion that Scott is right--the last
few dialins are not used much.  I am convinced that they have all been
in use at one point (I have a memory of this when working at the Pumpkin
once, seeing all the modei active) but overall, we have an excess of
capacity here.

   Would there be busy signals on Grex if we cut two lines?  Yes, there
could be for a nuber of reasons:

   - More local folks find out about Grex;

   - ISP problems cause local folks to drop back to the dialins while ISP
     problems prevent their use;

   - M-Net folds;

   - General randomness which we might not know about, but could happen;

   - We establish PPP abilities on the terminal server which proves wildly
     popular.

   There are a lot of variables here, and I have no idea what the future
will hold for Grex, besides the fact that usage will only increase.

   But right now, I have no data that indicates that there will be much 
of an impact on users abilities to dial into Grex.  Will it be a zero
effect?  I think not--I think there are going to be some times, most likely
at night, espically around holidays when there will be busy signals.

   I have to look at the two possibilities and weight them.  Most of you
reading this see the IC-Net link as something we don't use, and don't need
any more.  I agree that we haven't been using it, but see the uses of it
as something that Grex really needs to keep.

   Grex is no longer the little system we used to be!  We can't afford to
think about things quite in the manner we used to--meaning, we need to think
about the systems financial health as we always have had to, but we also
need to think about contingencies on things, becuase we're a whole lot
bigger now then we were, even in March of 1994 when we first got on the
net for the public.

   This is one of the pieces of Grex's infrastructure that is kind of like
the system of pipes underneath the street; it isn't noticed by most people,
but they have definite uses. 

We need to hang on to this piece of the infrastructure.  Will we keep it
forever?  No, certainly not.  At some point the technology is going to
change such that we can get access to something else (cable modem system
with appropriate encryption, T1 line, RF system, ADSL, ???) and the idea
of a analog phone PPP link will be be ancient, and useless.

                    Final words

For me it boils down to the following: what can we do, to save money that
is the easiest to recover from, and causes Grex the least problems?  I can
readily understand why there are so many people who thnk that the link can
be cut.  My view of Grex is different from most of you, becuase I'm usually
doing things to the system and watching the system at a different level.

The dialins can be cut easily, and can be re-established easily.  They have
a certain function and thats it.

The IC-Net link can be cutt easily, but not re-estabished easily.  Likely
the cost would be prohibitve (about $145 a month).  Further, it prohibits
us from using a second channel to the world for a variety of things.

Thats the difference, folks.  Thanks for reading this.

response 12 of 125:

Oct 2 19:33 UTC 1998

Scott is right that cutting the ISDN link was not in place of cutting dialins, 
but in addition.

Of course  it would be great to have a backup line.   But my feeling, as
I've said many times, is that all you have to do is look across town  to see
what can happen if we try to maintain a sytem whose expenses are not supported
by its income.  We are not in the kind of trouble M-Net has seen, but I think
we should take steps now to make sure we never get that way.

response 14 of 125:

Oct 3 01:37 UTC 1998

It sounds good in theory, Steve, and I'd certainly be willing to see us try
something else to meet the same need.

Problem is that membership has been static for a very long time, and we
haven't seen any response to any of our attempts to increase it.  (Granted,
our attempts haven't been well organized but part of the problem is that we
have no real idea how to go about convincing the average Grexer to pitch in
without "guilt-tripping" Grexers who really can't (and shouldn't) afford
memberships.

Do *you* have any ideas?  Does anyone?

response 16 of 125:

Oct 3 04:26 UTC 1998

There is no point in keeping the ICNet link.  Therefore, no matter how much
below market value it is, it's not worth anything to us, and is not worth
paying for.

STeve says we need it so staff can get in if the ISDN line is saturated.  He
seems to think we're losing that ability now.  In fact, while we've been
paying $480 to keep this connection up for the last year, that hasn't been
important enough for anybody to configure things so that could be done.  A
few times, when the ISDN line has been down for a few days, I have changed
the default route on gryps, the machine connected to the ICNet link, to send
traffic out the ICNet link rather than out the ISDN link.  The rest of the
time, we've always felt that it was more useful to have Gryps's set up to use
the good connection.

The idea of keeping the link to use at somebody else's home or office to spool
mail if Grex goes off line is also pretty pointless.  The thing to do at that
point would be to take the planned mail spooler machine that doesn't exist
yet somewhere where it can be plugged into an ethernet, rather than wasting
a phone line on it.  Spare ports on an ethernet hub are a lot easier to come
by than spare phone lines that people just have sitting around unused.

It's also pretty much agreed that Grex's ISDN line isn't full yet.  STeve is
talking about needing the really slow dial-up link to send all of Grex's mail
over in the event that our usage gets too big for the ISDN line.  The problem
with this is that the modem link in all likelyhood won't be able to handle
that.  Back when I was doing a lot of work on Grex, around two years ago, Grex
was considerably smaller.  Yet we would see Grex go off line for only an hour
or two, and when we brought it back, allowing only mail and nothing else to
come in over the link, it would often take two or three hours before the link
would unsaturate.  Grex now has more than double the amount of users we had
then, and we still aren't all that close to the point where we need to move
traffic off the ISDN link.  By the time we do get to that point, the idea that
we can move our mail over to this slow dial-up connection and somehow
magically expect it to work is rediculous.

Dial-up modem connections are a mess.  We can't really think of this as just
an extra 33K worth of bandwidth (which isn't much to begin with).  When you
throw in the latency imosed by the modem's attempts to compress the data, and
by the digital to analog conversions, modem connections are *slow*.  STeve's
claim that compression will make it behave like a 50K connection is sort of
true, given ideal conditions and a long download, if you're really lucky. 
In general, though, the compression slows things down a lot more than it
speeds it up.  What we were seeing a couple of years ago watching Grex's mail
system try to deal with this connection is that the mail connections would
start up, run really really slowly, and eventually hopefully be successful.
Meanwhile, the mail software would manage to pretty thoroughly bog down Grex.
With a separate machine handling mail, it's likely that that machine would
get pretty bogged down by the process as well.

The big issue, though, is that if we aren't using it we can't afford to keep
throwing money at it.  The same goes for the phone line or two that never get
used.  Unfortunately, on the phone lines, a very persuasive board member has
started screaming whenever we've talked about cutting them.  This isn't a one
or the other situation.  Grex's purpose is not to throw money at Ameritech.
We have an obsolete Internet connection that we haven't used in more than a
year and aren't likely to use in the future.  Why keep paying for it?  We have
phone lines that are never being used.  Why keep paying for them?  Even if
a few people here think the money that's donated to Grex is something we
should just be throwing away like that, those who are donating the money
probably could find ways to put it to better use.

response 18 of 125:

Oct 3 06:01 UTC 1998

The mail machine project shouldn't be hurt by this.  We can still run it on
the ISDN link, along with Grex itself.

response 20 of 125:

Oct 3 18:14 UTC 1998

I've started item #34 to discuss what we might do to increase membership. If
you have an interest in this topic, please read that item.

response 22 of 125:

Oct 4 03:16 UTC 1998

I never said it was pointless to be able to move the mail machine.  I said
it was unrealistic to just assume that we would move it somewhere where a
spare phone line would magically appear.  It's much more realistic to plan
on moving the mail machine to somewhere where it can be plugged into an
ethernet connection at that point.  Likely it would get significantly better
Net connectivity.  It also wouldn't tie up somebody's phone line that they
presumably got because they wanted to use it.

Yes, all of Grex lived on that link at one point.  Grex was about half the
size it is now.  Grex is growing, and will likely be bigger before we get
around to using that link.  When all of Grex, at around half its current size,
was on that link, we had huge problems.  That's why we replaced it.

The modem link was a huge sink of staff time.  Mostly it was my time.

In theory, latency shouldn't be a big deal for SMTP.  In practice it sometimes
isn't.  If you've got fairly high latency, almost no throughput, and a
connection that drops somewhat frequently, as dial-up connections tend to do,
the combination can be a killer.  Remember the huge drop in Grex's load
average when we switched to the ISDN link?  That corresponded to a vast
reduction in the number of concurrently running sendmail processes.  Sendmail,
on a system with a really large mail volume, does not handle really slow links
well.

response 24 of 125:

Oct 4 06:57 UTC 1998

do we really need to cut $80/mo right this very second?
  
if the email machine were currently set up on the ic.net link, this
conversation would be about cutting  4 dialins for that $80/mo.
  
currently we have a safety valve for everything. drop the ic.net
link and we will have a safety valve for ... nothing. and then the
cost of reestablshing that ic.net link would be extrordinarily out
of reach for both re-setup and per month costs .. 7 dial-up lines' cost
to regain what 2 lines cost us now. 
  
dropping the ic.net link is, imnsho, a short term puff for a long
term hurricane.
  
... adn i hvaen't even said how much i appreciated steve's #10 adn the
inherent value in those arguments.