response 25 of 49:

May 30 18:38 UTC 2002

Re #17 re #16:
    Are switching speeds keeping up with Moore's Law? How long before they
can keep up with a 600 nm laser beam? Perhaps we need only sit tight and wait
a few years?

response 27 of 49:

Jun 2 06:47 UTC 2002

Here in chicagoland the bottlenecks seem to be the NAPs - points
where networks connect.  The MCI one (or whomever they call
themselves) is notorious and I think is still subject to litigation.
The arguement between Ameritech and MCI was who should pay for the
access.  MCI didn't feel that it should pay to allow its competition
high bandwidth connectivity to its network and of course Ameritech
felt the same. Alter.net seems to be another one although that seems
to be more a reliability issue rather than bandwidth.

The big problem that a lot of ISPs are having is not that they
don't have the bandwith to support heavy users - they do.  The
reason they want to start charging heavy users is because their
current subscription revenue from existing base is not profitable.
Many ISPs over built and over bought and now with the economic
downturn and stock performance in the pits, credit lines drying
up, and the inability to float bonds, and other factors the 
turkeys are coming home to roost.  They need to fund operations
from existing users and know that they can't raise the rates 
across the board without their competition doing the same.  Perhaps
they are also counting on some sort of cache of being a 'heavy
user' causing said to not object to the raise in cost.  Another
source of revenue that many ISPs are going to go after is that
of 'servers'.  Many new subscriber agreements already prohibit
the operation of 'servers' especially those with static IP.
I'd expect to see more 'charges' associated with allowing them
croping up in the future. 

To sum up, its not a bandwith issue but a revenue issue.

response 29 of 49:

Jun 3 08:12 UTC 2002

re#28: Uh. Huh? Lets say yer town all have fiber or whatever.  You
all can connect to each other at warp speed.  But lets say the town
next to you has only a 64K link.  Yer effective bandwidth to any user
from yer town to that town is 64K divided by the simultaneous number
of users minus the overhead.  Who pays for upgrading the link?
You or them?  Lets say they have the hot porno site everyone in
your town wants to visit and all your town has to offer is town
meetings plus the crop report.  Why should they pay for the higher
speed link to allow your town to visit their site?  Because it allows
their users to visit your crop report? 50-50? They should pay half
so your users can visit their hot porno site while they get nothing
in return?

response 31 of 49:

Jun 4 06:54 UTC 2002

Of course if they pay for an upgrade, then they can visit their own hot 
porno site much more expeditiously, not to mention being better able to 
make money selling site access to other people in other towns who might 
otherwise go to other hot porno sites with faster links.

response 33 of 49:

Jun 4 08:07 UTC 2002

NAPs and taking naps and why the Internet as we know it is
doomed.

response 35 of 49:

Jun 5 04:12 UTC 2002

What happens when one of the links goes away, Russ?  How does that information
get transmitted to all of the endpoints that need to know?

response 37 of 49:

Jun 5 07:19 UTC 2002

Source routing went out with uucp.  With IP, the end nodes are not
responsible for deciding how packets get from point A to B; that's the
responsibility of the routers inbetween.  Modern routers are designed to
handle failing nodes (given a sufficient # of machines, there is
virtually a 100% chance some node is out of order), and some smarter
machines may also do load sharing.

Besides the probability of machine failure, there is also the "back hoe"
issue, which is actually more serious than it seems.  There are a
surprising number of points in our transcontinental communications
system where there are "choke points"; only one way to get from point A
to B.  There may be 3 dozen telecommunications companies offering
service from points near A to B, so it may seem like there are lots of
choices - but it generally turns out they mostly all bought or rent
cable rights that all go via one mountain pass or over one bridge --
very often that turns out to be some railroad right of way first laid
out for steam over a century ago, somewhere out west.

In order to handle the case that any given machine may be out of
service, most modern routing algorithms are designed to be
"distributed", with each machine generally only computing only the "next
hop".  One of the proofs for correctness of any given routing algorithm
is that it not route packets in a circle.  Another requirement is that
when overloaded, routing algorithms should "fail gracefully" (which is
harder than it seems, and imperfect at best.)

For routers on the periphery of the internet, these are all generally
easy problems; typically there are a few local attached networks, one or
a small number of other routers with local networks, and some single
"upstream" point where everything else goes.  It may only take a couple
of K of data to represent all this, and making computing "next hop" off
this data is generally fast.  For such machines, exchanging the whole
route table periodically is often sufficient, & handling network outages
is usually simply a matter of noticing some machine hasn't been heard
from in a while, and deleting it from the very small routing table.

For "backbone" routers, the routing problem is a nightmare, and modern
backbone routers probably start with having >64M of ram just to store
all the networks it has to know about.  One of the important
requirements of backbone routing protocols is to reduce the amount of
routing traffic as much as possibly simply to avoid shipping 64M of data
every time some remote network interface flaps (goes up or down).  This
is all a very big problem; there are numerous dissertations on the
design problem, and dozens of discarded routing protocols that seemed
like a good idea at the time, then turned out to have expensive
problems.

All of this is complicated to solve, and people who are truely competent
at it are scarce.  That is why network engineers can pull down some big
bucks, and sometimes have some truely strange educational backgrounds.
This is also why there are really only a few companies that make good
"backbone" routers - those are typically very specialized machines with
some terribly expensive and fast hardware in them.

response 39 of 49:

Jun 5 08:19 UTC 2002

Yes, and marcus is well underpaid.

response 41 of 49:

Jun 6 01:42 UTC 2002

Re #35:  What happens now?  New route information gets sent around
the net eventually, and until it does packets using the wrong route
get dropped on the floor.  The difference is that routing would only
have to be done once per connection (or set of connections between
two particular nodes), not once per packet.  This relieves the burden
on the routers because most of the data-handling requires no smarts.

If you wanted to use some smarts, you could always use the header-edit
system to re-route packets bound for one port to an alternate route
via some different set of hops.  You'd only have to do this once per
change in the routing table.

Re #37:  It may have gone out, but there's a really good argument
for bringing it back; the expense of the backbone might make it a good
idea to push more of the routing functions nearer to the edges.

response 43 of 49:

Jun 6 12:56 UTC 2002

Actually, the Internet today is heavily loaded with traffic and so a broken
node tends to result in massive loss of data.

response 45 of 49:

Jun 7 06:12 UTC 2002

At the very least, you're talking about a complete protocol redesign,
requiring software updates or replacement of every computer or other device
connected to the Internet.  Even if it were a good idea, it wouldn't be likely
to happen.

The path from any given destination to any other given destination tends to
be reasonably static in most cases.  Still, even in the current setup,
somebody reconfiguring their transit connections, or having a link to one of
their transit providers failing, will be seen by BGP speaking routers all over
the Net.  Networks releasing enough bad routing information have caused some
fairly significant Internet instability several times.  Since you're talking
about every edge device needing to know considerably more routing information
than today's core backbone routers (which only need to know the next hop, path
length, and a few other bits of information for making routing decisions),
and since most edge devices are optimized for something other than routing,
you're talking about a pretty significant job for the edge devices.

The number of routes in the table would have to increase dramatically as well.
Currently, the Internet is devided up into several thousand "autonomous
systems," each of which generally announce big aggregate blocks of IP address
space.  For example, an AS might have lots of customers with /29s (blocks of
8 IP addresses, but would announce them all to the world as a single /18
(16,384 IP addresses).  The rest of the world would only know to send anything
within that /18 to the AS making the announcement; the AS's internal routing
protocol, which doesn't share its data with the rest of the world, would take
it from there and get the data to the right place.  If you want all the
routing information to be known to all the edge devices, the edge devices all
over the world would have to know not only what's in the current global
routing tables, but the internal routing information of every network in the
world.  The overhead would be extremely significant.

response 47 of 49:

Jun 13 23:20 UTC 2002

Getting back to the subject of heavy usage, a bunch of engineering people from
various cable modem and DSL ISPs have been saying that a majority of the
traffic they're now seeing is Kazaa, the Napsteresque file sharing program.

If you're provisioning a network for end users, you can generally assume that
the users will sit there sending tiny amounts of text in telnet sessions, or
downloading files or web pages every few minutes.  You can further assume that
most of your users will have other things to do with some of their time, and
won't do even that constantly.  Kazaa, on the other hand, not only grabs files
for download, but serves the files it already has to other Kazaa users.  If
left running, it has a tendency to suck up lots of bandwidth, 24 hours a day.
I assume it's the Kazaa users, not just people who need to read mail and so
forth while working at home, who would be targeted by a heavy usage surcharge.
It would certainly make sense to do so.

response 49 of 49:

Jun 18 01:31 UTC 2002

Such blackholes are more likely to be the result of peering disputes than
small subnets.  What needs to be done, in terms of aggregation, to get a route
past the various filters and routed by everyone is pretty well documented.