Return-Path: <tytso@thunk.org>
Received: from pacific-carrier-annex.mit.edu by po11.mit.edu (8.9.2/4.7) id MAA13775; Sat, 22 Sep 2001 12:50:11 -0400 (EDT)
Received: from snap.thunk.org (SNAP.THUNK.ORG [216.175.175.173])
	by pacific-carrier-annex.mit.edu (8.9.2/8.9.2) with ESMTP id MAA01975;
	Sat, 22 Sep 2001 12:47:58 -0400 (EDT)
Received: from think.thunk.org ([216.175.175.162])
	by snap.thunk.org with asmtp (Exim 3.32 #1 (Debian))
	id 15kpwa-0004KE-00; Sat, 22 Sep 2001 12:47:56 -0400
Received: from tytso by think.thunk.org with local (Exim 3.32 #1 (Debian))
	id 15kpwa-0002AA-00; Sat, 22 Sep 2001 12:47:56 -0400
Date: Sat, 22 Sep 2001 12:47:56 -0400
From: Theodore Tso <tytso@MIT.EDU>
To: "McNerney, Tim" <TMcNerney@chipwrights.com>
Cc: "'John Hawkinson'" <jhawk@mit.edu>, "'sipb@mit.edu'" <sipb@mit.edu>
Subject: Re: Who do I talk to about e19-rtr-backbone.mit.edu lossage?
Message-ID: <20010922124756.E853@thunk.org>
References: <93A4BBB9515ED411AE9D00508BD955A5198C1F@intranet>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.3.15i
In-Reply-To: <93A4BBB9515ED411AE9D00508BD955A5198C1F@intranet>; from TMcNerney@chipwrights.com on Fri, Sep 21, 2001 at 04:06:53PM -0400

On Fri, Sep 21, 2001 at 04:06:53PM -0400, McNerney, Tim wrote:
> 
> The basic problem I am trying to address is that low-bandwidth, 
> interactive TCP sessions (e.g. ssh/telnet, X) are getting 
> particularly screwed when routers drop packets on the floor, 
> which, of course, is the "acceptable" way of dealing with network 
> congestion.  
> 
> It feels to me that the fundamental problem is that while network
> bandwidth is increasing, it is no longer being allocated in a
> way that is "fair" to interactive sessions.  The problem may be
> getting worse as routers get optimized for "bursty" traffic.

Tim,

	I think you're misdiagnosing the problem.  As others have
said, the main problem was caused by worms running on Microsoft boxes
which were sending out a lot of packets to non-existent IP addresses.
The Cisco routers aren't optimized to deal with this type of load, so
large numbers of packets were getting dropped.  In such an
environment, there's not much you can do to help solve the problem.

> Right now the MIT backbone accommodates lots of different types of
> traffic:  big packets, small packets, images, MP3s, streaming video, 
> etc.  The extremes are file transfers that need throughput but can 
> sacrifice latency, and interactive sessions that really need low 
> latency but only constitute a trickle's worth of bandwidth.
> 
> What I had in mind was having two (real or virtual) networks inside
> MIT, one low-bandwidth, low-latency, and one for everything else.
> The point being to make it so that interactive streams don't get 
> "starved" the way they are now.  
> 
> Without doing lots of research, I'm not sure I flesh this out too much 
> more, but I hope you get the idea.  
> 

No, actually it doesn't make any sense.  Remember, a router's job is
to as quickly as possible, take the packet and move it some where
else.  So if it's going to get handled at all, it's going to be moved
as expeditiously as possible to the output interface.  So packets are
*always* going to be handled as with as low a latency as possible.  A
router can't afford to hang onto a packet, simply because it would
take too much memory and processing power.

If you're trying to squeeze packets down a congested, slow link, such
as down a slow 28.8k PPP connection, then it might make sense to
reorder packets based on the port or type-of-service bits in the IP
header, so that telnet packets get sent ahead of bulk ftp packets.

But first of all, it's a lot harder to do this for a huge, fast
router; hanging on to packets so that other packets can go first is
one thing when the output interface is 56kbps, but it's quite another
thing at 100 megabits/second.  Secondly, it wouldn't have helped in
this case because it wasn't the case that the output links where
congested; it was the router's processing power which was overloaded,
and asking it to do any kind of extra work to give certain packets
higher priorities than others would have simply made the router fall
over faster.

Finally, this sort of thing which you're describing requires making
changes to the router's hardware and software, and as such is
something for which SIPB (or even MIT Network Opertions) could do much
about.  If you want to make such changes, you'd have to go work for
Cisco and then try to exaplain to them why your idea is one which
their thousands of network engineers haven't tried already.  :-)

						- Ted
