Return-Path: <jhawk@MIT.EDU>
Received: from grand-central-station.mit.edu by po12.mit.edu (8.9.2/4.7) id OAA23910; Sat, 17 Mar 2001 14:36:38 -0500 (EST)
Received: from manawatu-mail-centre.mit.edu (MANAWATU-MAIL-CENTRE.MIT.EDU [18.7.21.85]) by grand-central-station.mit.edu (8.9.2/8.9.2) with ESMTP id OAA09256; Sat, 17 Mar 2001 14:36:37 -0500 (EST)
Received: from multics.mit.edu (MULTICS.MIT.EDU [18.187.1.73]) by manawatu-mail-centre.mit.edu (8.9.2/8.9.2) with ESMTP id OAA12468; Sat, 17 Mar 2001 14:36:37 -0500 (EST)
Received: (from jhawk@localhost) by multics.mit.edu (8.9.3) id OAA19842; Sat, 17 Mar 2001 14:35:46 -0500 (EST)
Date: Sat, 17 Mar 2001 14:35:46 -0500 (EST)
Message-Id: <200103171935.OAA19842@multics.mit.edu>
To: network@mit.edu
Subject: E19-RTR instability
From: John Hawkinson <jhawk@MIT.EDU>
X-Evolution: 0000011c-0000

Hi.

In the past 10 minutes or so, there's been severe E19-rtr
instability, and possibly other MITnet backbone instability.

I'm not sure how best to characterize it, other than:

nw12-rtr>ping 18.168.0.15

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echoes to 18.168.0.15, timeout is 2 seconds:
.!...
Success rate is 20 percent (1/5), round-trip min/avg/max = 548/548/548 ms

and soforth.

Note that e19-rtr's ospf state is not too good:

vbns-router>sh ip ospf n 

Neighbor ID     Pri   State           Dead Time   Address         Interface
18.252.0.1        1   2WAY/DROTHER    00:00:33    18.168.0.16     Fddi5/1/0
18.168.0.20       1   2WAY/DROTHER    00:00:30    18.168.0.20     Fddi5/1/0
18.185.0.1        1   2WAY/DROTHER    00:00:38    18.168.0.11     Fddi5/1/0
18.168.0.17       1   2WAY/DROTHER    00:00:31    18.168.0.17     Fddi5/1/0
18.201.1.1        1   2WAY/DROTHER    00:00:31    18.168.0.14     Fddi5/1/0
192.233.33.3      1   FULL/BDR        00:00:31    18.168.0.12     Fddi5/1/0
18.248.0.1        2   EXCHANGE/DR     00:00:38    18.168.0.15     Fddi5/1/0

(EXCHANGE means it is on its way up, after a recent state
loss of adjacency).

If the fact that it is unstable is unrelated to the fact
that it is the DR (designated router), then perhaps any
other router should be made the DR, by adjusting the priority.

State changes are incrementing as I watch from
vbns-rtr (which is new enough IOS to keep a count):

vbns-router>sh ip o n 18.248.0.1
 Neighbor 18.248.0.1, interface address 18.168.0.15
    In the area 0.0.0.0 via interface Fddi5/1/0 
    Neighbor priority is 2, State is FULL, 38 state changes
    DR is 18.168.0.15 BDR is 18.168.0.12
    Options is 0x2
    Dead timer due in 00:00:36
    Neighbor is up for 00:00:57
    Index 2/2, retransmission queue length 0, number of retransmission 2623
    First 0x0(0)/0x0(0) Next 0x0(0)/0x0(0)
    Last retransmission scan length is 1, maximum is 40
    Last retransmission scan time is 0 msec, maximum is 4 msec

 Neighbor 18.248.0.1, interface address 18.168.0.15
    In the area 0.0.0.0 via interface Fddi5/1/0 
    Neighbor priority is 2, State is INIT, 39 state changes
    DR is 0.0.0.0 BDR is 0.0.0.0
    Options is 0x2
    Dead timer due in 00:00:38
    Index 0/0, retransmission queue length 0, number of retransmission 2626
    First 0x0(0)/0x0(0) Next 0x0(0)/0x0(0)
    Last retransmission scan length is 3, maximum is 40
    Last retransmission scan time is 0 msec, maximum is 4 msec

Thanks!

--jhawk