Sunday, February 28, 2010

BGPFLAP: a simple tcl script to flap an interface

Hi all,
today I was trying the bgp dampening feature, and I wrote this simple tclsh script to flap an interface during my lab tests:

first, configure an interface with the route to flap:

R3#sh run int lo 10 | beg int
interface Loopback10
ip address 10.10.10.10 255.255.255.0
end

then configure the network statement for that route under the bgp process:

R3#sh run | sec router bgp
router bgp 300
no synchronization
bgp log-neighbor-changes
bgp dampening
network 10.10.10.0 mask 255.255.255.0
neighbor 192.168.2.1 remote-as 200
neighbor 192.168.2.1 ebgp-multihop 2
neighbor 192.168.2.1 update-source Loopback0
no auto-summary
R3#


and here is the tclsh script, I reccomend a wait time of 40-50 sec with the default bgp timers


##################################################################################
## Tclsh BGPFLAP SCRIPT v0.1 Beta: a tclsh script to flap a bgp route
## By Marco Rizzi ( http://rizzitech.blogspot.com ) marco.rizzi.com[A_T]gmail.com
## Date Feb 28, 2010
## licensed under a Creative Commons Attribution 3.0 United States License
## ( http://creativecommons.org/licenses/by/3.0/us/ ) ;-)
##################################################################################
### USAGE: BGPFLAP [interface_to_flap] [number_of_flaps] [wait_secs]

## DON'T USE ON PRODUCTION SYSTEMS, IT'S ONLY FOR LAB TESTING
## No warranty, provided "AS IS"

## Main procedure
proc BGPFLAP {interface n_flaps wait} {

## Before you start, be shure you have a bgp process up and running
## and the interface you will flap declared under the bgp process
## BGP DAMPENING would be nice to have on neighbors too :-)

## calculates msecs to wait: 1 flap = 1 down + 1 up so wait/2
set wait [ expr {$wait/2 * 1000}]

## produces n flaps of the desired route shutting down the interface at wait/2 rate
for {set i 0} {$i <= $n_flaps} {incr i} {

ios_config "interface $interface" "shutdown"

#wait, let bgp withdraw the route
after $wait

ios_config "interface $interface" "no shutdown"

#wait, let bgp announce the route
after $wait
}
}


################################# END OF SCRIPT ###################################
##
##
### USAGE: BGPFLAP [interface_to_flap] [number_of_flaps] [wait_secs]
#### enjoy ;-)


Let's try with

R3(tcl)#
R3(tcl)#BGPFLAP loopback10 6 45

*Feb 28 16:08:43.443: %LINK-5-CHANGED: Interface Loopback10, changed state to administratively down
*Feb 28 16:08:44.447: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback10, changed state to down
*Feb 28 16:09:05.467: %LINK-3-UPDOWN: Interface Loopback10, changed state to up
*Feb 28 16:09:06.467: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback10, changed state to up
!-- ...and so on


on the other side you can see:

R2#debug ip bgp dampening
BGP dampening debugging is on for address family: IPv4 Unicast
R2#
*Feb 28 16:08:41.639: EvD: charge penalty 1000, new accum. penalty 1000, flap count 1
*Feb 28 16:08:41.643: BGP(0): charge penalty for 10.10.10.0/24 path 300 with halflife-time 15 reuse/suppress 750/2000
*Feb 28 16:08:41.647: BGP(0): flapped 1 times since 00:00:00. New penalty is 1000
R2#
*Feb 28 16:09:12.671: EvD: accum. penalty 977, not suppressed
R2#
*Feb 28 16:09:21.083: EvD: accum. penalty decayed to 973 after 8 second(s)
R2#
*Feb 28 16:09:42.759: EvD: accum. penalty decayed to 958 after 22 second(s)
*Feb 28 16:09:42.763: EvD: charge penalty 1000, new accum. penalty 1958, flap count 2
*Feb 28 16:09:42.767: BGP(0): charge penalty for 10.10.10.0/24 path 300 with halflife-time 15 reuse/suppress 750/2000
*Feb 28 16:09:42.767: BGP(0): flapped 2 times since 00:01:01. New penalty is 1958
R2#

R2#sh ip bgp 10.10.10.0
BGP routing table entry for 10.10.10.0/24, version 24
Paths: (1 available, no best path)
Flag: 0x820
Not advertised to any peer
300, (suppressed due to dampening)
192.168.3.1 (metric 65) from 192.168.3.1 (192.168.3.1)
Origin IGP, metric 0, localpref 100, valid, external
Dampinfo: penalty 3521, flapped 4 times in 00:05:19, reuse in 00:00:43
R2#

As you can see, 45 secs of flapping time is barely enough to let bgp announce/withdraw the route, here R2 has noticed only 4 flaps, with a penality of 3521 points...


hope this helps someone to play with bgp :-)

Marco

Friday, February 26, 2010

eBGP peering with Loopbacks: AD problems

Hi all,
while I'm 3 days at home ill, I've started reading the great "Routing TCP/IP Vol.II" from Cisco Press, and playing a little with bgp to refresh my knowledge on that.

During my tests I've encountered some small interesting and funny things:

eg: eBGP issues peering with Loopbacks and announcing loopback on bgp itself

The topology is really simple, I used the same of my old rip lab:








Here the .net file I used on Dynagen and the initial configs:






############## bgp.net file ###############
[10.3.3.2:7200]
udp = 10000
workingdir = /tmp
[[7200]]
image = /opt/c7200-adventerprisek9-mz.124-11.T.bin
npe = npe-400
ram = 160
[[ROUTER R1]]
console = 20001
s1/0 = R2 s1/0
[[ROUTER R2]]
console = 20002
s1/1 = R3 s1/1

[10.3.3.2:7201]
udp = 15000
workingdir = /tmp
[[7200]]
image = /opt/c7200-adventerprisek9-mz.124-11.T.bin
npe = npe-400
ram = 160
[[ROUTER R3]]
console = 20003
################## end bgp.net file ######################


!--- R1 initial config

hostname R1

no ip domain-look
line con 0
logging sync
no exec-tim

int lo 0
ip address 192.168.1.1 255.255.255.0
ip ospf 1 area 0
ip ospf network point-to-point

int ser 1/0
desc R1 - R2
ip address 10.12.12.1 255.255.255.192
no shut
ip ospf 1 area 0

router ospf 1
router-id 192.168.1.1

!--- END R1 initial config

!--- R2 initial config
hostname R2

no ip domain-look
line con 0
logging sync
no exec-tim

int lo 0
ip address 192.168.2.1 255.255.255.0
ip ospf 1 area 0
ip ospf network point-to-point

int ser 1/0
desc R1 - R2
ip address 10.12.12.2 255.255.255.192
ip ospf 1 area 0
no shut

int ser 1/1
desc R2 - R3
ip address 10.23.23.2 255.255.255.192
ip ospf 1 area 0
no shut

router ospf 1
router-id 192.168.2.1

!--- END R2 initial config

!--- R3 initial config
hostname R3

no ip domain-look
line con 0
logging sync
no exec-tim

int lo 0
ip address 192.168.3.1 255.255.255.0
ip ospf 1 area 0
ip ospf network point-to-point

int ser 1/1
desc R3 - R2
ip address 10.23.23.3 255.255.255.192
ip ospf 1 area 0
no shut

router ospf 1
router-id 192.168.3.1

!--- END R3 initial config



Well, now let's start peering eBGP between loopbacks 0 and announcing loopbacks into bgp:


!-- ON R1
router bgp 100
no auto
no sync
nei 192.168.2.1 remote 200
nei 192.168.2.1 update lo0
nei 192.168.2.1 ebgp 2
network 192.168.1.0

!-- ON R2
router bgp 200
no auto
no sync
nei 192.168.1.1 remote 100
nei 192.168.1.1 update lo0
nei 192.168.1.1 ebgp 2
nei 192.168.3.1 remote 300
nei 192.168.3.1 update lo0
nei 192.168.3.1 ebgp 2
network 192.168.2.0

!-- ON R3
router bgp 300
no auto
no sync
nei 192.168.2.1 remote 200
nei 192.168.2.1 update lo0
nei 192.168.2.1 ebgp 2
network 192.168.3.0



All seems to work fine, bgp peering slowly comes up, they're exchanging prefixes....

R2(config-router)#do sh ip bgp sum
BGP router identifier 192.168.2.1, local AS number 200
BGP table version is 4, main routing table version 4
3 network entries using 360 bytes of memory
3 path entries using 156 bytes of memory
4/3 BGP path/bestpath attribute entries using 496 bytes of memory
2 BGP AS-PATH entries using 48 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
Bitfield cache entries: current 1 (at peak 1) using 32 bytes of memory
BGP using 1092 total bytes of memory
BGP activity 3/0 prefixes, 3/0 paths, scan interval 60 secs

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
192.168.1.1 4 100 4 7 4 0 0 00:01:53 1
192.168.3.1 4 300 4 7 4 0 0 00:01:39 1

R2(config-router)#do sh ip bgp
BGP table version is 4, local router ID is 192.168.2.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

Network Next Hop Metric LocPrf Weight Path
*> 192.168.1.0 192.168.1.1 0 0 100 i
*> 192.168.2.0 0.0.0.0 0 32768 i
*> 192.168.3.0 192.168.3.1 0 0 300 i


(Scott Morris will say something like "life is pretty good" at this point :-) )

But.. wait.. network 192.168.1.0 next hop 192.168.1.1 ?? that sounds like a bad recursion!

Let me check on the routing table of R2:

R2(config-router)#do sh ip route | inc 192.168.1.0
B 192.168.1.0/24 [20/0] via 192.168.1.1, 00:02:58

R2(config-router)#do sh ip route 192.168.1.0
Routing entry for 192.168.1.0/24
Known via "bgp 200", distance 20, metric 0
Tag 100, type external
Last update from 192.168.1.1 00:00:09 ago
Routing Descriptor Blocks:
* 192.168.1.1, from 192.168.1.1, 00:00:09 ago
Route metric is 0, traffic share count is 1
AS Hops 1
Route tag 100

What's happened? R2 has learned from R1 the 192.168.1.0/24 prefix and it has installed into the rib because the eBGP administrative distance = 20. The next hop for this prefix is the eBGP peer address, the loopback0, so the routing table entry is correct, but that creates the next hop unreachable, a sort of rib l00p.
If we try, we can't ping the R1 lo0 from R2 after the eBGP prefix is installed:

R2(config-router)#do sh ip route 192.168.1.0
Routing entry for 192.168.1.0/24
Known via "bgp 200", distance 20, metric 0
Tag 100, type external
Last update from 192.168.1.1 00:00:06 ago
Routing Descriptor Blocks:
* 192.168.1.1, from 192.168.1.1, 00:00:06 ago
Route metric is 0, traffic share count is 1
AS Hops 1
Route tag 100

R2(config-router)#
R2(config-router)#do ping 192.168.1.1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.1.1, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)
R2(config-router)#

After a little while (BGP has "slow" timers by default) we see the eBGP peers going down, then up (they removes the eBGP rib entries and becomes reachable by ospf again..), then down again...

*Feb 26 12:20:25.651: %BGP-5-ADJCHANGE: neighbor 192.168.1.1 Down BGP Notification sent
*Feb 26 12:20:25.651: %BGP-3-NOTIFICATION: sent to neighbor 192.168.1.1 4/0 (hold time expired) 0 bytes
*Feb 26 12:20:57.667: %BGP-5-ADJCHANGE: neighbor 192.168.1.1 Up


How to solve this issue?
-don't create the network statement for the loopbacks used for peering, let igp do his job
OR
-announce loopbacks under eBGP but create static routes (lower AD wins..)
OR
-peer the eBGP neighbors with the directly connected interfaces (as every manual suggest :-) )
OR
-modify the default administrative distance for eBGP routes
OR
-use BGP Backdoor for the neighbor networks

Here the example using bgp backdoor from R3 perspective:
R3(config)#do sh ip route 192.168.2.0
Routing entry for 192.168.2.0/24
Known via "bgp 300", distance 20, metric 0
Tag 200, type external
Last update from 192.168.2.1 00:00:07 ago
Routing Descriptor Blocks:
* 192.168.2.1, from 192.168.2.1, 00:00:07 ago
Route metric is 0, traffic share count is 1
AS Hops 1
Route tag 200

R3(config)#router bgp 300
R3(config-router)#network 192.168.2.0 backdoor
R3(config-router)#do sh ip route 192.168.2.0
Routing entry for 192.168.2.0/24
Known via "ospf 1", distance 110, metric 65, type intra area
Last update from 10.23.23.2 on Serial1/1, 00:00:05 ago
Routing Descriptor Blocks:
* 10.23.23.2, from 192.168.2.1, 00:00:05 ago, via Serial1/1
Route metric is 65, traffic share count is 1

R3(config-router)#do sh ip route | beg Gate
Gateway of last resort is not set

10.0.0.0/26 is subnetted, 2 subnets
C 10.23.23.0 is directly connected, Serial1/1
O 10.12.12.0 [110/128] via 10.23.23.2, 00:23:30, Serial1/1
B 192.168.1.0/24 [20/0] via 192.168.2.1, 00:18:39
O 192.168.2.0/24 [110/65] via 10.23.23.2, 00:06:35, Serial1/1
C 192.168.3.0/24 is directly connected, Loopback0
R3(config-router)#

Note that R1 loopback still learned from bgp by R3.

have phun!
Marco

Friday, February 19, 2010

System Management: do and don't

Well, I'm working on system management this we, I've suspended blogging about my ospf labs (no time.. sorry), anyway, here are my suggestions about system management:

1) DON'T start the system management labs without reading the "Cisco IOS Configuration Fundamentals Configuration Guide, Release 12.4T" it's full of tricky small commands, easy to forget...

2) DON'T try terminal padding (at least not with my insane testing way ;-))

R4#terminal padding . 255

R4#show terminal
Line 0, Location: "Bl4ckh0l3R4ck", Type: ""
Length: 24 lines, Width: 80 columns
Baud rate (TX/RX) is 9600/9600, no parity, 2 stopbits, 8 databits
Status: PSI Enabled, Ready, Active, Automore On
Capabilities: Notification Set
Modem state: Ready
Group codes: 0
Modem hardware state: CTS* noDSR DTR RTS
Special Chars: Escape Hold Stop Start Disconnect Activation
^^x none - - none
Timeouts: Idle EXEC Idle Session Modem Answer Session Dispatch
never never none not set
Idle Session Disconnect Warning
never
Login-sequence User Response
00:00:30
Autoselect Initial Wait
not set
Modem type is unknown.
Session limit is not set.
Time since activation: 2d06h
Editing is enabled.
History is enabled, history size is 20.
DNS resolution in show commands is enabled
Full user help is disabled
Allowed input transports are none.
Allowed output transports are lat pad telnet rlogin lapb-ta mop v120 ssh.
Preferred transport is lat.
Characters padded with NUL bytes:
Char ASCII count
. 46 255
No special data dispatching characters
R4#

a show run tooks 1 minute, every dot is displayed really slow ... bad idea :-)
"terminal default padding" from privileged mode to remove it

3) DO! setting the escape caracter of a session or of the whole line, useful for terminal servers with asynchronus serials, that doesn't support CTRL+SHIFT+6 ...
(Ascii chars reference table: http://www.cisco.com/en/US/docs/ios/fundamentals/command/reference/cf_ap1.html)

R4#terminal escape-character ?
BREAK Cause escape on BREAK
CHAR or <0-255> Escape character or its ASCII decimal equivalent
DEFAULT Use default escape character
NONE Disable escape entirely
soft Set the soft escape character for this line

R4#terminal escape-character 27 !-- see on ascii table above: 27 = ESC key
"^[" is the escape character
R4#show terminal
Line 0, Location: "Bl4ckh0l3R4ck", Type: ""
Length: 24 lines, Width: 80 columns
Baud rate (TX/RX) is 9600/9600, no parity, 2 stopbits, 8 databits
Status: PSI Enabled, Ready, Active, Automore On
Capabilities: Notification Set
Modem state: Ready
Group codes: 0
Modem hardware state: CTS* noDSR DTR RTS
Special Chars: Escape Hold Stop Start Disconnect Activation
^[x none - - none

Timeouts: Idle EXEC Idle Session Modem Answer Session Dispatch
never never none not set
Idle Session Disconnect Warning
never
Login-sequence User Response
00:00:30
Autoselect Initial Wait
not set
Modem type is unknown.
Session limit is not set.
Time since activation: 2d07h
Editing is enabled.
History is enabled, history size is 20.
DNS resolution in show commands is enabled
Full user help is disabled
Allowed input transports are none.
Allowed output transports are lat pad telnet rlogin lapb-ta mop v120 ssh.
Preferred transport is lat.
No output characters are padded
No special data dispatching characters

R4#ping 10.1.1.1 repeat 100000 !--- after a little while, I hitted the ESC key and...

Type escape sequence to abort.
Sending 100000, 100-byte ICMP Echos to 10.1.1.1, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)
R4#

Nice experiment this one, at least for my lab!


4) DON'T use cpu intensive commands when you have configured snmp traps cpu rising ....
eg:
R4#sh snmp mib | inc tcp
tcp.1
tcp.2
tcp.3
tcp.4
tcp.5
tcp.6
tcp.7
tcp.8
tcp.9
tcp.10
tcp.11
tcp.12
tcpConnEntry.1
tcpConnEntry.2
tcpConnEntry.3
tcpConnEntry.4
tcpConnEntry.5
tcp.14
tcp.15

000192: *Feb 19 2010 23:53:38.682 GMT+1: %SYS-1-CPURISINGTHRESHOLD: Threshold:
Total CPU Utilization(Total/Intr): 99%/0%, Top 3 processes(Pid/Util): 3/99%, 2/0%, 97/0%
000193: *Feb 19 2010 23:54:53.646 GMT+1: %SYS-1-CPUFALLINGTHRESHOLD: Threshold:
Total CPU Utilization(Total/Intr) 1%/0%.

Niiice to stall yourself for a minute! :-) Not a snmp related issue, but nice to see your cpu high logged on console.


[[ Well, I will add more system menagement do and don't during this we.. stay tuned! ]]

Thursday, February 11, 2010

OSPF on Frame Relay

Hi all,
last week I've studied eigrp, now it's the turn of OSPF...

the first issue for me was using OSPF on frame-relay hub and spoke, I really have to keep in mind the various combinations of network ospf types and/or frame relay interface types.

So I hope this post will help anyone with my same problems.

The topology here is really a classic hub and spoke frame relay:


here the .net dynamips file:

autostart = False
[10.3.3.2:7200]
workingdir = /tmp
udp = 10000
[[3725]]
image = /opt/IOS/c3725-adventerprisek9-mz.124-15.T10.bin
ram = 128
ghostios = True
sparsemem = True
[[FRSW FR0]]
1:102 = 2:201
1:103 = 3:301
2:201 = 1:102
3:301 = 1:103
[[ROUTER R1]]
model = 3725
console = 20001
s0/0 = FR0 1
slot1 = NM-4T
[[ROUTER R2]]
model = 3725
console = 20002
s0/0 = FR0 2
[[ROUTER R3]]
model = 3725
console = 20003
s0/0 = FR0 3


Let's start with a simple initial config, using frame relay on physical interfaces with static dlci mapping:


!--- R1 initial config
ena
conf t

host R1

no ip domain-look
line con 0
loggin sync
no exec-time

int lo 0
ip address 1.1.1.1 255.255.255.255
ip ospf 1 area 0

int ser 0/0
ip address 10.0.0.1 255.255.255.0
encap frame
no frame inver
frame map ip 10.0.0.2 102 broad
frame map ip 10.0.0.3 103 broad
no shut
ip ospf 1 area 0
ip ospf priority 255

router ospf 1
router-id 1.1.1.1

!--- R2 initial config
ena
conf t

host R2

no ip domain-look
line con 0
loggin sync
no exec-time

int lo 0
ip address 2.2.2.2 255.255.255.255
ip ospf 1 area 0

int ser 0/0
ip address 10.0.0.2 255.255.255.0
encap frame
no frame inver
frame map ip 10.0.0.1 201 broad
no shut
ip ospf 1 area 0
ip ospf priority 0

router ospf 1
router-id 2.2.2.2

!--- R3 initial config
ena
conf t

host R3

no ip domain-look
line con 0
loggin sync
no exec-time

int lo 0
ip address 3.3.3.3 255.255.255.255
ip ospf 1 area 0

int ser 0/0
ip address 10.0.0.3 255.255.255.0
encap frame
no frame inver
frame map ip 10.0.0.1 301 broad
no shut
ip ospf 1 area 0
ip ospf priority 0

router ospf 1
router-id 3.3.3.3


Some observation about the initial config:
-the router-id configuration is really optional here, since there's only one loopback
-it's really important in a hub and spoke ospf scenario to have the HUB acting as DR, mainly because only the hub has complete rechability of all spokes. It's better if the spokes are DROTHER (not DB nor BDR), just in case of failure of the DR, no spoke will be the new DR. That's the reason of the "ip ospf priority" commands in the initial config.

With this initial config, we can note that no neighbor are present on our hub

R1#sh ip ospf interface brief
Interface PID Area IP Address/Mask Cost State Nbrs F/C
Se0/0 1 0 10.0.0.1/24 64 DR 0/0
Lo0 1 0 1.1.1.1/32 1 LOOP 0/0

R1#sh ip ospf neighbor

R1#


but why? we have L3 reachability, we can ping from hub to both spokes, and we have mapped the ip to dlci using the "broadcast", that means multicast is enabled on our frame relay interfaces too.
So why we didn't see any neighbor?
If we look at the serial interface in more depth, we can see that ospf assigns to frame realy interfaces the "NON_BROADCAST" type by default:

R1#sh ip ospf interface ser 0/0
Serial0/0 is up, line protocol is up
Internet Address 10.0.0.1/24, Area 0
Process ID 1, Router ID 1.1.1.1, Network Type NON_BROADCAST, Cost: 64
Enabled by interface config, including secondary ip addresses
Transmit Delay is 1 sec, State DR, Priority 255
Designated Router (ID) 1.1.1.1, Interface address 10.0.0.1
No backup designated router on this network
Timer intervals configured, Hello 30, Dead 120, Wait 120, Retransmit 5
oob-resync timeout 120
Hello due in 00:00:24
Supports Link-local Signaling (LLS)
Cisco NSF helper support enabled
IETF NSF helper support enabled
Index 2/2, flood queue length 0
Next 0x0(0)/0x0(0)
Last flood scan length is 0, maximum is 0
Last flood scan time is 0 msec, maximum is 0 msec
Neighbor Count is 0, Adjacent neighbor count is 0
Suppress hello for 0 neighbor(s)


So, even broadcast/multicast is enabled on the frame relay interfaces, ospf still thinking that they are NON_BROADCAST, and isn't sending multicasst hellos.
A simple way to solve this, without changing the default network type, is to configure ospf at least on one side with the neighbor command.
With the neighbor manually specified, ospf sends unicast hellos, and the other side will also respond unicast, so let's try at the hub side:


R1#conf t
Enter configuration commands, one per line. End with CNTL/Z.
R1(config)#router ospf 1
R1(config-router)#neighbor 10.0.0.2
R1(config-router)#neighbor 10.0.0.3
R1(config-router)#end
R1#sh ip
*Mar 1 00:01:20.119: %SYS-5-CONFIG_I: Configured from console by console

R1#sh ip ospf neighbor

Neighbor ID Pri State Dead Time Address Interface
N/A 0 ATTEMPT/DROTHER 00:01:51 10.0.0.2 Serial0/0
N/A 0 ATTEMPT/DROTHER 00:01:55 10.0.0.3 Serial0/0
R1#

After the neighbor command is entered, the neighbor is shown in state "ATTEMPT", but the process have to wait the interface "wait timer" before start the DR election. This wait timer is used to "listen" to hellos and try to determine if a DR/BDR already exists before starting a new election. It's equal to the dead timer.

Then it sends the unicast hello and forms adjacency:

R1#sh ip ospf int ser 0/0
Serial0/0 is up, line protocol is up
Internet Address 10.0.0.1/24, Area 0
Process ID 1, Router ID 1.1.1.1, Network Type NON_BROADCAST, Cost: 64
Enabled by interface config, including secondary ip addresses
Transmit Delay is 1 sec, State WAITING, Priority 255
No designated router on this network
No backup designated router on this network
Timer intervals configured, Hello 30, Dead 120, Wait 120, Retransmit 5
oob-resync timeout 120
Hello due in 00:00:00
Wait time before Designated router selection 00:00:04
Supports Link-local Signaling (LLS)
Cisco NSF helper support enabled
IETF NSF helper support enabled
Index 2/2, flood queue length 0
Next 0x0(0)/0x0(0)
Last flood scan length is 0, maximum is 0
Last flood scan time is 0 msec, maximum is 0 msec
Neighbor Count is 0, Adjacent neighbor count is 0
Suppress hello for 0 neighbor(s)

*Mar 1 00:03:26.943: %OSPF-5-ADJCHG: Process 1, Nbr 3.3.3.3 on Serial0/0 from LOADING to FULL, Loading Done
*Mar 1 00:03:26.963: %OSPF-5-ADJCHG: Process 1, Nbr 2.2.2.2 on Serial0/0 from LOADING to FULL, Loading Done

R1#sh ip ospf int ser 0/0
Serial0/0 is up, line protocol is up
Internet Address 10.0.0.1/24, Area 0
Process ID 1, Router ID 1.1.1.1, Network Type NON_BROADCAST, Cost: 64
Enabled by interface config, including secondary ip addresses
Transmit Delay is 1 sec, State DR, Priority 255
Designated Router (ID) 1.1.1.1, Interface address 10.0.0.1
No backup designated router on this network
Timer intervals configured, Hello 30, Dead 120, Wait 120, Retransmit 5
oob-resync timeout 120
Hello due in 00:00:11
Supports Link-local Signaling (LLS)
Cisco NSF helper support enabled
IETF NSF helper support enabled
Index 2/2, flood queue length 0
Next 0x0(0)/0x0(0)
Last flood scan length is 0, maximum is 0
Last flood scan time is 0 msec, maximum is 0 msec
Neighbor Count is 0, Adjacent neighbor count is 0
Suppress hello for 0 neighbor(s)
R1#


ok, now we have the routes correctly learned, but really isn't enough in a frame relay environment, we have to ensure to have a correct L3 to L2 mapping, eg:

R2#sh ip route | beg Gate
Gateway of last resort is not set

1.0.0.0/32 is subnetted, 1 subnets
O 1.1.1.1 [110/65] via 10.0.0.1, 00:05:01, Serial0/0
2.0.0.0/32 is subnetted, 1 subnets
C 2.2.2.2 is directly connected, Loopback0
3.0.0.0/32 is subnetted, 1 subnets
O 3.3.3.3 [110/65] via 10.0.0.3, 00:05:01, Serial0/0
10.0.0.0/24 is subnetted, 1 subnets
C 10.0.0.0 is directly connected, Serial0/0
R2#ping 3.3.3.3 r 2

Type escape sequence to abort.
Sending 2, 100-byte ICMP Echos to 3.3.3.3, timeout is 2 seconds:
..
Success rate is 0 percent (0/2)

R2#sh frame-relay map
Serial0/0 (up): ip 10.0.0.1 dlci 201(0xC9,0x3090), static,
broadcast,
CISCO, status defined, active

R2 has the route to R3's loopback, but it still unreachable due to the missing L2 mapping of the next hop on the serial interface.
To fix it, on Ser0/0 of R2: "frame-relay map ip 10.0.0.3 201"
on Ser 0/0 of R3: "frame-relay map ip 10.0.0.2 301"
Let's check it:

R2(config)#do sh frame map
Serial0/0 (up): ip 10.0.0.3 dlci 201(0xC9,0x3090), static,
CISCO, status defined, active
Serial0/0 (up): ip 10.0.0.1 dlci 201(0xC9,0x3090), static,
broadcast,
CISCO, status defined, active
R2(config)#do ping 10.0.0.3 r 2

Type escape sequence to abort.
Sending 2, 100-byte ICMP Echos to 10.0.0.3, timeout is 2 seconds:
!!
Success rate is 100 percent (2/2), round-trip min/avg/max = 12/20/28 ms
R2(config)#


But what we can do if we are not allowed to use a static frame relay map statement on serial interfaces of the spokes?
We have to find a way to modify the next hop, to instruct R2 to see R1 as next hop for the R3 loopback..
By the way, we can use the network type point-to-multipoint (broadcast|non-broadcast)
This network type threats the different neighbors as point-to-point connections, and modifies the next hop pointing to the hub:

!-- on R2 (spoke)
R2(config-router)#do sh ip route 3.3.3.3
Routing entry for 3.3.3.3/32
Known via "ospf 1", distance 110, metric 65, type intra area
Last update from 10.0.0.3 on Serial0/0, 05:10:04 ago
Routing Descriptor Blocks:
* 10.0.0.3, from 3.3.3.3, 05:10:04 ago, via Serial0/0
Route metric is 65, traffic share count is 1

!-- on R1 (hub)
R1#conf t
Enter configuration commands, one per line. End with CNTL/Z.
R1(config)#int ser 0/0
R1(config-if)#ip ospf network point-to-multipoint ?
non-broadcast Specify non-broadcast point-to-mpoint network


R1(config-if)#ip ospf network point-to-multipoint
R1(config-if)#
*Mar 1 05:18:24.318: %OSPF-5-ADJCHG: Process 1, Nbr 2.2.2.2 on Serial0/0 from FULL to DOWN, Neighbor Down: Interface down or detached
*Mar 1 05:18:24.318: %OSPF-5-ADJCHG: Process 1, Nbr 3.3.3.3 on Serial0/0 from FULL to DOWN, Neighbor Down: Interface down or detached
*Mar 1 05:18:24.398: %OSPF-5-ADJCHG: Process 1, Nbr 2.2.2.2 on Serial0/0 from LOADING to FULL, Loading Done
*Mar 1 05:18:24.442: %OSPF-5-ADJCHG: Process 1, Nbr 3.3.3.3 on Serial0/0 from LOADING to FULL, Loading Done
R1(config-if)#

... and don't forget to change the network type on R2 too! otherwise adjacency will form, but no routes will be installed, due to the "advertising neighbor unreachable" in the ospf database
(see: Document ID: 7112 "Why Are Some OSPF Routes in the Database but Not in the Routing Table?" )

R2#sh ip ospf neighbor

Neighbor ID Pri State Dead Time Address Interface
1.1.1.1 255 FULL/DR 00:01:46 10.0.0.1 Serial0/0
R2#sh ip route | beg Gate
Gateway of last resort is not set

2.0.0.0/32 is subnetted, 1 subnets
C 2.2.2.2 is directly connected, Loopback0
10.0.0.0/24 is subnetted, 1 subnets
C 10.0.0.0 is directly connected, Serial0/0

R2#sh ip ospf database

OSPF Router with ID (2.2.2.2) (Process ID 1)

Router Link States (Area 0)

Link ID ADV Router Age Seq# Checksum Link count
1.1.1.1 1.1.1.1 484 0x8000002E 0x007F97 4
2.2.2.2 2.2.2.2 632 0x80000011 0x00B2F0 2
3.3.3.3 3.3.3.3 1049 0x8000001C 0x00A4D1 3
R2#
R2# sh ip ospf database router 3.3.3.3

OSPF Router with ID (2.2.2.2) (Process ID 1)

Router Link States (Area 0)

Adv Router is not-reachable
LS age: 343
Options: (No TOS-capability, DC)
LS Type: Router Links
Link State ID: 3.3.3.3
Advertising Router: 3.3.3.3
LS Seq Number: 80000031
Checksum: 0x7AE6
Length: 60
Number of Links: 3

Link connected to: another Router (point-to-point)
(Link ID) Neighboring Router ID: 1.1.1.1
(Link Data) Router Interface address: 10.0.0.3
Number of TOS metrics: 0
TOS 0 Metrics: 64

Link connected to: a Stub Network
(Link ID) Network/subnet number: 10.0.0.3
(Link Data) Network Mask: 255.255.255.255
Number of TOS metrics: 0
TOS 0 Metrics: 0

Link connected to: a Stub Network
(Link ID) Network/subnet number: 3.3.3.3
(Link Data) Network Mask: 255.255.255.255
Number of TOS metrics: 0
TOS 0 Metrics: 1

R2#conf t
Enter configuration commands, one per line. End with CNTL/Z.
R2(config)#
R2(config)#int ser 0/0
R2(config-if)#ip ospf network point-to-multipoint
R2(config-if)#
*Mar 1 13:20:30.300: %OSPF-5-ADJCHG: Process 1, Nbr 1.1.1.1 on Serial0/0 from FULL to DOWN, Neighbor Down: Interface down or detached
*Mar 1 13:20:30.424: %OSPF-5-ADJCHG: Process 1, Nbr 1.1.1.1 on Serial0/0 from LOADING to FULL, Loading Done
R2(config-if)#end
R2#sh ip route | beg Gate
Gateway of last resort is not set

1.0.0.0/32 is subnetted, 1 subnets
O 1.1.1.1 [110/65] via 10.0.0.1, 00:00:12, Serial0/0
2.0.0.0/32 is subnetted, 1 subnets
C 2.2.2.2 is directly connected, Loopback0
3.0.0.0/32 is subnetted, 1 subnets
O 3.3.3.3 [110/129] via 10.0.0.1, 00:00:12, Serial0/0
10.0.0.0/8 is variably subnetted, 3 subnets, 2 masks
O 10.0.0.3/32 [110/128] via 10.0.0.1, 00:00:12, Serial0/0
C 10.0.0.0/24 is directly connected, Serial0/0
O 10.0.0.1/32 [110/64] via 10.0.0.1, 00:00:12, Serial0/0
R2#

Well done, now we have the 3.3.3.3/32 route with the next hop R1, we don't need the frame relay map statemnt for R3.
Note also that the point-to-multipoint interface type threats all neighbors as point-to-point links, even in a multi access interface, without elect DR/BDR:
R1(config-if)#do sh ip ospf nei

Neighbor ID Pri State Dead Time Address Interface
3.3.3.3 0 FULL/ - 00:01:46 10.0.0.3 Serial0/0
2.2.2.2 0 FULL/ - 00:01:54 10.0.0.2 Serial0/0
R1(config-if)#


More posts about ospf interface types will follow very soon... have fun!

Marco

Thursday, February 4, 2010

WCS server: a great management tool for WLCs

Hi all,

finally today I've found the time to play with our new WCS server, freshly installed on a Win2003 virtual machine.

I have the WCS version 6.0.170.0 with base feature set and 500 AP limit.

The first impression really positive, this tool looks like extremely useful to manage configuration changes across multiple wireless lan controllers and for multiple Lightweight Access Points.
In addition, the monitoring and reporting features are great to view performance issues and user activities.

I haven't already imported an autocad map, to map the access-point placement into the various buildings, but that will be the next week fun.

Here some screenshots of the home page, just to view the first impact when I opened the web interface today:





well, in my opinion, this is a great management tool, Cisco has done a good job here!

Marco

PS: thanks to Gian Paolo that has sent me the link for the WCS Online Learning Modules

Monday, February 1, 2010

Wism installation on 6509E

Hi all,
today a new Wism module has been inserted and configured on our 6509E.


Here the log at the insertion time:

Feb  1 11:07:33: %C6KERRDETECT-SP-2-SWBUSSTALL: The switching bus is experiencing stall for 3 seconds
Feb 1 11:07:34: %C6KERRDETECT-SP-2-SWBUSSTALL_RECOVERED: The switching bus stall is recovered and data traffic switching continues
00:00:02: BaseBoard Index:242
00:00:02: DaughterBoard Index:208 (Centralized Forwarding Card)
00:00:02: Gemini Rev#: 3

Firmware compiled 20-Feb-07 14:12 by integ Build [100]


00:00:04: %SYS-CFC7-5-RESTART: System restarted --
Cisco Internetwork Operating System Software
IOS (tm) c6lc2 Software (c6lc2-SP-M), Version 12.2(18)SXF9, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2007 by cisco Systems, Inc.
Compiled Wed 16-May-07 22:48 by kellythw
*Nov 30 00:00:02.155: CFC7: Currently running ROMMON from S (Gold) region
Feb 1 11:08:18: %DIAG-SP-6-RUN_MINIMUM: Module 7: Running Minimal Diagnostics...
Feb 1 11:08:22: %DIAG-SP-6-DIAG_OK: Module 7: Passed Online Diagnostics
Feb 1 11:08:24: %OIR-SP-6-INSCARD: Card inserted in slot 7, interfaces are now online
Feb 1 11:08:48: %WiSM-5-STATE: Controller 1 in slot 7 is Oper-Up
Feb 1 11:08:48: %WiSM-5-STATE: Controller 2 in slot 7 is Oper-Up
Feb 1 11:21:53: %WiSM-5-STATE: Controller 1 in slot 7 is Oper-Up
Feb 1 11:21:55: %WiSM-5-STATE: Controller 1 in slot 7 is Oper-Up
Feb 1 11:21:56: %WiSM-5-STATE: Controller 1 in slot 7 is Oper-Up
Feb 1 11:21:57: %WiSM-5-STATE: Controller 1 in slot 7 is Oper-Up
Feb 1 11:22:19: %WiSM-5-STATE: Controller 1 in slot 7 is Oper-Up
Feb 1 11:22:20: %WiSM-5-STATE: Controller 1 in slot 7 is Oper-Up
Feb 1 11:22:21: %WiSM-5-STATE: Controller 1 in slot 7 is Oper-Up
Feb 1 11:22:22: %WiSM-5-STATE: Controller 1 in slot 7 is Oper-Up


Well, I feel a little nervous at the moment of insertion, upgrading 6500s is not my daily routine at the moment, so analyzing the console output, we have had a 3-4 seconds switching stall.
I guess the switching stall was only in the bus, and not in the crossbar and inside the modules, so in fact no issues at all. :-)
Then, the new wism booted up and we (me and Mauri Ale) started the configuration tasks.

Well, tomorrow we will install the new WCS server to control our wisms, maybe I'll do some screenshots.

Marco