Archive

‘Classical CASE’ 分类存档

Troubleshooting IPoE Session that pending at “ACK-DPM-WAIT”

Introduction

The article will talk about what is “ACK-DPM-WAIT”, and how to troubleshooting the similar scenario. Due to limitation info that couldn’t narrow down, in my CASE, so will update the article if the issue happen again and find RCA.

Problem Description

Version: 5.1.3 + induvial SMU
Platform: 9010 + Mod80 + A9K-MPA-4X10GE
BNG: IPOE, DHCP Proxy, 28k session

My customer found part of BNG session was failure. Trigger is due to customer power supply have issue that cause the asr9k re-power. After 9k reload, found dhcpd and arp have so many alarms, dhcpd was recovery after tried restart process multi times, but arp continue have SPIO alarm even if tried restart process, customer had enabled arp local disable on the BNG port.

The issue sessions got address correct from DHCP, but session would be deleted after 15min. After checked on asr9k, we found issue session pending on ACK_DPM_WAIT status. And the issue was auto recovery at approx.19:00-19:30. And at that timeslot, arp alarm disappear too.
完整阅读

How to decode TCP, UDP and RAW for IOS-XR

做为工程师,常常遇到一些协议交互的问题,需要确认数据包的具体信息,这时常常会用到几种方法:
1. SPAN抓包
对于这种方法,结果分析起来最方便,但操作过程最麻烦
2. debug
这种方法最直观,但是debug数据如果非常多,会影响设备的正常运行

下面就是采用其他方法来达成这种需求,虽然用的是udp来说明,但是同样适用于TCP和RAW:

RP/0/RP1/CPU0:CRS2(config)#udp directory /tmp/udp
RP/0/RP1/CPU0:CRS2(config)#commit
RP/0/RP1/CPU0:CRS2(config)#ipv4 access-list hsrp-packet
RP/0/RP1/CPU0:CRS2(config-ipv4-acl)#20 permit udp any eq 1985 any eq 1985
RP/0/RP1/CPU0:CRS2(config-ipv4-acl)#30 deny ipv4 any any
RP/0/RP1/CPU0:CRS2(config-ipv4-acl)#exit
RP/0/RP1/CPU0:CRS2(config)#ipv6 access-list v6-filter
RP/0/RP1/CPU0:CRS2(config-ipv6-acl)#10 deny ipv6 any any
RP/0/RP1/CPU0:CRS2(config-ipv6-acl)#exit
RP/0/RP1/CPU0:CRS2(config)#commit
RP/0/RP1/CPU0:CRS2(config)#exit
RP/0/RP1/CPU0:CRS2#debug udp packet v4-access-list hsrp-packet v6-access-list v6-filter hex control-block location x/x/cpu0

You can check the capture by follow patch:
RP/0/RP1/CPU0:CRS2#run
# cd /tmp/udp
#ls
#more xxxx

Multi Hierarchical CEF / Load Share

环境

 --------+--------------------+---------
         |   22.22.22.22/32   |
         |                    |
    +----+----+          +----+----+
    | 2.2.2.2 |          | 3.3.3.3 |
    | RouterA |          | RouterB |
    +-\----\--+          +-/---/---+
       \    \             /   /
        \\   \           /   /
          \   \         /  //
           \   \F2/0   /  /
            \\  \     /  /
         F1/0 \  \ F3/0 / F4/0
               *--\-/--*
               |1.1.1.1|
               | CoreA |
               +-------+

不限设备,所有运行IOS的设备,包括GSR,7609等。
在早期版本,不支持Multi hierarchical CEF,仅仅支持一层递归后的转发。这样产生了很多限制,例如今天提到的双PE结构。在特定版本后(包括IOS和IOX),CEF的行为有了改变,并且支持多层CEF。不过CEF的行为也要看平台,因为GSR上任何版本都不支持这种多层CEF。
完整阅读

TS for 6748 output drop

When you found have output queue drop for CEF720 LC, you can check follow step:
1. which port have issue, whether at same ASIC.
2. check whether have other error or have qos on issue port.
3. whether hw queue is full.

Follow is TS example:

Problem description:

Output queue have drop
1. After checked by follow command:
– show tech
– show inter switching x/x
– show int x/x counter de
– show inter x/x summary
完整阅读

OSPF Alarm for %OSPF-4-FLOOD_WAR

根据下面的logging复习各个知识点

1w0d: OSPF: DR/BDR election on TenGigabitEthernet6/4
1w0d: OSPF: Elect BDR 0.0.0.0
1w0d: OSPF: Elect DR 7.7.7.7
1w0d:        DR: 7.7.7.7 (Id)   BDR: none
1w0d: OSPF: Rcv DBD from 8.8.8.8 on TenGigabitEthernet6/4 seq 0x2011 opt 0x2 flag 0x2 len 1472  mtu 0 state EXSTART
!---定义option的类型,详细参考TCP/IP卷一,这里0x2如下所示
Options: 0x02 (E)
0... .... = DN: DN-bit is NOT set
.0.. .... = O: O-bit is NOT set
..0. .... = DC: Demand circuits are NOT supported
...0 .... = L: The packet does NOT contain LLS data block
.... 0... = NP: Nssa is NOT supported
.... .0.. = MC: NOT multicast capable
.... ..1. = E: ExternalRoutingCapability
.... ...0 = MT
1w0d: OSPF: NBR Negotiation Done. We are the MASTER
1w0d: OSPF: Send DBD to 8.8.8.8 on TenGigabitEthernet6/4 seq 0x2012 opt 0x52 flag 0x3 len 1452
 完整阅读