How to capture packets that dropped by NP/CPU?

22 10 月, 2013,由 frank撰写

之前讨论过在XR上，当我们遇到与本设备交互的TCP/UDP和RAW有问题时，可以用下面方法抓下来，然后分析《How to decode TCP, UDP and RAW for IOS-XR》。在76/65上，可以用Netdr，ELAM，CPU span，PB capture，那在咱们的ASR9k上是否有类似好用的工具？答案当然是肯定的，在咱们ASR9k上有两个方法可以抓punt到CPU的包：

在4.3.1以后，XR支持NP Monitor

详细的可以看下面文章： https://supportforums.cisco.com/docs/DOC-29010
这里要注意的是：Note that a captured packet will be DROPPED!
所以一定要注意在选择monitor的counter时，要选择真正的drop counter，而不要把正常的counter给monitor了，那样数据转发就会受影响了。暂时没有测试环境，没法贴出详细测试步骤，文档写的很清楚，详细看上面的文档。

在ASR9k上通过SPP Monitor抓包

下面是一个抓包实例：

RP/0/RSP0/CPU0:ASR9K.18#run attach 0/6/cpu0
Sat Oct 19 01:15:36.985 UTC

attach: Starting session 1 to node 0/6/cpu0

# spp_ui
spp-ui> node counter
copp_sampler
     Disable messages received:               1
-------------------------------
port3/classify
                     forwarded:           59294
Punted to default punt process:            2264
-------------------------------
port3/rx
           whole pkts received:          768891
      begin fragments received:             500
      total fragments received:            1248
        end fragments received:             500
           RX bailout/longjmps:          765405
incomplete chopper ctx after b:              40
-------------------------------
port2/tx
           packets transmitted:            1136
-------------------------------
port3/tx
           packets transmitted:            1139
-------------------------------
client/inject
               inject to port2:            1136
               inject to port3:            1139
-------------------------------
client/punt
              punted to client:          769391
-------------------------------
spp-ui>
spp-ui> trace start 100
spp-ui> trace stop
spp-ui> trace print
Packet serial 2
client/punt:
  length 128 phys_int_index 1 next_ctx 0xdeadbeef

下面拿出一个包来详细说明：
用SPP抓包最好的方法就是先抓一些包下来，然后看看offset，根据具体的需求，用过滤的方法注意过滤，然后最终抓到想要的包。因为不同的平台，不同的包，offset是不一样的，所以要现抓样本分析。

Packet serial 4
client/punt:
length 128 phys_int_index 1 next_ctx 0xdeadbeef
00: 00 70 72 00 00 03 00 65 7a 00 00 03 05 09 ff ff
10: 08 00 00 00 05 02 0f 00 00 00 00 00 02 00 10 00
20: 00 c0 00 00 c7 de 00 00 09 01 00 1f 00 03 0e 01
30: 00 00 00 00 3c 00 00 00 00 00 00 00 00 00 00 00
40: 00 07 02 ff 00 26 98 29 4c e8 01 25 68 98 cf a6
50: 08 00 45 00 00 2e 00 00 00 00 40 06 64 b5 03 03
60: 03 03 08 08 08 08 01 4d 03 e7 00 00 00 00 00 00
70: 00 00 50 00 00 00 8e 8c 00 00 00 01 02 03 04 05
80: 09 8b 66 5d 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 01 00 00 20 43 00 00 00 d3 5c 02 df 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

下面是常用过滤方法和offset的使用方法：

 spp-ui> trace filter set 94 2 0x0303
 !---src address, 94=0x50+dec14=dec(5*16+0)+dec14
 spp-ui> trace filter set 98 2 0x0808
 !---dst address, 有时候设置长度2或4都有问题，只能是1
 spp-ui> trace filter set 91 1 0x06
 !---TCP flag
 spp-ui> trace filter show
 !---查看定义的过滤表项
 spp-ui> trace filter clear 94
 !---清掉之前设置的offset过滤
 spp-ui> trace ascii save
 !---自动保存到 /tmp/spp_packet_trace_ascii，用“q”退出spp_ui，用more命令查看也可以

注意：在用第二种方法时，尽量控制在100个包以内，防止影响正常业务。

2021-0611更新：eXR SPP以及Tcpdump抓包

在64bit的eXR中，SPP抓包有些变化，不再是直接run进去执行spp，而是需要ssh进去，具体方法如下：

RP/0/RSP0/CPU0:CR5#admin show vm
Thu Jun 10 04:04:22.962 UTC

Location: 0/0
Id                Status        IP Address       HB Sent/Recv
-------------------------------------------------------------
sysadmin          running       192.0.8.1        NA/NA
default-sdr       running       192.0.8.3        179906/179906

Location: 0/RSP0
Id                Status        IP Address       HB Sent/Recv
-------------------------------------------------------------
sysadmin          running       192.0.0.1        NA/NA
default-sdr       running       192.0.0.4        3599690/3599690

RP/0/RSP0/CPU0:CR5#run
Thu Jun 10 13:31:18.180 UTC
ssssh[xr-vm_node0_RSP0_CPU0:~]$ssh 192.0.8.3
Last login: Thu Jun 10 03:54:13 2021 from 192.0.0.4
[xr-vm_node0_0_CPU0:~]$
[xr-vm_node0_0_CPU0:~]$
[xr-vm_node0_0_CPU0:~]$spp_ui
spp-ui> trace start 100
spp-ui> trace stop
spp-ui> trace print

但在我的测试里，这种方法可能抓不到traceroute的报文，具体原因不是很清楚，所以换了另一种抓包方式，这也是转成eXR带来的好处，直接用tcpdump抓包，如下：

[xr-vm_node0_0_CPU0:~]$ifconfig
eth-bfd   Link encap:Ethernet  HWaddr be:17:a1:12:a1:f0  
          inet6 addr: fe80::bc17:a1ff:fe12:a1f0/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:9700  Metric:1
          RX packets:6 errors:0 dropped:0 overruns:0 frame:0
          TX packets:16 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:428 (428.0 B)  TX bytes:1296 (1.2 KiB)

eth-bfd.1282 Link encap:Ethernet  HWaddr 56:4b:47:50:4e:42  
          inet6 addr: fe80::544b:47ff:fe50:4e42/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:9700  Metric:1
          RX packets:4 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:232 (232.0 B)  TX bytes:648 (648.0 B)

eth-punt  Link encap:Ethernet  HWaddr ae:7d:f5:1a:cb:47  
          inet6 addr: fe80::ac7d:f5ff:fe1a:cb47/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:9700  Metric:1
          RX packets:45553297 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4179755 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:8728798955 (8.1 GiB)  TX bytes:693317196 (661.1 MiB)

eth-punt.1282 Link encap:Ethernet  HWaddr 56:4b:47:50:4e:53  
          inet6 addr: fe80::544b:47ff:fe50:4e53/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:9700  Metric:1
          RX packets:45553293 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4179747 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:8091052565 (7.5 GiB)  TX bytes:693316548 (661.1 MiB)

eth-vf1   Link encap:Ethernet  HWaddr 9a:8d:94:f9:48:86  
          inet6 addr: fe80::988d:94ff:fef9:4886/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:9700  Metric:1
          RX packets:21065229 errors:0 dropped:0 overruns:0 frame:0
          TX packets:16242307 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:18676667249 (17.3 GiB)  TX bytes:7018115092 (6.5 GiB)

eth-vf1.3073 Link encap:Ethernet  HWaddr 56:49:4b:00:02:03  
          inet addr:192.0.8.3  Bcast:192.255.255.255  Mask:255.0.0.0
          inet6 addr: fe80::988d:94ff:fef9:4886/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:9700  Metric:1
          RX packets:13698951 errors:0 dropped:1466854 overruns:0 frame:0
          TX packets:11465784 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:982255165 (936.7 MiB)  TX bytes:816854781 (779.0 MiB)

eth-vf1.3074 Link encap:Ethernet  HWaddr 56:4b:47:00:02:20  
          inet addr:172.0.2.32  Bcast:172.255.255.255  Mask:255.0.0.0
          inet6 addr: fe80::988d:94ff:fef9:4886/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:9700  Metric:1
          RX packets:7366278 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4776373 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:17399498878 (16.2 GiB)  TX bytes:6201250291 (5.7 GiB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:2320665896 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2320665896 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:364309539143 (339.2 GiB)  TX bytes:364309539143 (339.2 GiB)

[xr-vm_node0_0_CPU0:~]$tcpdump -i eth-punt.1282 -w traceroute.pcap
tcpdump: listening on eth-punt.1282, link-type EN10MB (Ethernet), capture size 262144 bytes
^C
9807 packets captured
9812 packets received by filter
0 packets dropped by kernel
[xr-vm_node0_0_CPU0:~]$scp traceroute.pcap 192.0.0.4:/harddisk:
Warning: Permanently added '192.0.0.4' (ECDSA) to the list of known hosts.
traceroute.pcap                                                                                       100% 1988KB   1.9MB/s   00:00 
[xr-vm_node0_0_CPU0:~]$
[xr-vm_node0_0_CPU0:~]$ssh 192.0.0.4
Last login: Thu Jun 10 03:58:17 2021 from 192.0.0.4
[xr-vm_node0_RSP0_CPU0:~]$cd /harddisk:
[xr-vm_node0_RSP0_CPU0:/harddisk:]$ls
traceroute.pcap

存下来的pcap文件没法直接看，因为这里仍然带着Internal Header，其实思科应该出一个小工具，直接去掉Internal header，这样可以让Wireshark直接读取；虽然没有，我们也可以通过其他方式查看，用Wireshark打开后，另存为“K12 text file”，用编辑器打开，然后直接搜索DST/SRC 的Mac地址，然后删掉Internal Header：

+---------+---------------+----------+
03:14:50,816,351   ETHER
|0   |56|4b|47|50|4e|53|00|65|7a|00|00|01|01|d8|00|00|05|00|0f|00|00|00|00|00|00|00|00|00|56|4b|20|02|10|0b|00|00|04|00|04|40|09|01|05|56|00|03|0e|03|00|00|00|00|46|00|00|00|00|01|e8|00|00|00|00|00|00|00|cc|03|dc|77|4c|f1|ef|68|00|2e|c7|ba|55|1b|88|47|03|e8|71|01|45|b8|00|34|b2|eb|00|00|01|11|f2|0c|0a|00|00|03|0a|00|00|07|b2|e6|82|9e|00|20|00|00|00|00|00|00|00|00|00|00|00|00|00|00|00|00|00|00|00|00|00|00|00|00|00|00|
+---------+---------------+----------+

用这个网址直接decode即可：https://hpd.gasmi.net

2023-0321更新：XRv9k Capture packets

之前在老东家时就像把xrv9k如何抓包的方法整理出来，但一直没做。。。最近用xrv9k做实验时，发现丢包，所以想看看进来的报文，因此顺便简单整理下，方便以后翻阅；

方法很简单，主要就是下面几个命令，具体可以参考这篇文章：https://community.cisco.com/t5/service-providers-knowledge-base/xrv9000-packet-path-troubleshooting-note/ta-p/3163691

debug controllers dpa packet-trace [line|inject] <# to trace>
show controllers dpa packet-trace 
clear controllers dpa packet-trace

本文出自 Frank's Blog

本文链接：How to capture packets that dropped by NP/CPU?
版权声明：本文为原创文章，仅代表个人观点，版权归 Frank Zhao 所有，转载时请注明本文出处及文章链接

学习笔记 ASR9k,WIRESHARK

你可以留言,或者trackback 从你的网站

Mix for Fair-Queue and WRED on ASR1k’s class-default«

»MAC(Mountain Lion) python2.7 + eclipse开发环境配置

留言哦

返回顶端

『无题』

I persist, therefore I am!!!

Memo

Perseverance sometimes equals genius!
There are only two creatures in the world who can surmount the pyramid:

—–the eagle and the snail.
Useful Link

1. IANA - AFI
2. IANA - SAFI
3. IANA - BGP Ex Comm
4. RPM pbone.net
5. Linux Packages Search
Useful Toots

Linux/http server:

python -m SimpleHTTPServer <port>

python -m http.server <port>

PS: only one thread, if require more threads, can replace by “caddy”.

Win10/http server:

http://www.rejetto.com/hfs/

Win10/md5:

certutil -hashfile .\xxxx MD5
分类目录
- CCDE (16)
- CCIE SEC (4)
- CCIE SP (55)
- CCIE VOICE (3)
- Classical CASE (26)
- Linux (75)
- Mobile Phone (4)
- Programmer (44)
- Solution (3)
- Uncategorized (6)
- Windows常用技术 (10)
- 似水流年 (17)
- 学习笔记 (191)
- 常用资料 (44)
- 网络技术 (39)
近期文章
其他操作

How to capture packets that dropped by NP/CPU?

在4.3.1以后，XR支持NP Monitor

在ASR9k上通过SPP Monitor抓包

2021-0611更新：eXR SPP以及Tcpdump抓包

2023-0321更新：XRv9k Capture packets

留言哦

『无题』

Memo

Useful Link

Useful Toots

分类目录

近期文章

其他操作