How To Check BGP Memory OF ASR9k
22 3 月, 2017,由 frank撰写
Introduction
Refer to some CASE, customer concern why BGP take so many memory resource, and how to optimize BGP, and why memory not release after optimize, and so on. That hard to answer, in order to get answer, we need check more information. From the article, you can simple to know how to check BGP memory, and how to analyzing the BGP memory, those informaiton will help you and customer to do special optimize.
Solution
1. Default scenario, no any BGP route
RP/0/RSP0/CPU0:xxx#show bgp vrf all sum
Fri Mar 13 11:17:50.267 GMT
VRF: CAP
--------
BGP VRF CAP, state: Active
BGP Route Distinguisher: 132:10042
VRF ID: 0x60000002
BGP router identifier 192.168.1.24, local AS number 132
Non-stop routing is enabled
BGP table state: Active
Table ID: 0xe0000011 RD version: 42
BGP main routing table version 42
BGP NSR Initial initsync version 9 (Reached)
BGP NSR/ISSU Sync-Group versions 42/0
BGP is operating in STANDALONE mode.
Process RcvTblVer bRIB/RIB LabelVer ImportVer SendTblVer StandbyVer
Speaker 42 42 42 42 42 42
Neighbor Spk AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down St/PfxRcd
12.216.156.105 0 1943 74 80 42 0 0 00:35:37 0
12.216.156.237 0 3491 74 80 42 0 0 00:35:34 0
12.216.157.105 0 1944 74 80 42 0 0 00:35:35 0
12.216.158.105 0 1945 0 57 0 0 0 00:00:00 Idle
RP/0/RSP0/CPU0:xxx#show bgp vpnv4 un sum
Fri Mar 13 11:18:02.771 GMT
BGP router identifier 192.168.1.24, local AS number 132
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0x0 RD version: 0
BGP main routing table version 42
BGP NSR Initial initsync version 9 (Reached)
BGP NSR/ISSU Sync-Group versions 42/0
BGP scan interval 60 secs
BGP is operating in STANDALONE mode.
Process RcvTblVer bRIB/RIB LabelVer ImportVer SendTblVer StandbyVer
Speaker 42 42 42 42 42 42
Neighbor Spk AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down St/PfxRcd
192.168.0.5 0 132. 27 30 0 0 0 00:00:53 Idle
RP/0/RSP0/CPU0:xxx#sh bgp
Fri Mar 13 11:18:18.105 GMT
RP/0/RSP0/CPU0:xxx#sh bgp vrf CAP
Fri Mar 13 11:18:22.584 GMT
RP/0/RSP0/CPU0:xxx# 2. Check bgp memory status by “sh bgp all all pro perfor det“
Number Memory Used
Attributes: 0 0
AS Paths: 0 0
Communities: 0 0
Extended communities: 0 0
PMSI Tunnel attr: 0 0
RIBRNH Tunnel attr: 0 0
PPMP attr: 0 0
PE distinguisher labels: 0 0
Route Reflector Entries: 0 0
Nexthop Entries: 30 22680
Number Mem Used
Remote Prefixes: 0 0
Remote Paths: 0 0
Remote Path-elems: 0 0
Remote RDs: 0 0
Local Prefixes: 0 0
Local Paths: 0 0
Local RDs: 1 80
Total Prefixes: 0 0
Total Paths: 0 0
Total Path-elems: 0 0
Imported Paths: 0 0
Total RDs: 1 80 3. Set up VPNv4 neighbor and sent 3 route from remote PE
RP/0/RSP0/CPU0:xxx#sh bgp vpnv4 un sum
Fri Mar 13 11:28:38.821 GMT
BGP router identifier 192.168.1.24, local AS number 132
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0x0 RD version: 0
BGP main routing table version 48
BGP NSR Initial initsync version 9 (Reached)
BGP NSR/ISSU Sync-Group versions 48/0
BGP scan interval 60 secs
BGP is operating in STANDALONE mode.
Process RcvTblVer bRIB/RIB LabelVer ImportVer SendTblVer StandbyVer
Speaker 48 48 48 48 48 48
Neighbor Spk AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down St/PfxRcd
192.168.0.5 0 132 33 35 48 0 0 00:00:06 3
RP/0/RSP0/CPU0:xxx#sh bgp
Fri Mar 13 11:28:45.432 GMT
RP/0/RSP0/CPU0:xxx#
RP/0/RSP0/CPU0:xxx#show bgp vpnv4 un
Fri Mar 13 11:28:53.706 GMT
BGP router identifier 192.168.1.24, local AS number 132
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0x0 RD version: 0
BGP main routing table version 48
BGP NSR Initial initsync version 9 (Reached)
BGP NSR/ISSU Sync-Group versions 48/0
BGP scan interval 60 secs
Status codes: s suppressed, d damped, h history, * valid, > best
i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 132:10041
*>i103.0.0.0/16 192.168.0.5 100 0 2003 i
*>i103.0.32.0/19 192.168.0.5 100 0 2003 i
*>i103.0.48.0/20 192.168.0.5 100 0 2003 i
Route Distinguisher: 132:10042 (default for vrf CAP)
*>i103.0.0.0/16 192.168.0.5 100 0 2003 i
*>i103.0.32.0/19 192.168.0.5 100 0 2003 i
*>i103.0.48.0/20 192.168.0.5 100 0 2003 i Number Memory Used
Attributes: 1 248
AS Paths: 1 486
Communities: 0 0
Extended communities: 1 496
PMSI Tunnel attr: 0 0
RIBRNH Tunnel attr: 0 0
PPMP attr: 0 0
PE distinguisher labels: 0 0
Route Reflector Entries: 0 0
Nexthop Entries: 30 22680
Number Mem Used
Remote Prefixes: 3 297
Remote Paths: 3 192
Remote Path-elems: 3 156
Remote RDs: 1 80
Local Prefixes: 3 297
Local Paths: 3 192
Local RDs: 1 80
Total Prefixes: 6 594 <<< remote+local
Total Paths: 6 384
Total Path-elems: 6 384
Imported Paths: 3 192
Total RDs: 2 160 4. Add multi AS-path on 3 bgp route at remote PE
RP/0/RSP0/CPU0:xxx#show bgp vpnv4 un
Fri Mar 13 11:53:07.595 GMT
BGP router identifier 192.168.1.24, local AS number 132
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0x0 RD version: 0
BGP main routing table version 72
BGP NSR Initial initsync version 9 (Reached)
BGP NSR/ISSU Sync-Group versions 72/0
BGP scan interval 60 secs
Status codes: s suppressed, d damped, h history, * valid, > best
i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 132:10041
*>i103.0.0.0/16 192.168.0.5 100 0 2003 100 200 300 400 i
*>i103.0.32.0/19 192.168.0.5 100 0 2003 500 600 700 800 i
*>i103.0.48.0/20 192.168.0.5 100 0 2003 900 1000 1100 1200 i
Route Distinguisher: 132:10042 (default for vrf CAP)
*>i103.0.0.0/16 192.168.0.5 100 0 2003 100 200 300 400 i
*>i103.0.32.0/19 192.168.0.5 100 0 2003 500 600 700 800 i
*>i103.0.48.0/20 192.168.0.5 100 0 2003 900 1000 1100 1200 i Number Memory Used
Attributes: 3 744
AS Paths: 3 1512 <<< if bgp route have diff as-path, will +1 at here. So it is 3.
Communities: 0 0
Extended communities: 1 496
PMSI Tunnel attr: 0 0
RIBRNH Tunnel attr: 0 0
PPMP attr: 0 0
PE distinguisher labels: 0 0
Route Reflector Entries: 0 0
Nexthop Entries: 30 22680
Number Mem Used
Remote Prefixes: 3 297
Remote Paths: 3 192
Remote Path-elems: 3 156
Remote RDs: 1 80
Local Prefixes: 3 297
Local Paths: 3 192
Local RDs: 1 80
Total Prefixes: 6 594
Total Paths: 6 384
Total Path-elems: 6 384
Imported Paths: 3 192
Total RDs: 2 160 5. Change follow as-path and check behavior
RP/0/RSP0/CPU0:xxx#sh bgp vpnv4 un
Fri Mar 13 12:17:46.717 GMT
BGP router identifier 192.168.1.24, local AS number 132
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0x0 RD version: 0
BGP main routing table version 148
BGP NSR Initial initsync version 9 (Reached)
BGP NSR/ISSU Sync-Group versions 148/0
BGP scan interval 60 secs
Status codes: s suppressed, d damped, h history, * valid, > best
i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 132:10041
*>i103.0.0.0/16 192.168.0.5 100 0 2003 100 i
*>i103.0.32.0/19 192.168.0.5 100 0 2003 100 i
*>i103.0.48.0/20 192.168.0.5 100 0 2003 300 i
Route Distinguisher: 132:10042 (default for vrf CAP)
*>i103.0.0.0/16 192.168.0.5 100 0 2003 100 i
*>i103.0.32.0/19 192.168.0.5 100 0 2003 100 i
*>i103.0.48.0/20 192.168.0.5 100 0 2003 300 iRP/0/RSP0/CPU0:xxx#sh bgp all all process performance-statistics detail
Fri Mar 13 12:18:35.881 GMT
Number Memory Used
Attributes: 2 496
AS Paths: 2 984 <<< two as-path, one is 2003 100, another is 2003 300
Communities: 0 0
Extended communities: 1 496
PMSI Tunnel attr: 0 0
RIBRNH Tunnel attr: 0 0
PPMP attr: 0 0
PE distinguisher labels: 0 0
Route Reflector Entries: 0 0
Nexthop Entries: 30 22680
Number Mem Used
Remote Prefixes: 3 297
Remote Paths: 3 192
Remote Path-elems: 3 156
Remote RDs: 1 80
Local Prefixes: 3 297
Local Paths: 3 192
Local RDs: 1 80
Total Prefixes: 6 594
Total Paths: 6 384
Total Path-elems: 6 384
Imported Paths: 3 192
Total RDs: 2 160 6. Enable local neighbor and sent 3 BGP route
RP/0/RSP0/CPU0:xxx#sh bgp vrf CAP sum
Fri Mar 13 12:22:30.598 GMT
BGP VRF CAP, state: Active
BGP Route Distinguisher: 132:10042
VRF ID: 0x60000002
BGP router identifier 192.168.1.24, local AS number 132
Non-stop routing is enabled
BGP table state: Active
Table ID: 0xe0000011 RD version: 151
BGP main routing table version 151
BGP NSR Initial initsync version 9 (Reached)
BGP NSR/ISSU Sync-Group versions 151/0
BGP is operating in STANDALONE mode.
Process RcvTblVer bRIB/RIB LabelVer ImportVer SendTblVer StandbyVer
Speaker 151 151 151 151 151 151
Neighbor Spk AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down St/PfxRcd
12.216.156.105 0 1943 203 251 151 0 0 01:40:18 0
12.216.156.237 0 3491 204 251 151 0 0 01:40:15 3
12.216.157.105 0 1944 203 251 151 0 0 01:40:15 0
12.216.158.105 0 1945 0 159 0 0 0 00:00:00 Idle
RP/0/RSP0/CPU0:xxx#sh bgp vrf CAP
Fri Mar 13 12:22:56.730 GMT
BGP VRF CAP, state: Active
BGP Route Distinguisher: 132:10042
VRF ID: 0x60000002
BGP router identifier 192.168.1.24, local AS number 132
Non-stop routing is enabled
BGP table state: Active
Table ID: 0xe0000011 RD version: 151
BGP main routing table version 151
BGP NSR Initial initsync version 9 (Reached)
BGP NSR/ISSU Sync-Group versions 151/0
Status codes: s suppressed, d damped, h history, * valid, > best
i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 132:10042 (default for vrf CAP)
*> 100.0.0.0/28 12.216.156.237 0 3491 i
*> 100.0.0.16/28 12.216.156.237 0 3491 i
*> 100.0.0.32/28 12.216.156.237 0 3491 i
*>i103.0.0.0/16 192.168.0.5 100 0 2003 100 i
*>i103.0.32.0/19 192.168.0.5 100 0 2003 100 i
*>i103.0.48.0/20 192.168.0.5 100 0 2003 300 i RP/0/RSP0/CPU0:xxx#sh bgp all all process performance-statistics detail
Fri Mar 13 12:23:07.331 GMT
Number Memory Used
Attributes: 3 744
AS Paths: 3 1470
Communities: 0 0
Extended communities: 1 496
PMSI Tunnel attr: 0 0
RIBRNH Tunnel attr: 0 0
PPMP attr: 0 0
PE distinguisher labels: 0 0
Route Reflector Entries: 0 0
Nexthop Entries: 30 22680
Number Mem Used
Remote Prefixes: 3 297
Remote Paths: 3 192
Remote Path-elems: 3 156
Remote RDs: 1 80
Local Prefixes: 6 594
Local Paths: 6 384
Local RDs: 1 80
Total Prefixes: 9 891
Total Paths: 9 576
Total Path-elems: 9 576
Imported Paths: 3 192
Total RDs: 2 160 7. Enable local 2nd neighbor and sent same 3 BGP route
RP/0/RSP0/CPU0:xxx#sh bgp vrf CAP sum
Fri Mar 13 12:25:52.032 GMT
BGP VRF CAP, state: Active
BGP Route Distinguisher: 132:10042
VRF ID: 0x60000002
BGP router identifier 192.168.1.24, local AS number 132
Non-stop routing is enabled
BGP table state: Active
Table ID: 0xe0000011 RD version: 154
BGP main routing table version 154
BGP NSR Initial initsync version 9 (Reached)
BGP NSR/ISSU Sync-Group versions 154/0
BGP is operating in STANDALONE mode.
Process RcvTblVer bRIB/RIB LabelVer ImportVer SendTblVer StandbyVer
Speaker 154 154 154 154 154 154
Neighbor Spk AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down St/PfxRcd
12.216.156.105 0 1943 211 258 154 0 0 01:43:39 3
12.216.156.237 0 3491 211 258 154 0 0 01:43:36 3
12.216.157.105 0 1944 210 258 154 0 0 01:43:37 0
12.216.158.105 0 1945 0 164 0 0 0 00:00:00 Idle
RP/0/RSP0/CPU0:xxx#sh bgp vrf CAP
Fri Mar 13 12:26:07.952 GMT
BGP VRF CAP, state: Active
BGP Route Distinguisher: 132:10042
VRF ID: 0x60000002
BGP router identifier 192.168.1.24, local AS number 132
Non-stop routing is enabled
BGP table state: Active
Table ID: 0xe0000011 RD version: 154
BGP main routing table version 154
BGP NSR Initial initsync version 9 (Reached)
BGP NSR/ISSU Sync-Group versions 154/0
Status codes: s suppressed, d damped, h history, * valid, > best
i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 132:10042 (default for vrf CAP)
*> 100.0.0.0/28 12.216.156.105 0 1943 i
* 12.216.156.237 0 3491 i
*> 100.0.0.16/28 12.216.156.105 0 1943 i
* 12.216.156.237 0 3491 i
*> 100.0.0.32/28 12.216.156.105 0 1943 i
* 12.216.156.237 0 3491 i
*>i103.0.0.0/16 192.168.0.5 100 0 2003 100 i
*>i103.0.32.0/19 192.168.0.5 100 0 2003 100 i
*>i103.0.48.0/20 192.168.0.5 100 0 2003 300 i
Processed 6 prefixes, 9 paths RP/0/RSP0/CPU0:xxx#sh bgp all all process performance-statistics detail
Fri Mar 13 12:26:21.961 GMT
Number Memory Used
Attributes: 5 1240 <<<
AS Paths: 4 1956
Communities: 0 0
Extended communities: 1 496
PMSI Tunnel attr: 0 0
RIBRNH Tunnel attr: 0 0
PPMP attr: 0 0
PE distinguisher labels: 0 0
Route Reflector Entries: 0 0
Nexthop Entries: 30 22680
Number Mem Used
Remote Prefixes: 3 297
Remote Paths: 3 192
Remote Path-elems: 3 156
Remote RDs: 1 80
Local Prefixes: 6 594
Local Paths: 9 576
Local RDs: 1 80
Multipath Prefixes: 3 297
Multipath Paths: 6 384
Total Prefixes: 9 891
Total Paths: 12 768
Total Path-elems: 9 768
Imported Paths: 3 192
Total RDs: 2 160 8. Check BGP memory change by follow Table
As follow example, we can check prefix/path had reduced, check by “Mem Used”, but BGP memory didn’t reduce, after checked, all memory hold on chunk, please check by “Top 8”.
9. BGP Chunk not release resource to system
After customer optimize BGP, scales downgrade, but memory not reduce, as above example, that maybe match follow scenario. Follow 1000 byte each trunk only example
– normal scenario:
– abnormal scenario, need to restart BGP/RPFO to release mem resource:
