OpenR on NCS5500
For detail info, please refer to following Akshat’s article, for my article only simply summary step to build openr on ncs5500. Thanks Akshat’s help for the openr set up 🙂
1. Set up the private insecure registry on your server
Refer as follow:
https://docs.docker.com/registry/deploying/
[root@nso ~]# nano /etc/yum.repos.d/docker.repo [root@nso ~]# yum install docker-engine [root@nso ~]# systemctl start docker [root@nso ~]# docker run -d -p 5000:5000 --restart=always --name registry registry:2 [root@nso ~]# docker pull akshshar/openr-xr [root@nso ~]# docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES c4636568f48a registry:2 "/entrypoint.sh /e..." 6 minutes ago Up 6 minutes 0.0.0.0:5000->5000/tcp registry [root@nso ~]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE registry 2 f32a97de94e1 13 days ago 25.8MB akshshar/openr-xr latest b51c260b060e 2 months ago 1.76GB [root@nso ~]# docker tag akshshar/openr-xr 10.75.58.72:5000/openr-xr [root@nso ~]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE registry 2 f32a97de94e1 13 days ago 25.8MB akshshar/openr-xr latest b51c260b060e 2 months ago 1.76GB 10.75.58.72:5000/openr-xr latest b51c260b060e 2 months ago 1.76GB [root@nso ~]# docker push 10.75.58.72:5000/openr-xr The push refers to a repository [10.75.58.72:5000/openr-xr] Get https://10.75.58.72:5000/v1/_ping: http: server gave HTTP response to HTTPS client [root@nso ~]#
2. Add follow at “/etc/sysconfig/docker”
After changing the docker opts, please wait some seconds, docker will auto restart.
DOCKER_OPTS=" --insecure-registry 10.75.58.72:5000"
3. Config GRPC on ncs55
grpc port 57777 no-tls <<< after 651, auto enable tls, base on now openr package, disable tls service-layer
4. docker pull openr from priviate registry
[xr-vm_node0_RP0_CPU0:~]$docker pull 10.75.58.72:5000/openr-xr Using default tag: latest latest: Pulling from openr-xr 4f1bb8b65720: Pull complete 4791a9f80860: Pull complete c7bccbb1d183: Pull complete 94925a7a8f89: Pull complete 1a776d5f8f21: Pull complete 21601f5e9bd9: Extracting [=========================================> ] 383.3 MB/462.9 MB 21601f5e9bd9: Pull complete Digest: sha256:e0fed850c4f0da4ee8f64ed2739cc6efd117f78f7a98491c6688963bfe0c2afd Status: Downloaded newer image for 10.75.58.72:5000/openr-xr:latest [xr-vm_node0_RP0_CPU0:~]$docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES [xr-vm_node0_RP0_CPU0:~]$docker images REPOSITORY TAG IMAGE ID CREATED SIZE 10.75.58.72:5000/openr-xr latest b51c260b060e 11 weeks ago 1.756 GB
5. Create “run_openr_.sh” and “hosts_” in /misc/app_host
Detail follow by https://github.com/akshshar/openr-xr/tree/openr20171212/docker/iosxr/slapi/rtr1
,btw please attention, must add “x” priviledge for “run_openr_R31.sh”:
[R31:/misc/app_host]$ ls -l total 528 lrwxrwxrwx. 1 root root 14 Mar 29 02:33 app_host -> /misc/app_host drwxr-xr-x. 2 root root 4096 Mar 30 08:36 app_hosting_apply_cmd drwxr-xr-x. 5 root root 4096 Dec 7 03:18 app_repo drwx-----x. 9 root root 4096 Dec 7 03:19 docker srw-rw----. 1 root root 0 Mar 31 14:55 docker.sock drwxr-xr-x. 5 root root 4096 Dec 7 03:18 etc -rw-r--r--. 1 root root 60 Apr 1 15:43 hosts_R31 <<< create drwx------. 2 root root 16384 Dec 7 03:12 lost+found -rw-r--r--. 1 root root 486071 Apr 1 16:04 openr_logs -rwxr-xr--. 1 root root 5047 Apr 1 15:44 run_openr_R31.sh <<< create drwxrwxr-x. 2 root sudo 4096 Dec 7 03:18 scratch drwxr-xr-x. 3 root root 4096 Dec 7 03:18 var [R31:/misc/app_host]$ chmod +x /misc/app_host/run_openr_R31.sh [R31:/misc/app_host]$ ls -l /misc/app_host/run_openr_R31.sh -rwxr-xr-x. 1 root root 5048 Apr 3 04:05 /misc/app_host/run_openr_R31.sh
6. docker run the openr
As above instruct, all the expected files must be created before docker run, e.g “/var/run/netns”, “/misc/app_host” and “/misc/app_host/hosts_R31”. If docker run have issue, suggest remove all -v option, then add back one by one for troubleshooting.
docker run -itd --name openr --cap-add=SYS_ADMIN --cap-add=NET_ADMIN -v /var/run/netns:/var/run/netns -v /misc/app_host:/root -v /misc/app_host/hosts_R31:/etc/hosts --hostname R31 10.75.58.72:5000/openr-xr bash
If use “bash” keyword, I must manual login docker and enable openr after docker start each time, so replaced by follow. Then stop it only using “docker stop openr” and start again using “docker start openr”, not need manual enable openr again 🙂
docker run -itd --name openr --cap-add=SYS_ADMIN --cap-add=NET_ADMIN -v /var/run/netns:/var/run/netns -v /misc/app_host:/root -v /misc/app_host/hosts_R31:/etc/hosts --hostname R31 10.75.58.72:5000/openr-xr /root/run_openr_R31.sh
7. Check SL info after docker and openr runing
RP/0/RP0/CPU0:R31#show service-layer vrf all Mon Apr 1 16:12:33.961 UTC vrf name: default, vrf afi: IPv4, vrf state: eof, vrf magic: valid, vrf purge time: 500, vrf admin distance: 99, vrf flags: eof , vrf name: default, vrf afi: IPv6, vrf state: eof, vrf magic: valid, vrf purge time: 500, vrf admin distance: 99, vrf flags: eof ,
8. Enable peer openr, then check openr interface info
After enabling 1st openr, no any interface info. Until 2nd openr enable, link monitor will send “hello” and auto detect adj by a link-local ipv6 multicast address/udp, then update to KV-Store. But you will find only interface up, but no any adj info, that mean no hello interaction between openr.
RP/0/RP0/CPU0:R31#bash Tue Apr 2 00:13:33.262 UTC [R31:~]$ docker exec -it openr bash root@R31:/# ip netns exec global-vrf bash root@R31:/# breeze kvstore interfaces > R31's interfaces Interface Status ifIndex Addresses ----------- -------- --------- ----------------------- Hg0_0_1_1 UP 55 fe80::2bc:60ff:fe72:4a4 Hg0_0_1_3 UP 53 fe80::2bc:60ff:fe72:4ac Tg0_0_0_19 UP 29 fe80::2bc:60ff:fe72:44c root@R31:/# breeze kvstore adj root@R31:/#
9. Troubleshooting and enable openr adj
After checked, there are two issues, after fixed, adj info is ok.
– Peer devices R32 miss a cmd
RP/0/RP0/CPU0:R32#sh run int hun0/0/1/1
Tue Apr 2 15:04:24.116 UTC
interface HundredGigE0/0/1/1
description To-R31-HundredGigE0/0/1/1
mtu 9000
ipv6 nd unicast-ra <<< this must have
ipv6 address 2001:ab::2/64
ipv6 enable <<< this must have, miss the cmd that will enable ipv6 in openr
load-interval 30
– Due to I am not enable ipv4 on physical port, so need to disable ipv4 in “run_openr_R31.sh”, at default, open/R will not send discovery messages out of interfaces without ipv4 addresses. Setting it to false will restrict operation to ipv6 neighbors and ipv6 routes only.
# Enable v4
ENABLE_V4=false <<< default is true
– Due to I use the same “run_openr_<xxx>.sh” at R31 and R32, that will cuase two openr send same ipv4 and ipv6 prefix…this is incorrect. A neighbor will only learn routes and program them into IOS-XR RIB if they are new/unique, so correct that:
R32:
# List of comma separated list of prefixes to announce
# e.g. “face:cafe::1/128,face:b00c::/64”
#ROUTE_LIST=$(python /root/increment_ipv4_prefix2.py)
ROUTE_LIST=
PREFIXES=“60.1.1.1/32,${ROUTE_LIST},face:cafe::20/128,face:b00c::20/128”R31:
# List of comma separated list of prefixes to announce
# e.g. “face:cafe::1/128,face:b00c::/64”
#ROUTE_LIST=$(python /root/increment_ipv4_prefix1.py)
ROUTE_LIST=
PREFIXES=“50.1.1.1/32,${ROUTE_LIST},face:cafe::10/128,face:b00c::10/128”
Then check adj again:
RP/0/RP0/CPU0:R31#bash
Wed Apr 3 03:14:19.317 UTC
[R31:~]$ docker exec -it openr bash
root@R31:/# ip netns exec global-vrf bash
root@R31:/# breeze kvstore adj
> R31's adjacencies, version: 529, Node Label: 27028, Overloaded?: False
Neighbor Local Interface Remote Interface Metric Weight Adj Label NextHop-v4 NextHop-v6 Uptime
R32 Hg0_0_1_1 Hg0_0_1_1 2 1 50051 0.0.0.0 fe80::2bc:60ff:fe71:eca4 11h28m
root@R31:/# breeze fib list
== R31's FIB routes by client 786 ==
> face:b00c::20/128
via fe80::2bc:60ff:fe71:eca4@Hg0_0_1_1
> face:cafe::20/128
via fe80::2bc:60ff:fe71:eca4@Hg0_0_1_1
root@R31:/# exit
exit
root@R31:/# exit
exit
[R31:~]$ exit
logout
RP/0/RP0/CPU0:R31#show ipv6 interface hun0/0/1/1
Wed Apr 3 03:18:04.457 UTC
HundredGigE0/0/1/1 is Up, ipv6 protocol is Up, Vrfid is default (0x60000000)
IPv6 is enabled, link-local address is fe80::2bc:60ff:fe72:4a4
Global unicast address(es):
2001:ab::1, subnet is 2001:ab::/64
Joined group address(es): ff02::1:ff00:1 ff02::1:ff72:4a4 ff02::2
ff02::1
MTU is 9000 (8986 is available to IPv6)
ICMP redirects are disabled
ICMP unreachables are enabled
ND DAD is enabled, number of DAD attempts 1
ND reachable time is 0 milliseconds
ND cache entry limit is 1000000000
ND advertised retransmit interval is 0 milliseconds
ND router advertisements are sent every 160 to 240 seconds
ND router advertisements live for 1800 seconds
Hosts use stateless autoconfig for addresses.
Outgoing access list is not set
Inbound common access list is not set, access list is not set
Table Id is 0xe0800000
Complete protocol adjacency: 0
Complete glean adjacency: 1
Incomplete protocol adjacency: 0
Incomplete glean adjacency: 0
Dropped protocol request: 0
Dropped glean request: 0
RP/0/RP0/CPU0:R31#sh ipv6 nei hun0/0/1/1
Wed Apr 3 03:18:14.247 UTC
IPv6 Address Age Link-layer Add State Interface Location
fe80::2bc:60ff:fe71:eca4 92 00bc.6071.eca4 REACH Hu0/0/1/1 0/0/CPU0
[Mcast adjacency] - 0000.0000.0000 REACH Hu0/0/1/1 0/0/CPU0
RP/0/RP0/CPU0:R31#sh route ipv6
Wed Apr 3 03:18:29.241 UTC
Codes: C - connected, S - static, R - RIP, B - BGP, (>) - Diversion path
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2, E - EGP
i - ISIS, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, su - IS-IS summary null, * - candidate default
U - per-user static route, o - ODR, L - local, G - DAGR, l - LISP
A - access/subscriber, a - Application route
M - mobile route, r - RPL, t - Traffic Engineering, (!) - FRR Backup path
Gateway of last resort is not set
C 2001:1a::/64 is directly connected,
12:37:58, TenGigE0/0/0/19
L 2001:1a::2/128 is directly connected,
12:37:58, TenGigE0/0/0/19
C 2001:ab::/64 is directly connected,
12:37:58, HundredGigE0/0/1/1
L 2001:ab::1/128 is directly connected,
12:37:58, HundredGigE0/0/1/1
C 2001:ac::/64 is directly connected,
12:37:59, HundredGigE0/0/1/3
L 2001:ac::1/128 is directly connected,
12:37:59, HundredGigE0/0/1/3
i L2 2001:bc::/64
[115/2] via fe80::2bc:60ff:fe71:d4a0, 11:47:58, HundredGigE0/0/1/3
[115/2] via fe80::2bc:60ff:fe71:eca4, 11:47:58, HundredGigE0/0/1/1
a face:b00c::20/128 <<<---
[99/0] via fe80::2bc:60ff:fe71:eca4, 00:01:09, HundredGigE0/0/1/1
a face:cafe::20/128 <<<---
[99/0] via fe80::2bc:60ff:fe71:eca4, 00:01:09, HundredGigE0/0/1/1
S fc00:1::1/128
[1/0] via 2001:1a::1, 12:37:58
i L2 fc00:c::/64
[115/1] via fe80::2bc:60ff:fe71:d4a0, 12:37:59, HundredGigE0/0/1/3
RP/0/RP0/CPU0:R31#