Manual set up two containers by “docker run”

6月 7th, 2022,由 frank撰写

安装docker步骤此处忽略，详细可以看之前的文章；

容器的网络类型

默认docker三种网络：

[root@localhost ~]# docker network ls
NETWORK ID     NAME      DRIVER    SCOPE
0dc7621fa6c0   bridge    bridge    local
09d08085f98c   host      host      local
8da76deb9eb9   none      null      local

先随便开一个centos的docker，“-t” 指定一个伪终端，保持docker可以打开，否则会出现“Exited (0)”的问题：

[root@localhost ~]# docker run --name centos -dt centos

查看容器的网络类型，可以看到当创建容器不指定网络类型，默认会绑定到bridge上：

[root@localhost ~]# docker inspect 355a |grep "Networks" -A 100
            "Networks": {
                "bridge": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "NetworkID": "0dc7621fa6c0e40a1129104ed94deac543cf82ce44ab79fdf7ed38b9975f9a78",
                    "EndpointID": "2dae4a431efd76066081424e47a7e6bc53e4102b14bf36bbdfda34492efd6369",
                    "Gateway": "172.17.0.1",
                    "IPAddress": "172.17.0.2",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:ac:11:00:02",
                    "DriverOpts": null

手动创建容器并绑定自定义的mynet0

安装bridge-utils，使用brctl命令：

[root@localhost ~]# yum install bridge-utils  //可以使用brctl相关命令
[root@localhost ~]# brctl show
bridge name     bridge id               STP enabled     interfaces
docker0         8000.0242891b26f5       no

创建一个网桥：

[root@localhost ~]# brctl addbr mynet0
[root@localhost ~]# ip link set mynet0 up
[root@localhost ~]# brctl show
bridge name     bridge id               STP enabled     interfaces
docker0         8000.0242891b26f5       no
mynet0          8000.000000000000       no

重新建立一个centos1，端口为空，进去可以看到只有一个lo的端口：

[root@localhost ~]# docker run --net=none --name centos1 -dt centos
[root@localhost ~]# docker ps -a
CONTAINER ID   IMAGE     COMMAND       CREATED         STATUS         PORTS     NAMES
aa4c6dda4f2b   centos    "/bin/bash"   4 seconds ago   Up 3 seconds             centos1
[root@localhost ~]# docker exec -it aa4c bash
[root@aa4c6dda4f2b /]# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever

手动添加veth命名空间：

[root@localhost ~]# nspid=$(docker inspect -f '{{.State.Pid}}' centos1)   //获得该容器的PID信息
[root@localhost ~]# mkdir -p /var/run/netns
[root@localhost ~]# ln -s /proc/${nspid}/ns/net /var/run/netns/${nspid}

创建veth端口并与mynet0绑定，在宿主机上只能看到veth端口A，并设置mynet0的IP地址：

[root@localhost ~]# ip link add A type veth peer name B
[root@localhost ~]# ip link set A up
[root@localhost ~]# brctl addif mynet0 A
[root@localhost ~]# ip addr add 172.16.0.1/24 dev mynet0
[root@localhost ~]# ifconfig mynet0
mynet0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.16.0.1  netmask 255.255.255.0  broadcast 0.0.0.0
        inet6 fe80::9098:7aff:fe72:a341  prefixlen 64  scopeid 0x20<link>
        ether 92:98:7a:72:a3:41  txqueuelen 1000  (Ethernet)
        RX packets 4  bytes 224 (224.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 16  bytes 1264 (1.2 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
[root@localhost ~]# brctl show
bridge name     bridge id               STP enabled     interfaces
docker0         8000.0242891b26f5       no
mynet0          8000.92987a72a341       no              A

把veth端口B放到容器centos1中，并设置IP，可以直接进去配置，也可以通过下面方式配置：

[root@localhost netns]# ip link set B netns $nspid
[root@localhost netns]# ip netns exec $nspid ip link set dev B name eth0
[root@localhost netns]# ip netns exec $nspid ip link set eth0 up
[root@localhost netns]# ip netns exec $nspid ip addr add 172.16.0.14/24 dev eth0 
[root@localhost netns]# ip netns exec $nspid ip route add default via 172.16.0.1
[root@localhost netns]# docker exec -it aa4c bash
[root@aa4c6dda4f2b /]# ifconfig
bash: ifconfig: command not found
[root@aa4c6dda4f2b /]# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
30: eth0@if31: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 86:da:99:83:2e:85 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 172.16.0.14/24 scope global eth0
       valid_lft forever preferred_lft forever

从容器中验证，可以ping同网关mynet0：

[root@aa4c6dda4f2b /]# ping 172.16.0.1
PING 172.16.0.1 (172.16.0.1) 56(84) bytes of data.
64 bytes from 172.16.0.1: icmp_seq=1 ttl=64 time=0.040 ms
64 bytes from 172.16.0.1: icmp_seq=2 ttl=64 time=0.102 ms
^C
--- 172.16.0.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1007ms
rtt min/avg/max/mdev = 0.040/0.071/0.102/0.031 ms

手动建立第二个容器并使其互通

通过同样的方法建立centos2，并设置veth端口并成功绑定网桥，但发现两个容器不同：

[root@localhost ~]# docker ps -a
CONTAINER ID   IMAGE     COMMAND       CREATED          STATUS          PORTS     NAMES
6708e3f9e8ba   centos    "/bin/bash"   18 minutes ago   Up 18 minutes             centos2
aa4c6dda4f2b   centos    "/bin/bash"   2 hours ago      Up 2 hours                centos1
bridge name     bridge id               STP enabled     interfaces
docker0         8000.0242891b26f5       no
mynet0          8000.92987a72a341       no              A
                                                        A1

[root@localhost ~]# docker exec -it 6708 bash
[root@6708e3f9e8ba /]# 
[root@6708e3f9e8ba /]# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
32: eth0@if33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether aa:6c:fa:00:ab:45 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 172.16.0.15/24 scope global eth0
       valid_lft forever preferred_lft forever
[root@6708e3f9e8ba /]# ping 172.16.0.1
PING 172.16.0.1 (172.16.0.1) 56(84) bytes of data.
64 bytes from 172.16.0.1: icmp_seq=1 ttl=64 time=0.041 ms
^C
--- 172.16.0.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.041/0.041/0.041/0.000 ms
[root@6708e3f9e8ba /]# 
[root@6708e3f9e8ba /]# ping 172.16.0.14
PING 172.16.0.14 (172.16.0.14) 56(84) bytes of data.
^C
--- 172.16.0.14 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1061ms

[root@localhost ~]# docker exec -it aa4c bash
[root@aa4c6dda4f2b /]# 
[root@aa4c6dda4f2b /]# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
30: eth0@if31: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 86:da:99:83:2e:85 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 172.16.0.14/24 scope global eth0
       valid_lft forever preferred_lft forever
[root@aa4c6dda4f2b /]#      
[root@aa4c6dda4f2b /]# ping 172.16.0.1
PING 172.16.0.1 (172.16.0.1) 56(84) bytes of data.
64 bytes from 172.16.0.1: icmp_seq=1 ttl=64 time=0.044 ms
64 bytes from 172.16.0.1: icmp_seq=2 ttl=64 time=0.061 ms
^C
--- 172.16.0.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1055ms
rtt min/avg/max/mdev = 0.044/0.052/0.061/0.011 ms
[root@aa4c6dda4f2b /]# 
[root@aa4c6dda4f2b /]# ping 172.16.0.15
PING 172.16.0.15 (172.16.0.15) 56(84) bytes of data.
^C
--- 172.16.0.15 ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 3075ms

通过tcpdump抓包可以发现，只有request，没有respond，但ARP是回复的：

[root@localhost ~]# tcpdump -i A
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on A, link-type EN10MB (Ethernet), capture size 262144 bytes
08:14:39.970819 IP 172.16.0.14 > 172.16.0.15: ICMP echo request, id 13, seq 1, length 64
08:14:41.007793 IP 172.16.0.14 > 172.16.0.15: ICMP echo request, id 13, seq 2, length 64
08:14:42.031822 IP 172.16.0.14 > 172.16.0.15: ICMP echo request, id 13, seq 3, length 64
08:14:43.055822 IP 172.16.0.14 > 172.16.0.15: ICMP echo request, id 13, seq 4, length 64
08:14:44.079807 IP 172.16.0.14 > 172.16.0.15: ICMP echo request, id 13, seq 5, length 64
08:14:45.039768 ARP, Request who-has 172.16.0.15 tell 172.16.0.14, length 28
08:14:45.039805 ARP, Reply 172.16.0.15 is-at aa:6c:fa:00:ab:45 (oui Unknown), length 28
08:14:45.103818 IP 172.16.0.14 > 172.16.0.15: ICMP echo request, id 13, seq 6, length 64
08:14:46.127786 IP 172.16.0.14 > 172.16.0.15: ICMP echo request, id 13, seq 7, length 64
08:14:47.151787 IP 172.16.0.14 > 172.16.0.15: ICMP echo request, id 13, seq 8, length 64
08:14:48.175795 IP 172.16.0.14 > 172.16.0.15: ICMP echo request, id 13, seq 9, length 64

查看网桥的mac信息，是存在的：

[root@localhost ~]# brctl showmacs mynet0
port no mac addr                is local?       ageing timer
  1     86:da:99:83:2e:85       no                14.50
  1     92:98:7a:72:a3:41       yes                0.00
  1     92:98:7a:72:a3:41       yes                0.00
  2     aa:6c:fa:00:ab:45       no                14.50
  2     fa:91:f1:44:36:a7       yes                0.00
  2     fa:91:f1:44:36:a7       yes                0.00

也可以通过“bridge fdb”（MAC学习表，转发数据库）查看信息：

[root@localhost ~]# bridge fdb |grep mynet0
33:33:00:00:00:01 dev mynet0 self permanent
01:00:5e:00:00:6a dev mynet0 self permanent
33:33:00:00:00:6a dev mynet0 self permanent
01:00:5e:00:00:01 dev mynet0 self permanent
33:33:ff:72:a3:41 dev mynet0 self permanent
86:da:99:83:2e:85 dev A master mynet0 
92:98:7a:72:a3:41 dev A vlan 1 master mynet0 permanent
92:98:7a:72:a3:41 dev A master mynet0 permanent
aa:6c:fa:00:ab:45 dev A1 master mynet0 
fa:91:f1:44:36:a7 dev A1 vlan 1 master mynet0 permanent
fa:91:f1:44:36:a7 dev A1 master mynet0 permanent

解决两个容器不通的问题

经过与朋友讨论后，发现问题是由于iptables默认只允许了docker0互通，新建的mynet0的网桥没有被允许，如下我ping了100个包，drop了100个，添加策略后，恢复正常；另外最开始我想直接关闭iptables，但看上去没法关闭，只能删除规则：

-L：默认列出filter表，iptables有5个表，优先级是raw->mangle->nat->filter，如果想看其他表，通过”-t xxx“来显示；
-n：以IP地址和端口的方式输出，如果不用，会议anywhere显示，否则以0.0.0.0/0显示；
–line-number：显示行号；

[root@localhost ~]# iptables -L -nv --line-number
Chain INPUT (policy ACCEPT 972 packets, 277K bytes)
num   pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy DROP 100 packets, 8400 bytes)
num   pkts bytes target     prot opt in     out     source               destination         
1      100  8400 DOCKER-USER  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
2      100  8400 DOCKER-ISOLATION-STAGE-1  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
3        0     0 ACCEPT     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
4        0     0 DOCKER     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0           
5        0     0 ACCEPT     all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0           
6        0     0 ACCEPT     all  --  docker0 docker0  0.0.0.0/0            0.0.0.0/0           

Chain OUTPUT (policy ACCEPT 1020 packets, 351K bytes)
num   pkts bytes target     prot opt in     out     source               destination         

Chain DOCKER (1 references)
num   pkts bytes target     prot opt in     out     source               destination         

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 DOCKER-ISOLATION-STAGE-2  all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0           
2      100  8400 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain DOCKER-ISOLATION-STAGE-2 (1 references)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 DROP       all  --  *      docker0  0.0.0.0/0            0.0.0.0/0           
2        0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain DOCKER-USER (1 references)
num   pkts bytes target     prot opt in     out     source               destination         
1      100  8400 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

下面是增加策略后，又ping了12个报文后的信息：

[root@localhost ~]# iptables -A FORWARD -i mynet0 ! -o mynet0 -j ACCEPT
[root@localhost ~]# iptables -A FORWARD -i mynet0 -o mynet0 -j ACCEPT
[root@localhost ~]# iptables -L -nv --line-number
Chain INPUT (policy ACCEPT 82 packets, 5352 bytes)
num   pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy DROP 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination         
1      112  9408 DOCKER-USER  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
2      112  9408 DOCKER-ISOLATION-STAGE-1  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
3        0     0 ACCEPT     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
4        0     0 DOCKER     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0           
5        0     0 ACCEPT     all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0           
6        0     0 ACCEPT     all  --  docker0 docker0  0.0.0.0/0            0.0.0.0/0           
7        0     0 ACCEPT     all  --  mynet0 !mynet0  0.0.0.0/0            0.0.0.0/0           
8       12  1008 ACCEPT     all  --  mynet0 mynet0  0.0.0.0/0            0.0.0.0/0           

Chain OUTPUT (policy ACCEPT 74 packets, 8840 bytes)
num   pkts bytes target     prot opt in     out     source               destination         

Chain DOCKER (1 references)
num   pkts bytes target     prot opt in     out     source               destination         

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 DOCKER-ISOLATION-STAGE-2  all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0           
2      112  9408 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain DOCKER-ISOLATION-STAGE-2 (1 references)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 DROP       all  --  *      docker0  0.0.0.0/0            0.0.0.0/0           
2        0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain DOCKER-USER (1 references)
num   pkts bytes target     prot opt in     out     source               destination         
1      112  9408 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

下面是常用iptables命令：

# iptables -D FORWARD 7 // 删除Forward chain中的第7条规则
# iptables -A OUTPUT -d x.x.x.x -j DROP // 在Output中添加drop规则
# iptables-save > test.iptables.bak // 备份iptables的规则
# iptables -F // 删除所有规则
# iptables-restore test.iptables.bak // 恢复iptables的规则

Reference

本文出自 Frank's Blog

本文链接：Manual set up two containers by “docker run”
版权声明：本文为原创文章，仅代表个人观点，版权归 Frank Zhao 所有，转载时请注明本文出处及文章链接

Linux, 学习笔记 Docker,Linux

你可以留言,或者trackback 从你的网站

K8s – Manual Deployment«

»K8s – Core Resource Management Method

留言哦

返回顶端

『无题』

I persist, therefore I am!!!

Memo

Perseverance sometimes equals genius!
There are only two creatures in the world who can surmount the pyramid:

—–the eagle and the snail.
Useful Link

1. IANA - AFI
2. IANA - SAFI
3. IANA - BGP Ex Comm
4. RPM pbone.net
5. Linux Packages Search
Useful Toots

Linux/http server:

python -m SimpleHTTPServer <port>

python -m http.server <port>

PS: only one thread, if require more threads, can replace by “caddy”.

Win10/http server:

http://www.rejetto.com/hfs/

Win10/md5:

certutil -hashfile .\xxxx MD5
分类目录
- CCDE (16)
- CCIE SEC (4)
- CCIE SP (55)
- CCIE VOICE (3)
- Classical CASE (26)
- Linux (73)
- Mobile Phone (4)
- Programmer (44)
- Solution (2)
- Uncategorized (5)
- Windows常用技术 (10)
- 似水流年 (17)
- 学习笔记 (186)
- 常用资料 (44)
- 网络技术 (37)
近期文章
其他操作

Manual set up two containers by “docker run”

容器的网络类型

手动创建容器并绑定自定义的mynet0

手动建立第二个容器并使其互通

解决两个容器不通的问题

Reference

留言哦

『无题』

Memo

Useful Link

Useful Toots

分类目录

近期文章

其他操作