Manual set up two containers by “docker run”
安装docker步骤此处忽略,详细可以看之前的文章;
容器的网络类型
默认docker三种网络:
[[email protected] ~]# docker network ls
NETWORK ID NAME DRIVER SCOPE
0dc7621fa6c0 bridge bridge local
09d08085f98c host host local
8da76deb9eb9 none null local
先随便开一个centos的docker,“-t” 指定一个伪终端,保持docker可以打开,否则会出现“Exited (0)”的问题:
[[email protected] ~]# docker run --name centos -dt centos
查看容器的网络类型,可以看到当创建容器不指定网络类型,默认会绑定到bridge上:
[[email protected] ~]# docker inspect 355a |grep "Networks" -A 100
"Networks": {
"bridge": {
"IPAMConfig": null,
"Links": null,
"Aliases": null,
"NetworkID": "0dc7621fa6c0e40a1129104ed94deac543cf82ce44ab79fdf7ed38b9975f9a78",
"EndpointID": "2dae4a431efd76066081424e47a7e6bc53e4102b14bf36bbdfda34492efd6369",
"Gateway": "172.17.0.1",
"IPAddress": "172.17.0.2",
"IPPrefixLen": 16,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:ac:11:00:02",
"DriverOpts": null
手动创建容器并绑定自定义的mynet0
安装bridge-utils,使用brctl命令:
[[email protected] ~]# yum install bridge-utils //可以使用brctl相关命令
[[email protected] ~]# brctl show
bridge name bridge id STP enabled interfaces
docker0 8000.0242891b26f5 no
创建一个网桥:
[[email protected] ~]# brctl addbr mynet0
[[email protected] ~]# ip link set mynet0 up
[[email protected] ~]# brctl show
bridge name bridge id STP enabled interfaces
docker0 8000.0242891b26f5 no
mynet0 8000.000000000000 no
重新建立一个centos1,端口为空,进去可以看到只有一个lo的端口:
[[email protected] ~]# docker run --net=none --name centos1 -dt centos
[[email protected] ~]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
aa4c6dda4f2b centos "/bin/bash" 4 seconds ago Up 3 seconds centos1
[[email protected] ~]# docker exec -it aa4c bash
[[email protected] /]# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
手动添加veth命名空间:
[[email protected] ~]# nspid=$(docker inspect -f '{{.State.Pid}}' centos1) //获得该容器的PID信息
[[email protected] ~]# mkdir -p /var/run/netns
[[email protected] ~]# ln -s /proc/${nspid}/ns/net /var/run/netns/${nspid}
创建veth端口并与mynet0绑定,在宿主机上只能看到veth端口A,并设置mynet0的IP地址:
[[email protected] ~]# ip link add A type veth peer name B
[[email protected] ~]# ip link set A up
[[email protected] ~]# brctl addif mynet0 A
[[email protected] ~]# ip addr add 172.16.0.1/24 dev mynet0
[[email protected] ~]# ifconfig mynet0
mynet0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.16.0.1 netmask 255.255.255.0 broadcast 0.0.0.0
inet6 fe80::9098:7aff:fe72:a341 prefixlen 64 scopeid 0x20<link>
ether 92:98:7a:72:a3:41 txqueuelen 1000 (Ethernet)
RX packets 4 bytes 224 (224.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 16 bytes 1264 (1.2 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[[email protected] ~]# brctl show
bridge name bridge id STP enabled interfaces
docker0 8000.0242891b26f5 no
mynet0 8000.92987a72a341 no A
把veth端口B放到容器centos1中,并设置IP,可以直接进去配置,也可以通过下面方式配置:
[[email protected] netns]# ip link set B netns $nspid
[[email protected] netns]# ip netns exec $nspid ip link set dev B name eth0
[[email protected] netns]# ip netns exec $nspid ip link set eth0 up
[[email protected] netns]# ip netns exec $nspid ip addr add 172.16.0.14/24 dev eth0
[[email protected] netns]# ip netns exec $nspid ip route add default via 172.16.0.1
[[email protected] netns]# docker exec -it aa4c bash
[[email protected] /]# ifconfig
bash: ifconfig: command not found
[[email protected] /]# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
30: [email protected]: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 86:da:99:83:2e:85 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 172.16.0.14/24 scope global eth0
valid_lft forever preferred_lft forever
从容器中验证,可以ping同网关mynet0:
[[email protected] /]# ping 172.16.0.1
PING 172.16.0.1 (172.16.0.1) 56(84) bytes of data.
64 bytes from 172.16.0.1: icmp_seq=1 ttl=64 time=0.040 ms
64 bytes from 172.16.0.1: icmp_seq=2 ttl=64 time=0.102 ms
^C
--- 172.16.0.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1007ms
rtt min/avg/max/mdev = 0.040/0.071/0.102/0.031 ms
手动建立第二个容器并使其互通
通过同样的方法建立centos2,并设置veth端口并成功绑定网桥,但发现两个容器不同:
[[email protected] ~]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6708e3f9e8ba centos "/bin/bash" 18 minutes ago Up 18 minutes centos2
aa4c6dda4f2b centos "/bin/bash" 2 hours ago Up 2 hours centos1
bridge name bridge id STP enabled interfaces
docker0 8000.0242891b26f5 no
mynet0 8000.92987a72a341 no A
A1
[[email protected] ~]# docker exec -it 6708 bash
[[email protected] /]#
[[email protected] /]# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
32: [email protected]: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether aa:6c:fa:00:ab:45 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 172.16.0.15/24 scope global eth0
valid_lft forever preferred_lft forever
[[email protected] /]# ping 172.16.0.1
PING 172.16.0.1 (172.16.0.1) 56(84) bytes of data.
64 bytes from 172.16.0.1: icmp_seq=1 ttl=64 time=0.041 ms
^C
--- 172.16.0.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.041/0.041/0.041/0.000 ms
[[email protected] /]#
[[email protected] /]# ping 172.16.0.14
PING 172.16.0.14 (172.16.0.14) 56(84) bytes of data.
^C
--- 172.16.0.14 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1061ms
[[email protected] ~]# docker exec -it aa4c bash
[[email protected] /]#
[[email protected] /]# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
30: [email protected]: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 86:da:99:83:2e:85 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 172.16.0.14/24 scope global eth0
valid_lft forever preferred_lft forever
[[email protected] /]#
[[email protected] /]# ping 172.16.0.1
PING 172.16.0.1 (172.16.0.1) 56(84) bytes of data.
64 bytes from 172.16.0.1: icmp_seq=1 ttl=64 time=0.044 ms
64 bytes from 172.16.0.1: icmp_seq=2 ttl=64 time=0.061 ms
^C
--- 172.16.0.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1055ms
rtt min/avg/max/mdev = 0.044/0.052/0.061/0.011 ms
[[email protected] /]#
[[email protected] /]# ping 172.16.0.15
PING 172.16.0.15 (172.16.0.15) 56(84) bytes of data.
^C
--- 172.16.0.15 ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 3075ms
通过tcpdump抓包可以发现,只有request,没有respond,但ARP是回复的:
[[email protected] ~]# tcpdump -i A
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on A, link-type EN10MB (Ethernet), capture size 262144 bytes
08:14:39.970819 IP 172.16.0.14 > 172.16.0.15: ICMP echo request, id 13, seq 1, length 64
08:14:41.007793 IP 172.16.0.14 > 172.16.0.15: ICMP echo request, id 13, seq 2, length 64
08:14:42.031822 IP 172.16.0.14 > 172.16.0.15: ICMP echo request, id 13, seq 3, length 64
08:14:43.055822 IP 172.16.0.14 > 172.16.0.15: ICMP echo request, id 13, seq 4, length 64
08:14:44.079807 IP 172.16.0.14 > 172.16.0.15: ICMP echo request, id 13, seq 5, length 64
08:14:45.039768 ARP, Request who-has 172.16.0.15 tell 172.16.0.14, length 28
08:14:45.039805 ARP, Reply 172.16.0.15 is-at aa:6c:fa:00:ab:45 (oui Unknown), length 28
08:14:45.103818 IP 172.16.0.14 > 172.16.0.15: ICMP echo request, id 13, seq 6, length 64
08:14:46.127786 IP 172.16.0.14 > 172.16.0.15: ICMP echo request, id 13, seq 7, length 64
08:14:47.151787 IP 172.16.0.14 > 172.16.0.15: ICMP echo request, id 13, seq 8, length 64
08:14:48.175795 IP 172.16.0.14 > 172.16.0.15: ICMP echo request, id 13, seq 9, length 64
查看网桥的mac信息,是存在的:
[[email protected] ~]# brctl showmacs mynet0
port no mac addr is local? ageing timer
1 86:da:99:83:2e:85 no 14.50
1 92:98:7a:72:a3:41 yes 0.00
1 92:98:7a:72:a3:41 yes 0.00
2 aa:6c:fa:00:ab:45 no 14.50
2 fa:91:f1:44:36:a7 yes 0.00
2 fa:91:f1:44:36:a7 yes 0.00
也可以通过“bridge fdb”(MAC学习表,转发数据库)查看信息:
[[email protected] ~]# bridge fdb |grep mynet0
33:33:00:00:00:01 dev mynet0 self permanent
01:00:5e:00:00:6a dev mynet0 self permanent
33:33:00:00:00:6a dev mynet0 self permanent
01:00:5e:00:00:01 dev mynet0 self permanent
33:33:ff:72:a3:41 dev mynet0 self permanent
86:da:99:83:2e:85 dev A master mynet0
92:98:7a:72:a3:41 dev A vlan 1 master mynet0 permanent
92:98:7a:72:a3:41 dev A master mynet0 permanent
aa:6c:fa:00:ab:45 dev A1 master mynet0
fa:91:f1:44:36:a7 dev A1 vlan 1 master mynet0 permanent
fa:91:f1:44:36:a7 dev A1 master mynet0 permanent
解决两个容器不通的问题
经过与朋友讨论后,发现问题是由于iptables默认只允许了docker0互通,新建的mynet0的网桥没有被允许,如下我ping了100个包,drop了100个,添加策略后,恢复正常;另外最开始我想直接关闭iptables,但看上去没法关闭,只能删除规则:
- -L:默认列出filter表,iptables有5个表,优先级是raw->mangle->nat->filter,如果想看其他表,通过”-t xxx“来显示;
- -n:以IP地址和端口的方式输出,如果不用,会议anywhere显示,否则以0.0.0.0/0显示;
- –line-number:显示行号;
[[email protected] ~]# iptables -L -nv --line-number
Chain INPUT (policy ACCEPT 972 packets, 277K bytes)
num pkts bytes target prot opt in out source destination
Chain FORWARD (policy DROP 100 packets, 8400 bytes)
num pkts bytes target prot opt in out source destination
1 100 8400 DOCKER-USER all -- * * 0.0.0.0/0 0.0.0.0/0
2 100 8400 DOCKER-ISOLATION-STAGE-1 all -- * * 0.0.0.0/0 0.0.0.0/0
3 0 0 ACCEPT all -- * docker0 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
4 0 0 DOCKER all -- * docker0 0.0.0.0/0 0.0.0.0/0
5 0 0 ACCEPT all -- docker0 !docker0 0.0.0.0/0 0.0.0.0/0
6 0 0 ACCEPT all -- docker0 docker0 0.0.0.0/0 0.0.0.0/0
Chain OUTPUT (policy ACCEPT 1020 packets, 351K bytes)
num pkts bytes target prot opt in out source destination
Chain DOCKER (1 references)
num pkts bytes target prot opt in out source destination
Chain DOCKER-ISOLATION-STAGE-1 (1 references)
num pkts bytes target prot opt in out source destination
1 0 0 DOCKER-ISOLATION-STAGE-2 all -- docker0 !docker0 0.0.0.0/0 0.0.0.0/0
2 100 8400 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0
Chain DOCKER-ISOLATION-STAGE-2 (1 references)
num pkts bytes target prot opt in out source destination
1 0 0 DROP all -- * docker0 0.0.0.0/0 0.0.0.0/0
2 0 0 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0
Chain DOCKER-USER (1 references)
num pkts bytes target prot opt in out source destination
1 100 8400 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0
下面是增加策略后,又ping了12个报文后的信息:
[[email protected] ~]# iptables -A FORWARD -i mynet0 ! -o mynet0 -j ACCEPT
[[email protected] ~]# iptables -A FORWARD -i mynet0 -o mynet0 -j ACCEPT
[[email protected] ~]# iptables -L -nv --line-number
Chain INPUT (policy ACCEPT 82 packets, 5352 bytes)
num pkts bytes target prot opt in out source destination
Chain FORWARD (policy DROP 0 packets, 0 bytes)
num pkts bytes target prot opt in out source destination
1 112 9408 DOCKER-USER all -- * * 0.0.0.0/0 0.0.0.0/0
2 112 9408 DOCKER-ISOLATION-STAGE-1 all -- * * 0.0.0.0/0 0.0.0.0/0
3 0 0 ACCEPT all -- * docker0 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
4 0 0 DOCKER all -- * docker0 0.0.0.0/0 0.0.0.0/0
5 0 0 ACCEPT all -- docker0 !docker0 0.0.0.0/0 0.0.0.0/0
6 0 0 ACCEPT all -- docker0 docker0 0.0.0.0/0 0.0.0.0/0
7 0 0 ACCEPT all -- mynet0 !mynet0 0.0.0.0/0 0.0.0.0/0
8 12 1008 ACCEPT all -- mynet0 mynet0 0.0.0.0/0 0.0.0.0/0
Chain OUTPUT (policy ACCEPT 74 packets, 8840 bytes)
num pkts bytes target prot opt in out source destination
Chain DOCKER (1 references)
num pkts bytes target prot opt in out source destination
Chain DOCKER-ISOLATION-STAGE-1 (1 references)
num pkts bytes target prot opt in out source destination
1 0 0 DOCKER-ISOLATION-STAGE-2 all -- docker0 !docker0 0.0.0.0/0 0.0.0.0/0
2 112 9408 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0
Chain DOCKER-ISOLATION-STAGE-2 (1 references)
num pkts bytes target prot opt in out source destination
1 0 0 DROP all -- * docker0 0.0.0.0/0 0.0.0.0/0
2 0 0 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0
Chain DOCKER-USER (1 references)
num pkts bytes target prot opt in out source destination
1 112 9408 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0
下面是常用iptables命令:
- # iptables -D FORWARD 7 // 删除Forward chain中的第7条规则
- # iptables -A OUTPUT -d x.x.x.x -j DROP // 在Output中添加drop规则
- # iptables-save > test.iptables.bak // 备份iptables的规则
- # iptables -F // 删除所有规则
- # iptables-restore test.iptables.bak // 恢复iptables的规则
Reference
版权声明:
本文链接:Manual set up two containers by “docker run”
版权声明:本文为原创文章,仅代表个人观点,版权归 Frank Zhao 所有,转载时请注明本文出处及文章链接