Manual Deploy K8s

终于尝试手动部署K8s了,在此感谢朋友分享的”老男孩“教学视频👍🏻。以及朋友分享的实验环境(基于EVE-NG);持续更新直到完成部署为止;

更新说明:

  • 2022-01-17:更新拓扑信息;
  • 2022-01-19:添加基础配置;
  • 2022-01-23:添加证书配置及Bind9部署;
  • 2022-01-25:添加Docker及私有仓库Harbor相关部署配置;

Topology

基础配置

端口配置

只展示frank0-11,其他Server等同;

root@frank0-11:~# more /etc/netplan/00-installer-config.yaml
# This is the network config written by 'subiquity'
network:
  ethernets:
    ens3:
      dhcp4: false
      addresses: [172.20.0.11/24]
      gateway4: 172.20.0.1
      nameservers:
        addresses: [114.114.114.114]
  version: 2
root@frank0-12:~# netplan apply

修改本地更新源

只展示frank0-11,其他Server等同;阿里源没有20.10的,20.04的focal源也可以用,在vim可以用下面方法直接替换:

root@frank0-11:~# cp /etc/apt/sources.list /etc/apt/sources.list.bak
root@frank0-11:~# lsb_release -a
root@frank0-11:~# vim /etc/apt/sources.list
:%s#http://cn.archive.ubuntu.com/ubuntu#http://mirrors.aliyun.com/ubuntu/#g
:%s#groovy#focal#g
root@frank0-11:~# apt update

组件版本信息

组件命令版本
bind9# named -v9.16.1-Ubuntu (Stable Release)
cfssl# cfssl version1.2.0
docker# docker -v20.10.8, build 3967b7d
docker-compose# docker-compose -v1.25.0
harbor1.10.10
nginx# nginx -vnginx/1.18.0 (Ubuntu)

配置Bind9

Bind9是一个开源的DNS,为什么要专门搭建一个DNS Server,而不是直接在host里写呢?这是由于K8s后面的互访都是通过域名调用的(docker之间),这些没法在host里写,所以需要一个专门的DNS Server,这个DNS Server将部署在frank0-11上,具体部署步骤如下,更详细的DNS信息,可以参考下面两篇文章:

版本信息

安装Bind9

默认带的bind9根要安装的有冲突,所以先移除,再安装:

root@frank0-11:~# apt remove bind9-libs
root@frank0-11:~# apt install bind9*

配置全局配置文件

从主配置文件中可以看到DNS主要有3个需要配置的地方:

root@frank0-11:~# more /etc/bind/named.conf

include "/etc/bind/named.conf.options";          //全局配置文件
include "/etc/bind/named.conf.local";            //区域配置文件
include "/etc/bind/named.conf.default-zones";    //设置zone

配置全局配置文件,注意这里有些配置跟CentOS可能会有些不同,例如目录:

root@frank0-11:~# more /etc/bind/named.conf.options
options {
        listen-on port 53 {172.20.0.11; };
        directory       "/var/cache/bind";
        allow-query     { any; };
        forwarders      { 114.114.114.114; };
        recursion       yes;
        dnssec-validation no;
};

配置区域配置文件

在这里定义两个domain,一个是主机域 host.com,另一个是业务域 frank.com:

root@frank0-11:~# more /etc/bind/named.conf.local

zone "host.com" {
        type master;
        file "/etc/bind/db.host.com";
        allow-update { 172.20.0.11; };
};

zone "frank.com" {
        type master;
        file "/etc/bind/db.frank.com";
        allow-update { 172.20.0.11; };
};

下面是主机域配置文件:

root@frank0-11:~# more /etc/bind/db.host.com 
;
; BIND data file for local loopback interface
;
$ORIGIN host.com.
$TTL    600
@       IN      SOA     dns.host.com.   dnsadmin.host.com. (
                        2022010308      ; Serial
                        10800           ; Refresh
                        900             ; Retry
                        604800          ; Expire
                        86400           ; Negative Cache TTL
                        )
                        NS      dns.host.com.
$TTL    60 ; 1 minute
dns             A       172.20.0.11
frank0-11       A       172.20.0.11
frank0-12       A       172.20.0.12
frank0-13       A       172.20.0.13
frank0-14       A       172.20.0.14
frank0-15       A       172.20.0.15

下面是业务域配置文件:

root@frank0-11:~# more db.frank.com 
more: cannot open db.frank.com: No such file or directory
root@frank0-11:~# more /etc/bind/db.frank.com 
;
; BIND data file for local loopback interface
;
$ORIGIN frank.com.
$TTL    600
@       IN      SOA     dns.frank.com.  dnsadmin.frank.com. (
                        2022011701      ; Serial
                        10800           ; Refresh
                        900             ; Retry
                        604800          ; Expire
                        86400           ; Negative Cache TTL
                        )
                        NS      dns.frank.com.
$TTL    60 ; 1 minute
dns             A       172.20.0.11
test            A       172.20.0.11

验证配置文件是否正确:

root@frank0-11:/etc/bind# named-checkzone frank.com /etc/bind/frank.com
zone frank.com/IN: loaded serial 2021123001
OK
root@frank0-11:/etc/bind# 
root@frank0-11:/etc/bind# named-checkzone host.com /etc/bind/host.com
zone host.com/IN: loaded serial 2021123001
OK

重启bind9后生效,注意我这里没有特意配置rndc(如key啥的),这个组件的作用是更新DNS配置不需要重启,但需要变更配置文件中的序列号。由于实验环境,每次该配置后直接重启就ok了,重启后可以通过下面验证:

root@frank0-11:~# systemctl restart bind9
root@frank0-11:~# 
root@frank0-11:~# dig -t A test.frank.com @172.20.0.11 +short
172.20.0.11
root@frank0-11:~# dig -t A frank0-15.host.com @172.20.0.11 +short
172.20.0.15

注意如果此时用nslookup,会发现根据hostname找不到对应的IP,必须写全域名(FQDN),如下:

root@frank0-11:~# nslookup test
Server:         127.0.0.53
Address:        127.0.0.53#53

** server can't find test: SERVFAIL

root@frank0-11:~# nslookup frank0-15
Server:         127.0.0.53
Address:        127.0.0.53#53

** server can't find frank0-15: SERVFAIL

root@frank0-11:~# nslookup test.frank.com
Server:         127.0.0.53
Address:        127.0.0.53#53

Non-authoritative answer:
Name:   test.frank.com
Address: 172.20.0.11

root@frank0-11:~# nslookup frank0-15.host.com
Server:         127.0.0.53
Address:        127.0.0.53#53

Non-authoritative answer:
Name:   frank0-15.host.com
Address: 172.20.0.15

这是因为需要先设置Search Domain才可以省略domain信息,而且Search Domain只能设置1个,如果设置2个,那么先解析的就成为默认的Search Domain,另一个就没法解析了,如下所示,注意常规情况下Search Domain只设置主机域

root@frank0-11:~# more /etc/resolv.conf 

nameserver 127.0.0.53
options edns0 trust-ad
search host.com
root@frank0-11:~# 
root@frank0-11:~# nslookup frank0-15
Server:         127.0.0.53
Address:        127.0.0.53#53

Non-authoritative answer:
Name:   frank0-15.host.com
Address: 172.20.0.15

不过Ubuntu改resolv.conf无法持续保持,一会就会变回去,需要在配置IP的地方强行加Search Domain,如下:

root@frank0-11:/home/lab# more /etc/netplan/00-installer-config.yaml 
......
      gateway4: 172.20.0.1
      nameservers:
        addresses: [172.20.0.11]
        search: [host.com]
  version: 2

修改其他主机的DNS

配置12,13,14,15主机的DNS,并指向11;至于为什么在nslookup中显示的Server不是11而是本地,查了下好像ubuntu中就是会指向本地缓存,而不是上游的DNS Server,所以以”systemd-resolve –status“为主;

root@frank0-13:~# more /etc/netplan/00-installer-config.yaml 
# This is the network config written by 'subiquity'
network:
  ethernets:
    ens3:
      dhcp4: false      
      addresses: [172.20.0.13/24]
      #- 100.64.0.36/24
      gateway4: 172.20.0.1
      nameservers:
        addresses: [172.20.0.11]
  version: 2
root@frank0-13:~# netplan apply 
root@frank0-13:~# systemd-resolve --status |grep Server
         DNS Servers: 172.20.0.11    
root@frank0-13:~# nslookup harbor.frank.com
Server:         127.0.0.53
Address:        127.0.0.53#53

Non-authoritative answer:
Name:   harbor.frank.com
Address: 172.20.0.15

配置证书

版本信息

运维主机安装证书工具

国内网络环境不好,多试几次就可以了:

root@frank0-15:~# wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 -O /usr/bin/cfssl
root@frank0-15:~# wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 -O /usr/bin/cfssl-json
root@frank0-15:~# wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64 -O /usr/bin/cfssl-certinfo
root@frank0-15:~# chmod +x /usr/bin/cfssl*
root@frank0-15:~# which cfssl
/usr/bin/cfssl
root@frank0-15:~# which cfssl-json
/usr/bin/cfssl-json
root@frank0-15:~# which cfssl-certinfo
/usr/bin/cfssl-certinfo

创建CA证书请求(CSR)的Jason文件

这里的证书主要指HTTPS证书,对于做网站的博主来说,会常听到的免费签发HTTPS证书,那么这种免费的,收费的和自签发根证书之间有什么区别呢?我简单做了下对比,这样比较容易理解;更详细的信息可以自行搜索:

免费证书收费证书自签发根证书
申请方式域名/IP/服务器对应即可向第三方机构申请免费证书,自动审批需要申请人或机构提交相关证明,人工审核自由签发,不需要任何审批
有效时长3个月 – 1年1年自定义
网站可信
数据传输保护
root@frank0-15:/opt# cd certs/
root@frank0-15:/opt/certs# more ca-csr.json
{
        "CN":   "Frank",                   #机构名称
        "hosts": [
        ],
        "key": {
                "algo": "rsa",
                "size": 2048
        },
        "name": [
                {
                        "C": "CN",         #国家
                        "ST": "ShangHai",  #州
                        "L": "ShangHai",   #城市
                        "O": "FZ",         #组织
                        "OU": "arch"       #部门
                }
        ],
        "ca": {
                "expiry": "8000h"          #有效时间,自签证书可以随便写,自签根证书是无法信任的,但在内网使用主要用于加密传输;
        }
}

签根证书

root@frank0-15:/opt/certs# cfssl -h
Usage:
Available commands:
        genkey
        ocspserve
        certinfo
        gencert
        ocspsign
        selfsign
        print-defaults
        bundle
        sign
        ocsprefresh
        info
        revoke
        serve
        version
        gencrl
        ocspdump
        scan
Top-level flags:
  -allow_verification_with_non_compliant_keys
        Allow a SignatureVerifier to use keys which are technically non-compliant with RFC6962.
  -loglevel int
        Log level (0 = DEBUG, 5 = FATAL) (default 1)
root@frank0-15:/opt/certs# 
root@frank0-15:/opt/certs#
root@frank0-15:/opt/certs# cfssl-json -h
Usage of cfssl-json:
  -bare
        \the response from CFSSL is not wrapped in the API standard response
  -f string
        JSON input (default "-")
  -stdout
        output the response instead of saving to a file
root@frank0-15:/opt/certs#
root@frank0-15:/opt/certs# cfssl gencert -initca ca-csr.json | cfssl-json -bare ca   #如果不加cfssl-json,那么无法生成文件格式的证书和私钥
2021/12/25 12:13:09 [INFO] generating a new CA key and certificate from CSR
2021/12/25 12:13:09 [INFO] generate received request
2021/12/25 12:13:09 [INFO] received CSR
2021/12/25 12:13:09 [INFO] generating key: rsa-2048
2021/12/25 12:13:10 [INFO] encoded CSR
2021/12/25 12:13:10 [INFO] signed certificate with serial number 604479646102437002690754421478010540266616363407
root@frank0-15:/opt/certs# 
root@frank0-15:/opt/certs#
root@frank0-15:/opt/certs# ls -l
total 16
-rw-r--r-- 1 root root  891 Dec 25 12:13 ca.csr
-rw-r--r-- 1 root root  226 Dec 25 12:07 ca-csr.json
-rw------- 1 root root 1679 Dec 25 12:13 ca-key.pem  #根证书私钥
-rw-r--r-- 1 root root 1139 Dec 25 12:13 ca.pem      #根证书

Docker安装部署

在13,14,15上安装Docker。具体方法参考官网,不同版本方法可能会有差异,所以不列出了,直接参考官网最准确:https://docs.docker.com/engine/install/ubuntu/,只展示在13上安装和配置的方法,14和15相同;

版本信息

配置daemon.jason

具体信息可以参考 ”dockerd -h“ 或者这篇文章:docker daemon.json文件配置

另外对于overlay2的存储驱动,可以参考这里:docker镜像(一)overlayfs;简而言之就是分层存储内容,然后merge到一起统一展示,这样有助于记录回滚,同Git很类似;另外注意daemon.json中不能有注释(不论是中文还是英文),否则启动不了!下面注释只为了个人查阅方便;

对于为什么要改变docker的cgroup driver,可以参考这篇文章:linux(centos8):配置docker的cgroup driver为systemd

root@frank0-13:~# mkdir -p /data/docker
root@frank0-13:~# more /etc/docker/daemon.json 
{
        "graph": "/data/docker",   #废弃了
        "data-root":"/data/docker",  #Docker运行时使用的根路径
        "storage-driver": "overlay2",
        "insecure-registries": ["registry.access.redhat.com", "quay.io", "harbor.frank.com"], #配置docker的私库地址
        "registry-mirrors": ["https://docker.mirrors.ustc.edu.cn/"], #镜像加速的地址
        "bip": "172.1.13.1/24", #指定docker0的IP
        "exec-opts": ["native.cgroupdriver=systemd"],  #kubenate建议使用systemd来限制,所以改驱动
        "live-restore": true
}
root@frank0-13:~# systemctl start docker
root@frank0-13:~# systemctl enable docker
root@frank0-13:~# docker info |grep -E "Cgroup Driver| Docker Root Dir|registry"
 Cgroup Driver: systemd
 Docker Root Dir: /data/docker
  registry.access.redhat.com
root@frank0-13:~# ifconfig docker0|grep inet
        inet 172.1.13.1  netmask 255.255.255.0  broadcast 172.1.13.255

Docker私有库安装部署

私有库采用的是harbor,vmware开源的软件,用的比较多,目前两个主版本1.x和2.x,我用了1.x的版本;详细信息可以参考这篇文章:Docker之分布式仓库Harbor部署 

版本信息

安装Harbor并部署

root@frank0-15:~# apt install docker-compose
root@frank0-15:~# mkdir /opt/src
root@frank0-15:~# cd /opt/src/
root@frank0-15:/opt/src# wget https://github.com/goharbor/harbor/releases/download/v1.10.10/harbor-offline-installer-v1.10.10.tgz
root@frank0-15:/opt/src# tar -xvf harbor-offline-installer-v1.10.10.tgz -C /opt/
root@frank0-15:/opt/src# cd ..
root@frank0-15:/opt# mv harbor/ harbor-v1.10.10
root@frank0-15:/opt# ln -s /opt/harbor-v1.10.10/ /opt/harbor
root@frank0-15:/opt# ls -l
total 16
drwxr-xr-x 2 root root 4096 Dec 25 12:13 certs
drwx--x--x 4 root root 4096 Dec 27 02:19 containerd
lrwxrwxrwx 1 root root   21 Jan 25 06:09 harbor -> /opt/harbor-v1.10.10/
drwxr-xr-x 2 root root 4096 Jan 25 06:07 harbor-v1.10.10
drwxr-xr-x 2 root root 4096 Jan 25 06:07 src
root@frank0-15:/opt# cd harbor
root@frank0-15:/opt/harbor# ls -l
total 597992
-rw-r--r-- 1 root root      3398 Jan 12 04:08 common.sh
-rw-r--r-- 1 root root 612306524 Jan 12 04:09 harbor.v1.10.10.tar.gz
-rw-r--r-- 1 root root      5882 Jan 12 04:08 harbor.yml
-rwxr-xr-x 1 root root      2284 Jan 12 04:08 install.sh
-rw-r--r-- 1 root root     11347 Jan 12 04:08 LICENSE
-rwxr-xr-x 1 root root      1750 Jan 12 04:08 prepare
root@frank0-15:/opt/harbor# mkdir -p /data/harbor/logs
root@frank0-15:/opt/harbor# cp harbor.yml harbor.yml.bak
root@frank0-15:/opt/harbor# diff harbor.yml harbor.yml.bak 
5c5
< hostname: harbor.frank.com
---
> hostname: reg.mydomain.com
10c10
<   port: 800
---
>   port: 80
13c13
< #https:
---
> https:
15c15
<   #  port: 443
---
>   port: 443
17,18c17,18
<   #  certificate: /your/certificate/path
<   # private_key: /your/private/key/path
---
>   certificate: /your/certificate/path
>   private_key: /your/private/key/path
27c27
< harbor_admin_password: 12345678
---
> harbor_admin_password: Harbor12345
40c40
< data_volume: /data/harbor
---
> data_volume: /data
77c77
<   level: debug
---
>   level: info
87c87
<     location: /data/harbor/logs
---
>     location: /var/log/harbor
root@frank0-15:/opt/harbor# ./install.sh 

[Step 0]: checking if docker is installed ...

Note: docker version: 20.10.8

[Step 1]: checking docker-compose is installed ...

Note: docker-compose version: 1.25.0

[Step 2]: loading Harbor images ...
Loaded image: goharbor/harbor-portal:v1.10.10
Loaded image: goharbor/registry-photon:v1.10.10
Loaded image: goharbor/clair-adapter-photon:v1.10.10
Loaded image: goharbor/chartmuseum-photon:v1.10.10
Loaded image: goharbor/notary-signer-photon:v1.10.10
Loaded image: goharbor/prepare:v1.10.10
Loaded image: goharbor/harbor-core:v1.10.10
Loaded image: goharbor/harbor-registryctl:v1.10.10
Loaded image: goharbor/redis-photon:v1.10.10
Loaded image: goharbor/harbor-log:v1.10.10
Loaded image: goharbor/harbor-db:v1.10.10
Loaded image: goharbor/harbor-jobservice:v1.10.10
Loaded image: goharbor/notary-server-photon:v1.10.10
Loaded image: goharbor/nginx-photon:v1.10.10
Loaded image: goharbor/clair-photon:v1.10.10


[Step 3]: preparing environment ...

[Step 4]: preparing harbor configs ...
prepare base dir is set to /opt/harbor-v1.10.10
/usr/src/app/utils/configs.py:100: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  configs = yaml.load(f)
WARNING:root:WARNING: HTTP protocol is insecure. Harbor will deprecate http protocol in the future. Please make sure to upgrade to https
/usr/src/app/utils/configs.py:90: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  versions = yaml.load(f)
Generated configuration file: /config/log/logrotate.conf
Generated configuration file: /config/log/rsyslog_docker.conf
Generated configuration file: /config/nginx/nginx.conf
Generated configuration file: /config/core/env
Generated configuration file: /config/core/app.conf
Generated configuration file: /config/registry/config.yml
Generated configuration file: /config/registryctl/env
Generated configuration file: /config/db/env
Generated configuration file: /config/jobservice/env
Generated configuration file: /config/jobservice/config.yml
Generated and saved secret to file: /secret/keys/secretkey
Generated certificate, key file: /secret/core/private_key.pem, cert file: /secret/registry/root.crt
Generated configuration file: /compose_location/docker-compose.yml
Clean up the input dir



[Step 5]: starting Harbor ...
Creating network "harbor-v11010_harbor" with the default driver
Creating harbor-log ... done
Creating harbor-portal ... done
Creating redis         ... done
Creating registryctl   ... done
Creating registry      ... done
Creating harbor-db     ... done
Creating harbor-core   ... done
Creating nginx             ... done
Creating harbor-jobservice ... done
✔ ----Harbor has been installed and started successfully.----
root@frank0-15:/opt/harbor# docker-compose ps
      Name                     Command                  State                       Ports                 
----------------------------------------------------------------------------------------------------------
harbor-core         /harbor/harbor_core              Up (healthy)                                         
harbor-db           /docker-entrypoint.sh            Up (healthy)   5432/tcp                              
harbor-jobservice   /harbor/harbor_jobservice  ...   Up (healthy)                                         
harbor-log          /bin/sh -c /usr/local/bin/ ...   Up (healthy)   127.0.0.1:1514->10514/tcp             
harbor-portal       nginx -g daemon off;             Up (healthy)   8080/tcp                              
nginx               nginx -g daemon off;             Up (healthy)   0.0.0.0:800->8080/tcp,:::800->8080/tcp
redis               redis-server /etc/redis.conf     Up (healthy)   6379/tcp                              
registry            /home/harbor/entrypoint.sh       Up (healthy)   5000/tcp                              
registryctl         /home/harbor/start.sh            Up (healthy)                                         

安装Nginx反向代理Harbor

注意:我在域名后面加了IP,这样访问127.0.0.1也可以被代理,否则只会代理域名(SSH Tunnel访问IP更简单);另外nginx默认会占用80端口,由于我的电脑要需要通过SSH Tunnel访问这台设备(具体配置和方案可以看这篇文章:),所以需要把默认的nginx的80端口关掉,这个可以移除“sites-enabled”的配置;

root@frank0-15:~# more /etc/nginx/nginx.conf |grep include
include /etc/nginx/modules-enabled/*.conf;
        include /etc/nginx/mime.types;
        include /etc/nginx/conf.d/*.conf;
#       include /etc/nginx/sites-enabled/*;
root@frank0-15:~# apt install nginx -y
root@frank0-15:~# more /etc/nginx/nginx.conf |grep -E "64|include"
include /etc/nginx/modules-enabled/*.conf;
        server_names_hash_bucket_size 64;
        include /etc/nginx/mime.types;
        include /etc/nginx/conf.d/*.conf;
        include /etc/nginx/sites-enabled/*;
root@frank0-15:~# more /etc/nginx/conf.d/harbor.frank.com.conf 
server {
        listen  80;
        server_name     harbor.frank.com,127.0.0.1;
        client_max_body_size    1000m;
        location / {
                proxy_pass http://127.0.0.1:800;
        }
}
root@frank0-15:~# nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
root@frank0-15:~# systemctl enable nginx
Synchronizing state of nginx.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable nginx
root@frank0-15:~# systemctl restart nginx

在DNS的业务域内增加解析

root@frank0-11:~# more /etc/bind/db.frank.com |grep harbor
harbor          A       172.20.0.15
root@frank0-11:~# dig -t A harbor.frank.com @172.20.0.11 +short
172.20.0.15

去15上验证下:

root@frank0-15:~# curl harbor.od.com
curl: (6) Could not resolve host: harbor.od.com
root@frank0-15:~# curl harbor.frank.com
<!doctype html>
<html>

<head>
    <meta charset="utf-8">
    <title>Harbor</title>
    <base href="/">
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <link rel="icon" type="image/x-icon" href="favicon.ico?v=2">
<link rel="stylesheet" href="styles.701dc5ee3007bd83bfa4.css"></head>

<body>
    <harbor-app>
        <div class="spinner spinner-lg app-loading">
            Loading...
        </div>
    </harbor-app>
<script src="runtime.9ad22a88fcc70a015907.js" defer></script><script src="polyfills-es5.d01e8ad6bc0c07b49ab6.js" nomodule defer></script><script src="scripts.7fa3fa51e1a86dfba2c8.js" defer></script><script src="main.dc691566affd40507d3b.js" defer></script></body>

在harbor上新建一个public的project:

从docker hub上下载Image并push到harbor上

在pull前如果想确认image的版本信息,可以参考我另一篇总结

root@frank0-15:/opt/harbor# docker pull nginx:1.9.8
1.9.8: Pulling from library/nginx
Image docker.io/library/nginx:1.9.8 uses outdated schema1 manifest format. Please upgrade to a schema2 image for better future compatibility. More information at https://docs.docker.com/registry/spec/deprecated-schema-v1/
d4bce7fd68df: Pull complete 
a3ed95caeb02: Pull complete 
573113c4751a: Pull complete 
31917632be33: Pull complete 
b4f281573cac: Pull complete 
3a650fa8762e: Pull complete 
e2f06f61b5b6: Pull complete 
Digest: sha256:5d3b56065cf133f8968a557bf49c639320b2fc4c57c635b5707978fb6b738e1f
Status: Downloaded newer image for nginx:1.9.8
docker.io/library/nginx:1.9.8
root@frank0-15:/opt/harbor# docker images |grep 1.9.8
nginx                           1.9.8      602a1c175cff   6 years ago   134MB
root@frank0-15:/opt/harbor# docker tag 602a1c175cff harbor.frank.com/public/nginx:v1.9.8
root@frank0-15:/opt/harbor# docker images |grep 1.9.8
nginx                           1.9.8      602a1c175cff   6 years ago   134MB
harbor.frank.com/public/nginx   v1.9.8     602a1c175cff   6 years ago   134MB
root@frank0-15:/opt/harbor# docker login harbor.frank.com
Username: admin
Password: 
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
root@frank0-15:/opt/harbor# docker push harbor.frank.com/public/nginx:v1.9.8
The push refers to repository [harbor.frank.com/public/nginx]
5f70bf18a086: Pushed 
83ee3f858827: Pushed 
63af2286bcc7: Pushed 
6e052756d5ce: Pushed 
ebfc3a74f160: Pushed 
031458dc7254: Pushed 
12e469267d21: Pushed 
v1.9.8: digest: sha256:2dea8e6cdf1629c45b9d8f6d92850ff18459beb6771c80282f76568b58deba72 size: 2805
本文出自 Frank's Blog

版权声明:


本文链接:Manual Deploy K8s
版权声明:本文为原创文章,仅代表个人观点,版权归 Frank Zhao 所有,转载时请注明本文出处及文章链接
你可以留言,或者trackback 从你的网站

留言哦

blonde teen swallows load.xxx videos