K8s – Core Resource Management Method

11 8 月, 2022,由 frank撰写

陈述式管理（CLI）

陈述式管理方式主要通过kubectl来进行管理，kubectl用于与apiserver进行通信，实现管理k8s各种资源的一种有效途径；

管理名称空间

查看名称空间

下面显示的都是默认生成的命名空间，简称为kubectl get ns：

root@f0-13:~# kubectl get namespace
NAME              STATUS   AGE
default           Active   44h
kube-node-lease   Active   44h
kube-public       Active   44h
kube-system       Active   44h

查询特定命名空间中的所有资源，下面是default命名空间：

root@f0-13:~# kubectl get all -n default
NAME                 READY   STATUS    RESTARTS   AGE
pod/nginx-ds-lc59t   1/1     Running   0          39h
pod/nginx-ds-pkh75   1/1     Running   0          39h


NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
service/kubernetes   ClusterIP   10.254.0.1   <none>        443/TCP   44h

NAME                      DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
daemonset.apps/nginx-ds   2         2         2       2            2           <none>          39h

创建名称空间

root@f0-13:~# kubectl create ns app
namespace/app created

删除名称空间

root@f0-13:~# kubectl delete ns app
namespace "app" deleted

管理deployment资源

创建deployment

root@f0-13:~# kubectl create deploy nginx-test --image=harbor.frank.com/public/nginx:v1.9.8 -n kube-public
deployment.apps/nginx-test created

查看deployment

root@f0-13:~# kubectl get deployment -n kube-public
NAME         READY   UP-TO-DATE   AVAILABLE   AGE
nginx-test   1/1     1            1           62s

root@f0-13:~# kubectl describe deploy nginx-test -n kube-public
Name:                   nginx-test
Namespace:              kube-public
CreationTimestamp:      Thu, 11 Aug 2022 10:24:43 +0800
Labels:                 app=nginx-test
Annotations:            deployment.kubernetes.io/revision: 1
Selector:               app=nginx-test
Replicas:               1 desired | 1 updated | 1 total | 1 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  25% max unavailable, 25% max surge
Pod Template:
  Labels:  app=nginx-test
  Containers:
   nginx:
    Image:        harbor.frank.com/public/nginx:v1.9.8
    Port:         <none>
    Host Port:    <none>
    Environment:  <none>
    Mounts:       <none>
  Volumes:        <none>
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Available      True    MinimumReplicasAvailable
  Progressing    True    NewReplicaSetAvailable
OldReplicaSets:  <none>
NewReplicaSet:   nginx-test-7d6f7dbd58 (1/1 replicas created)
Events:
  Type    Reason             Age   From                   Message
  ----    ------             ----  ----                   -------
  Normal  ScalingReplicaSet  31s   deployment-controller  Scaled up replica set nginx-test-7d6f7dbd58 to 1

查看pods资源

root@f0-13:~# kubectl get pods -n kube-public -o wide
NAME                          READY   STATUS        RESTARTS   AGE   IP           NODE                 NOMINATED NODE   READINESS GATES
nginx-test-7d6f7dbd58-rf59b   1/1     Running       0          90s   172.1.14.3   f0-14.host.com       <none>

进入pods资源

这是通过域名访问进去，如果f0-14.host.com无法解析，那么就无法登录进去；当然也可以通过docker exec -it bash进去；

root@f0-13:~# kubectl exec -ti nginx-test-7d6f7dbd58-rf59b /bin/bash -n kube-public
root@nginx-test-7d6f7dbd58-rf59b:/# 
root@nginx-test-7d6f7dbd58-rf59b:/# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
7: eth0@if8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default 
    link/ether 02:42:ac:01:0e:03 brd ff:ff:ff:ff:ff:ff
    inet 172.1.14.3/24 brd 172.1.14.255 scope global eth0
       valid_lft forever preferred_lft forever

删除pods

delete pods不是真正的删除，感觉就是销毁并重新建了一个，可以看到删除前和删除后的name已经不一样了，“–force –grace-period=0”可以强制删除（注意不管是force还是grace前面都是两个横线，web显示可能只有一个横线），但我看到的结果是仍然是删了重建：

root@f0-14:~# kubectl get pods -n kube-public -o wide
NAME                          READY   STATUS    RESTARTS   AGE   IP           NODE             NOMINATED NODE   READINESS GATES
nginx-test-7d6f7dbd58-rf59b   1/1     Running   0          19m   172.1.14.3   f0-14.host.com   <none>           <none>
root@f0-14:~# 
root@f0-14:~# 
root@f0-14:~# kubectl delete pod nginx-test-7d6f7dbd58-rf59b -n kube-public
pod "nginx-test-7d6f7dbd58-rf59b" deleted
root@f0-14:~# 
root@f0-14:~# kubectl get pods -n kube-public -o wide
NAME                          READY   STATUS    RESTARTS   AGE   IP           NODE             NOMINATED NODE   READINESS GATES
nginx-test-7d6f7dbd58-6rwxh   1/1     Running   0          26s   172.1.14.4   f0-14.host.com   <none>           <none>
root@f0-14:~#

删除deployment


root@f0-14:~# kubectl delete deploy nginx-test -n kube-public
deployment.extensions "nginx-test" deleted
root@f0-14:~# kubectl get deployment -n kube-public
No resources found.

管理Service资源

创建Service资源

可以看到上面删除pod后，scheduler根据算法，会在14或13上创建pod，那么pod的name/ip是变化的，为了对外提供稳定的服务，引入了service，可以为pod提供一个稳定的接入点（service是通过label selector找到的pod）；使用expose后，可以看到多出了一个service/nginx-test：

root@f0-13:~# kubectl expose deployment nginx-test --port=80 -n kube-public
service/nginx-test exposed
root@f0-13:~# kubectl get all -n kube-public -o wide
NAME                              READY   STATUS    RESTARTS   AGE   IP           NODE             NOMINATED NODE   READINESS GATES
pod/nginx-test-7d6f7dbd58-dgbvx   1/1     Running   0          15m   172.1.14.3   f0-14.host.com   <none>           <none>


NAME                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE     SELECTOR
service/nginx-test   ClusterIP   10.254.111.99   <none>        80/TCP    2m21s   app=nginx-test


NAME                         READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS   IMAGES                                 SELECTOR
deployment.apps/nginx-test   1/1     1            1           15m   nginx        harbor.frank.com/public/nginx:v1.9.8   app=nginx-test

NAME                                    DESIRED   CURRENT   READY   AGE   CONTAINERS   IMAGES                                 SELECTOR
replicaset.apps/nginx-test-7d6f7dbd58   1         1         1       15m   nginx        harbor.frank.com/public/nginx:v1.9.8   app=nginx-test,pod-template-hash=7d6f7dbd58

并且根据上面的信息，这个pod是在14上建立的，因此可以去14上curl下：

root@f0-14:~# curl 10.254.111.99
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

通过ipvsadm可以看出这个service，以及service后面藏着的pod的真正IP：

root@f0-14:~# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  10.254.0.1:443 nq
  -> 172.20.0.13:6443             Masq    1      0          0         
  -> 172.20.0.14:6443             Masq    1      0          0         
TCP  10.254.111.99:80 nq
  -> 172.1.14.3:80                Masq    1      0          1

可以对这个deployment资源扩容，然后看下service的变化：

root@f0-14:~# kubectl scale deployment nginx-test --replicas=2 -n kube-public
deployment.extensions/nginx-test scaled
root@f0-14:~# kubectl get pods -n kube-public -o wide
NAME                          READY   STATUS    RESTARTS   AGE   IP           NODE             NOMINATED NODE   READINESS GATES
nginx-test-7d6f7dbd58-4t4j9   1/1     Running   0          76s   172.1.13.3   f0-13.host.com   <none>           <none>
nginx-test-7d6f7dbd58-dgbvx   1/1     Running   0          25m   172.1.14.3   f0-14.host.com   <none>           <none>

root@f0-14:~# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  10.254.0.1:443 nq
  -> 172.20.0.13:6443             Masq    1      0          0         
  -> 172.20.0.14:6443             Masq    1      0          0         
TCP  10.254.111.99:80 nq
  -> 172.1.13.3:80                Masq    1      0          0         
  -> 172.1.14.3:80                Masq    1      0          0

查看service资源

root@f0-14:~# kubectl describe svc nginx-test -n kube-public
Name:              nginx-test
Namespace:         kube-public
Labels:            app=nginx-test
Annotations:       <none>
Selector:          app=nginx-test
Type:              ClusterIP
IP:                10.254.111.99
Port:              <unset>  80/TCP
TargetPort:        80/TCP
Endpoints:         172.1.13.3:80,172.1.14.3:80
Session Affinity:  None
Events:            <none>

声明式管理方法

声明式资源管理方法依赖于配置清单（yaml/jason），对比陈述式管理方法，配置清单更适合更改；

查看资源配置清单

直接在get命令后加上“-o yaml”即可，如果配置清单里不清楚的部分，可以用“kubectl explain”来查看（只能查看大类，如metadata，spec等）：

root@f0-13:~# kubectl get pods nginx-test-7d6f7dbd58-dgbvx -o yaml -n kube-public
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: "2022-08-11T06:38:06Z"
  generateName: nginx-test-7d6f7dbd58-
  labels:
    app: nginx-test
    pod-template-hash: 7d6f7dbd58
  name: nginx-test-7d6f7dbd58-dgbvx
  namespace: kube-public
  ownerReferences:
  - apiVersion: apps/v1
    blockOwnerDeletion: true
    controller: true
    kind: ReplicaSet
    name: nginx-test-7d6f7dbd58
    uid: ebe80c40-cc34-493e-bce0-954fb87f7dbc
  resourceVersion: "240564"
  selfLink: /api/v1/namespaces/kube-public/pods/nginx-test-7d6f7dbd58-dgbvx
  uid: 936c191c-6342-4e00-becd-f31c3db24436
spec:
  containers:
  - image: harbor.frank.com/public/nginx:v1.9.8
    imagePullPolicy: IfNotPresent
    name: nginx
    resources: {}
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: default-token-f5jx5
      readOnly: true
  dnsPolicy: ClusterFirst
  enableServiceLinks: true
  nodeName: f0-14.host.com
  priority: 0
  restartPolicy: Always
  schedulerName: default-scheduler
  securityContext: {}
  serviceAccount: default
  serviceAccountName: default
  terminationGracePeriodSeconds: 30
  tolerations:
  - effect: NoExecute
    key: node.kubernetes.io/not-ready
    operator: Exists
    tolerationSeconds: 300
  - effect: NoExecute
    key: node.kubernetes.io/unreachable
    operator: Exists
    tolerationSeconds: 300
  volumes:
  - name: default-token-f5jx5
    secret:
      defaultMode: 420
      secretName: default-token-f5jx5
status:
  conditions:
  - lastProbeTime: null
    lastTransitionTime: "2022-08-11T06:38:06Z"
    status: "True"
    type: Initialized
  - lastProbeTime: null
    lastTransitionTime: "2022-08-11T06:38:09Z"
    status: "True"
    type: Ready
  - lastProbeTime: null
    lastTransitionTime: "2022-08-11T06:38:09Z"
    status: "True"
    type: ContainersReady
  - lastProbeTime: null
    lastTransitionTime: "2022-08-11T06:38:06Z"
    status: "True"
    type: PodScheduled
  containerStatuses:
  - containerID: docker://1dc894676a194a3a2a0e8ff2a239964318a3d732bb4b8106967f1d47e6597a58
    image: harbor.frank.com/public/nginx:v1.9.8
    imageID: docker-pullable://harbor.frank.com/public/nginx@sha256:2dea8e6cdf1629c45b9d8f6d92850ff18459beb6771c80282f76568b58deba72
    lastState: {}
    name: nginx
    ready: true
    restartCount: 0
    state:
      running:
        startedAt: "2022-08-11T06:38:08Z"
  hostIP: 172.20.0.14
  phase: Running
  podIP: 172.1.14.3
  qosClass: BestEffort
  startTime: "2022-08-11T06:38:06Z"

root@f0-13:~# kubectl get svc nginx-test -n kube-public -o yaml
apiVersion: v1
kind: Service
metadata:
  creationTimestamp: "2022-08-11T06:50:51Z"
  labels:
    app: nginx-test
  name: nginx-test
  namespace: kube-public
  resourceVersion: "241660"
  selfLink: /api/v1/namespaces/kube-public/services/nginx-test
  uid: d857935c-91b8-4687-888e-885a12d7ac7b
spec:
  clusterIP: 10.254.111.99
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx-test
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}

root@f0-13:~# kubectl explain svc.status -n kube-public
KIND:     Service
VERSION:  v1

RESOURCE: status <Object>

DESCRIPTION:
     Most recently observed status of the service. Populated by the system.
     Read-only. More info:
     https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status

     ServiceStatus represents the current status of a service.

FIELDS:
   loadBalancer <Object>
     LoadBalancer contains the current status of the load-balancer, if one is
     present.

创建资源配置清单

要通过配置清单创建资源，就需要找一个模板，这个可以从kubenets的中文网站上找，比如创建service，下面标记的地方都是根据情况添加进来的：

root@f0-13:~# more nginx-ds-svc.yaml 
apiVersion: v1
kind: Service
metadata:
  name: my-service
  labels:
    app: nginx-test
  namespace: kube-public
spec:
  selector:
    app: nginx-test
  ports:
    - protocol: TCP
      port: 81
      targetPort: 81

root@f0-13:~# kubectl create -f nginx-ds-svc.yaml 
service/my-service created
root@f0-13:~# kubectl get svc -n kube-public
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
my-service   ClusterIP   10.254.69.225   <none>        81/TCP    19s
nginx-test   ClusterIP   10.254.111.99   <none>        80/TCP    18h

修改资源配置清单

在线修改

不推荐，因为修改完后没有记录

root@f0-13:~# kubectl edit svc my-service -n kube-public
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
......
spec:
  clusterIP: 10.254.69.225
  ports:
  - port: 82
    protocol: TCP
    targetPort: 81
......
"/tmp/kubectl-edit-zfygz.yaml" 30L, 1009C written                                                                                                                                 
service/my-service edited
root@f0-13:~# kubectl get svc -n kube-public
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
my-service   ClusterIP   10.254.69.225   <none>        82/TCP    16m

离线修改

这里是直接修改yaml/json配置文件，然后apply（建议每次修改都copy一份原始文件，这样好溯源）；对于一些报错（比如修改端口，可能会出现“spec.ports[0].name: Required value”），我看有人直接加上“–force”完成的，暂时没看到解释为什么；

删除资源配置清单

通过指定的配置清单删除特定的资源，这个对于声明式的删除要麻烦些，需要找到yaml文件才可以；

root@f0-13:~# kubectl delete -f nginx-ds-svc.yaml 
service "my-service" deleted
root@f0-13:~# kubectl get svc -n kube-public
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
nginx-test   ClusterIP   10.254.111.99   <none>        80/TCP    18h

GUI式管理方法

Dashboard是k8s的GUI管理组件，对于如何部署和登录相关内容，可以看我另一篇文章：K8s – Dashboard，这里只涉及如何使用。创建一个新账户，权限和需求如下：

新账户：frank-test；
只能访问 default命名空间下的pods；

新建一个ServiceAccount

新建一个服务账号，名字叫“frank”：

root@f0-13:~# kubectl create sa frank -n default
serviceaccount/frank created
root@f0-13:~# 
root@f0-13:~# kubectl get sa -n default
NAME      SECRETS   AGE
default   1         14d
frank     1         20s

新建一个角色Role

新建一个角色，名字叫“frank”，并赋予相应的权限，比如不能删除pods：

root@f0-15:/data/k8s-yaml/dashboard# more user-rbac.yaml 
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  namespace: default
  name: frank
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "watch", "list"]
- apiGroups: ["extensions", "apps"]
  resources: ["deployments"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]

root@f0-13:~# kubectl apply -f http://k8s-yaml.frank.com/dashboard/user-rbac.yaml
role.rbac.authorization.k8s.io/frank created

新建一个角色绑定RoleBinding

新建一个角色绑定，名字叫“role-bind-frank”，绑定Role ”frank“和Service Account “frank”，这里我起的是一样的名字，可以是不同的名字，只要跟上面对上：

root@f0-15:/data/k8s-yaml/dashboard# more user-role-bind.yaml 
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: role-bind-frank
  namespace: default
subjects:
- kind: ServiceAccount
  name: frank
  namespace: default
roleRef:
  kind: Role
  name: frank
  apiGroup: rbac.authorization.k8s.io

root@f0-13:~# kubectl apply -f http://k8s-yaml.frank.com/dashboard/user-role-bind.yaml
rolebinding.rbac.authorization.k8s.io/role-bind-frank created

通过Token登录并确认权限

root@f0-13:~# kubectl get secret
NAME                  TYPE                                  DATA   AGE
default-token-7s2vl   kubernetes.io/service-account-token   3      14d
frank-token-pjqjx     kubernetes.io/service-account-token   3      115m
root@f0-13:~# 
root@f0-13:~# kubectl describe secret frank-token-pjqjx
Name:         frank-token-pjqjx
Namespace:    default
Labels:       <none>
Annotations:  kubernetes.io/service-account.name: frank
              kubernetes.io/service-account.uid: 6ff29c71-bd52-4095-94f9-1c3ea7cdce7f

Type:  kubernetes.io/service-account-token

Data
====
ca.crt:     1127 bytes
namespace:  7 bytes
token:      eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZWZhdWx0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6ImZyYW5rLXRva2VuLXBqcWp4Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImZyYW5rIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiNmZmMjljNzEtYmQ1Mi00MDk1LTk0ZjktMWMzZWE3Y2RjZTdmIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50OmRlZmF1bHQ6ZnJhbmsifQ.LJkO673_Xs973hnaxfMSckRaakfQBuBElLU81OU5AIIZHbMJ60lBANXrVjeVD-u9VpsigAdCx6JTc25p605z1otEK8TOH39T3Kg9xnqyVbP9qk836xFpDKUbd3hHg7MEw0guOwolmrClUhHecMHihuh1ucAs1TlShrPBotmu21sYLA78FVnWMQo0iT-2zC5UbvO7ufJ5AlTBYOxOcboO-Mlp0e-xuFroyD4xTBbEVKiFBTSFFbd4PsSVfmDatbbUE1XL9v5E9s1_oIXa4ce92mWxIoePcgxHEtlO0wYLLhzu8Z8L7Y17AjmdGo44UX-_zBQdeuQV3f5ngLxndr44yQ

可以看到namespace中只有default，没有其他的，另外打开后有很多告警，这部分就是说无法列出default里除了Pods外的其他资源：

删除其中一个pod看看：

本文出自 Frank's Blog

本文链接：K8s – Core Resource Management Method
版权声明：本文为原创文章，仅代表个人观点，版权归 Frank Zhao 所有，转载时请注明本文出处及文章链接

Linux, 学习笔记 k8s,Linux

你可以留言,或者trackback 从你的网站

Manual set up two containers by “docker run”«

»K8s – CNI Network Component

留言哦

返回顶端

『无题』

I persist, therefore I am!!!

Memo

Perseverance sometimes equals genius!
There are only two creatures in the world who can surmount the pyramid:

—–the eagle and the snail.
Useful Link

1. IANA - AFI
2. IANA - SAFI
3. IANA - BGP Ex Comm
4. RPM pbone.net
5. Linux Packages Search
Useful Toots

Linux/http server:

python -m SimpleHTTPServer <port>

python -m http.server <port>

PS: only one thread, if require more threads, can replace by “caddy”.

Win10/http server:

http://www.rejetto.com/hfs/

Win10/md5:

certutil -hashfile .\xxxx MD5
分类
- CCDE (16)
- CCIE SEC (4)
- CCIE SP (55)
- CCIE VOICE (3)
- Classical CASE (26)
- Linux (75)
- Mobile Phone (4)
- Programmer (44)
- Solution (3)
- Uncategorized (6)
- Windows常用技术 (10)
- 似水流年 (17)
- 学习笔记 (191)
- 常用资料 (44)
- 网络技术 (39)
近期文章
其他操作