kubernetes1.30集群部署+dashboard+heapster

文章目录[x]
  1. 1.1、禁用防火墙、禁用selinux
  2. 1.2、创建修改内核文件/etc/sysctl.d/k8s.conf
  3. 1.3、禁用swap
  4. 6.1初始化您的主节点
  5. 6.2按提示配置常规用户如何使用kubectl访问集群
  6. 6.3查看集群状态
  7. 7、准备接下来用到的yml文件
  8. 8、安装Pod Network
  9. 9、添加 Slave节点
  10. 9.1、在两个 Slave节点上分别执行jion
  11. 9.2、node1取于前面kubeadm init 生成的(node2省略)
  12. 9.3、效果验证
  13. 10、安装 dashboard
  14. 10.1、创建
  15. 10.2、查看 dashboard的 pod是否正常启动
  16. 10.3、查看 dashboard的外网暴露端口
  17. 10.4、生成私钥和证书签名
  18. 10.5、 生成SSL证书,并移动到 /etc/kubernetes/pki
  19. 10.6、创建 dashboard用户
  20. 10.7、获取登陆token
  21. 10.8、访问UI页面
  22. 11.1、准备相应的yaml文件
  23. 11.3、验证结果
  24. 12.1、拆卸集群
  25. 12.2、master处理节点
  26. 12.3节点重置(node2省略)
  27. 12.4、master重置

v2.1

1、系统配置

1.1、禁用防火墙、禁用selinux

#防火墙禁用
systemctl stop firewalld
systemctl disable firewalld

#SELinux禁用
setenforce 0
sed -i '/^SE/s/enforcing/disabled/' /etc/selinux/config 

1.2、创建修改内核文件/etc/sysctl.d/k8s.conf

#写入文件
cat <<EOF > /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF

#使用sysctl使其生效
sysctl -p /etc/sysctl.d/k8s.conf

1.3、禁用swap


#关闭系统的Swap
swapoff -a

#永久生效
echo "vm.swappiness=0" >> /etc/sysctl.d/k8s.conf

#修改 /etc/fstab 文件,注释掉 SWAP 的自动挂载
#/dev/mapper/centos-swap swap                    swap    defaults        0 0

2、配置yum源

#配置阿里的docker-ce 
yum-config-manager --add-repo  https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

#配置kubernetes
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

3、安装docker

#建立缓存
yum makecache fast

#安装依赖
yum install -y xfsprogs yum-utils device-mapper-persistent-data lvm2

#安装最新版本的docker
yum install -y --setopt=obsoletes=0 docker-ce-18.06.1.ce-3.el7

#可以使用以下命令查看docker版本
yum list docker-ce.x86_64  --showduplicates |sort -r

#启动docker
systemctl start docker
systemctl enable docker

 

4、在各节点安装kubeadm、kubelet和kubectl

#安装kubelet kubeadm kubectl
yum install -y kubelet kubeadm kubectl

5、准备需要用到的镜像

#因为kubeadm使用的镜像源是k8s.gcr.io,但是网站已被墙。所以需要提前准备所需要的镜像
#可以使用kubeadm 查看默认需要准备的镜像。并且在此可以看到k8s.gcr.io是无法连接的

[root@K8s-master ~]# kubeadm config images list
I0116 10:11:34.091410   91268 version.go:94] could not fetch a Kubernetes version from the internet: unable to get URL "https://dl.k8s.io/release/stable-1.txt": Get https://dl.k8s.io/release/stable-1.txt: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
I0116 10:11:34.107299   91268 version.go:95] falling back to the local client version: v1.13.2
k8s.gcr.io/kube-apiserver:v1.13.2
k8s.gcr.io/kube-controller-manager:v1.13.2
k8s.gcr.io/kube-scheduler:v1.13.2
k8s.gcr.io/kube-proxy:v1.13.2
k8s.gcr.io/pause:3.1
k8s.gcr.io/etcd:3.2.24
k8s.gcr.io/coredns:1.2.6

根据提示,到docker hub 搜索到对应的镜像后pull下来再tag标签

kube-apiserver:v1.13.0
kube-controller-manager:v1.13.0
kube-scheduler:v1.13.0
kube-proxy:v1.13.0
pause:3.1
etcd:3.2.24
coredns:1.2.6

下载完后,tag标签为k8s.gcr.io/***

如:

docker pull mirrorgooglecontainers/kube-apiserver:v1.13.0

docker tag  mirrorgooglecontainers/kube-apiserver:v1.13.0  k8s.gcr.io/kube-apiserver:v1.13.0

[root@K8s-master ~]# docker images
REPOSITORY                             TAG                 IMAGE ID            CREATED             SIZE
k8s.gcr.io/kube-proxy                  v1.13.0             8fa56d18961f        6 weeks ago         80.2MB
k8s.gcr.io/kube-apiserver              v1.13.0             f1ff9b7e3d6e        6 weeks ago         181MB
k8s.gcr.io/kube-controller-manager     v1.13.0             d82530ead066        6 weeks ago         146MB
k8s.gcr.io/kube-scheduler              v1.13.0             9508b7d8008d        6 weeks ago         79.6MB
k8s.gcr.io/coredns                     1.2.6               f59dcacceff4        2 months ago        40MB
k8s.gcr.io/etcd                        3.2.24              3cab8e1b9802        3 months ago        220MB
k8s.gcr.io/pause                       3.1                 da86e6ba6ca1        13 months ago       742kB

镜像准备完毕,开始集群初始化

 

6、初始化集群

master 节点

6.1初始化您的主节点

主节点是集群里运行控制面的机器,包括 etcd (集群的数据库)和 API 服务(kubectl CLI 与之交互)。

  1. 选择一个 Pod 网络插件,并检查是否在 kubeadm 初始化过程中需要传入什么参数。这个取决于 您选择的网络插件,您可能需要设置 --Pod-network-cidr 来指定网络驱动的 CIDR。请参阅安装网络插件
  2. (可选) 除非特别指定,kubeadm 会使用默认网关所在的网络接口广播其主节点的 IP 地址。若需使用其他网络接口,请 给 kubeadm init 设置 --apiserver-advertise-address=<ip-address> 参数。如果需要部署 IPv6 的集群,则需要指定一个 IPv6 地址,比如 --apiserver-advertise-address=fd00::101
  3. (可选) 在运行 kubeadm init 之前请先执行 kubeadm config images pull 来测试与 gcr.io 的连接。

-- 取自kubernetes官网

https://kubernetes.io/zh/docs/setup/independent/create-cluster-kubeadm/

开始初始化

#master 集群初始化
[root@K8s-master ~]# kubeadm init --kubernetes-version=v1.13.0 --pod-network-cidr=10.244.0.0/16 --apiserver-advertise-address=192.168.146.10
[init] Using Kubernetes version: v1.13.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [k8s-master localhost] and IPs [192.168.146.10 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [k8s-master localhost] and IPs [192.168.146.10 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [k8s-master kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.146.10]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[apiclient] All control plane components are healthy after 184.547282 seconds
[uploadconfig] storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.13" in namespace kube-system with the configuration for the kubelets in the cluster
[patchnode] Uploading the CRI Socket information "/var/run/dockershim.sock" to the Node API object "k8s-master" as an annotation
[mark-control-plane] Marking the node k8s-master as control-plane by adding the label "node-role.kubernetes.io/master=''"
[mark-control-plane] Marking the node k8s-master as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[bootstrap-token] Using token: fl099d.dxy00288pqxl2xj0
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstraptoken] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstraptoken] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstraptoken] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstraptoken] creating the "cluster-info" ConfigMap in the "kube-public" namespace
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes master has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of machines by running the following on each node
as root:

  kubeadm join 192.168.146.10:6443 --token f0hxzt.q34cvw84otvdnca8 --discovery-token-ca-cert-hash sha256:76c15a976e3bd80c5cea54afeba0587682a131cfc5485cb28e980000102bd945

kubeadm init --kubernetes-version=v1.13.0 --pod-network-cidr=10.244.0.0/16 --apiserver-advertise-address=192.168.146.10

选择flannel作为Pod网络插件,所以指定–pod-network-cidr=10.244.0.0/16,API server 用来告知集群中其它成员的地址

 

6.2按提示配置常规用户如何使用kubectl访问集群

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

6.3查看集群状态

[root@K8s-master ~/kubernetes]# kubectl get cs
NAME                 STATUS    MESSAGE              ERROR
scheduler            Healthy   ok                   
controller-manager   Healthy   ok                   
etcd-0               Healthy   {"health": "true"}   

7、准备接下来用到的yml文件

  1. kube-flannel.yml
  2. kube-dashboard.yaml
  3. dashboard-user-role.yaml

使用git 拉取准备文件

[root@K8s-master ~]# git clone https://github.com/Nedved-liao/kubernetes.1.30_CN
正克隆到 'kubernetes.1.30_CN'...
remote: Enumerating objects: 23, done.
remote: Counting objects: 100% (23/23), done.
remote: Compressing objects: 100% (22/22), done.
remote: Total 23 (delta 6), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (23/23), done.

[root@K8s-master ~]# ll kubernetes.1.30_CN/
总用量 32
-rw-r--r--. 1 root root   515 1月  16 10:44 dashboard-user-role.yaml
-rw-r--r--. 1 root root  4675 1月  16 10:44 kube-dashboard.yaml
-rw-r--r--. 1 root root 10739 1月  16 10:44 kube-flannel.yml
-rw-r--r--. 1 root root   276 1月  16 10:44 kubernetes.repo
-rw-r--r--. 1 root root   490 1月  16 10:44 README.md

8、安装Pod Network


#安装flannel
[root@K8s-master ~/kubernetes.1.30_CN]# kubectl create -f kube-flannel.yml 
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
serviceaccount/flannel created
configmap/kube-flannel-cfg created
daemonset.extensions/kube-flannel-ds-amd64 created
daemonset.extensions/kube-flannel-ds-arm64 created
daemonset.extensions/kube-flannel-ds-arm created
daemonset.extensions/kube-flannel-ds-ppc64le created
daemonset.extensions/kube-flannel-ds-s390x created


#粗略查看
[root@K8s-master ~/kubernetes]# kubectl get ds -l app=flannel -n kube-system
NAME                      DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR                     AGE
kube-flannel-ds-amd64     1         1         1       1            1           beta.kubernetes.io/arch=amd64     113m
kube-flannel-ds-arm       0         0         0       0            0           beta.kubernetes.io/arch=arm       113m
kube-flannel-ds-arm64     0         0         0       0            0           beta.kubernetes.io/arch=arm64     113m
kube-flannel-ds-ppc64le   0         0         0       0            0           beta.kubernetes.io/arch=ppc64le   113m
kube-flannel-ds-s390x     0         0         0       0            0           beta.kubernetes.io/arch=s390x     113m

#确保结果无疑
[root@K8s-master ~/kubernetes]# kubectl get pod --all-namespaces -o wide
NAMESPACE     NAME                                    READY   STATUS    RESTARTS   AGE    IP               NODE         NOMINATED NODE   READINESS GATES
kube-system   coredns-86c58d9df4-65nbv                1/1     Running   1          179m   10.244.0.4       k8s-master   <none>           <none>
kube-system   coredns-86c58d9df4-gnczd                1/1     Running   1          179m   10.244.0.5       k8s-master   <none>           <none>
kube-system   etcd-k8s-master                         1/1     Running   1          178m   192.168.146.10   k8s-master   <none>           <none>
kube-system   kube-apiserver-k8s-master               1/1     Running   1          179m   192.168.146.10   k8s-master   <none>           <none>
kube-system   kube-controller-manager-k8s-master      1/1     Running   1          179m   192.168.146.10   k8s-master   <none>           <none>
kube-system   kube-flannel-ds-amd64-9x8mc             1/1     Running   0          113m   192.168.146.10   k8s-master   <none>           <none>
kube-system   kube-proxy-l9pgs                        1/1     Running   1          179m   192.168.146.10   k8s-master   <none>           <none>
kube-system   kube-scheduler-k8s-master               1/1     Running   1          179m   192.168.146.10   k8s-master   <none>           <none>
kube-system   kubernetes-dashboard-6cb88fb59c-fccp4   1/1     Running   0          51m    10.244.0.7       k8s-master   <none>           <none>

9、添加 Slave节点

9.1、在两个 Slave节点上分别执行jion

如下命令来让其加入Master上已经就绪了的 k8s集群:

kubeadm join --token <token> <master-ip>:<master-port> --discovery-token-ca-cert-hash sha256:<hash>

9.2、node1取于前面kubeadm init 生成的(node2省略)

[root@K8s-node1 ~]# kubeadm join 192.168.146.10:6443 --token f0hxzt.q34cvw84otvdnca8 --discovery-token-ca-cert-hash sha256:76c15a976e3bd80c5cea54afeba0587682a131cfc5485cb28e980000102bd945
[preflight] Running pre-flight checks
[discovery] Trying to connect to API Server "192.168.146.10:6443"
[discovery] Created cluster-info discovery client, requesting info from "https://192.168.146.10:6443"
[discovery] Requesting info from "https://192.168.146.10:6443" again to validate TLS against the pinned public key
[discovery] Cluster info signature and contents are valid and TLS certificate validates against pinned roots, will use API Server "192.168.146.10:6443"
[discovery] Successfully established connection with API Server "192.168.146.10:6443"
[join] Reading configuration from the cluster...
[join] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[kubelet] Downloading configuration for the kubelet from the "kubelet-config-1.13" ConfigMap in the kube-system namespace
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Activating the kubelet service
[tlsbootstrap] Waiting for the kubelet to perform the TLS Bootstrap...
[patchnode] Uploading the CRI Socket information "/var/run/dockershim.sock" to the Node API object "k8s-node1" as an annotation

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the master to see this node join the cluster.

如果 token忘记,则可以去 Master上执行如下命令来获取:

[root@K8s-master ~/kubernetes.1.30_CN]# kubeadm token list
TOKEN                     TTL       EXPIRES                     USAGES                   DESCRIPTION                                                EXTRA GROUPS
f0hxzt.q34cvw84otvdnca8   23h       2019-01-17T11:16:24+08:00   authentication,signing   The default bootstrap token generated by 'kubeadm init'.   system:bootstrappers:kubeadm:default-node-token

9.3、效果验证

查看节点状态

[root@K8s-master ~/kubernetes.1.30_CN]# kubectl get no
NAME         STATUS   ROLES    AGE     VERSION
k8s-master   Ready    master   5m28s   v1.13.1
k8s-node1    Ready    <none>   107s    v1.13.2
k8s-node2    Ready    <none>   110s    v1.13.2

查看所有 Pod状态

[root@K8s-master ~/kubernetes.1.30_CN]# kubectl get pods --all-namespaces -o wide
NAMESPACE     NAME                                 READY   STATUS    RESTARTS   AGE     IP               NODE         NOMINATED NODE   READINESS GATES
kube-system   coredns-86c58d9df4-xrz2j             1/1     Running   0          5m26s   10.244.0.19      k8s-master   <none>           <none>
kube-system   coredns-86c58d9df4-z2lcj             1/1     Running   0          5m26s   10.244.0.18      k8s-master   <none>           <none>
kube-system   etcd-k8s-master                      1/1     Running   0          4m53s   192.168.146.10   k8s-master   <none>           <none>
kube-system   kube-apiserver-k8s-master            1/1     Running   0          5m2s    192.168.146.10   k8s-master   <none>           <none>
kube-system   kube-controller-manager-k8s-master   1/1     Running   0          4m51s   192.168.146.10   k8s-master   <none>           <none>
kube-system   kube-flannel-ds-amd64-dwpcr          1/1     Running   0          4m9s    192.168.146.10   k8s-master   <none>           <none>
kube-system   kube-flannel-ds-amd64-h2tm9          1/1     Running   0          2m4s    192.168.146.20   k8s-node1    <none>           <none>
kube-system   kube-flannel-ds-amd64-ssh2d          1/1     Running   0          2m7s    192.168.146.21   k8s-node2    <none>           <none>
kube-system   kube-proxy-4lr8q                     1/1     Running   0          5m26s   192.168.146.10   k8s-master   <none>           <none>
kube-system   kube-proxy-7b22t                     1/1     Running   0          2m7s    192.168.146.21   k8s-node2    <none>           <none>
kube-system   kube-proxy-j6qkx                     1/1     Running   0          2m4s    192.168.146.20   k8s-node1    <none>           <none>
kube-system   kube-scheduler-k8s-master            1/1     Running   0          4m58s   192.168.146.10   k8s-master   <none>           <none>

10、安装 dashboard

就像给elasticsearch配一个可视化的管理工具一样,我们最好也给 k8s集群配一个可视化的管理工具,便于管理集群。
因此我们接下来安装 v1.10.0版本的 kubernetes-dashboard,用于集群可视化的管理。

10.1、创建

[root@K8s-master ~/kubernetes.1.30_CN]# kubectl create -f kube-dashboard.yaml 
secret/kubernetes-dashboard-certs created
serviceaccount/kubernetes-dashboard created
role.rbac.authorization.k8s.io/kubernetes-dashboard-minimal created
rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard-minimal created
deployment.apps/kubernetes-dashboard creat

10.2、查看 dashboard的 pod是否正常启动

如果正常说明安装成功:

[root@K8s-master ~/kubernetes.1.30_CN]# kubectl get pods --namespace=kube-system
NAME                                   READY   STATUS    RESTARTS   AGE
coredns-86c58d9df4-xrz2j               1/1     Running   0          165m
coredns-86c58d9df4-z2lcj               1/1     Running   0          165m
etcd-k8s-master                        1/1     Running   0          164m
kube-apiserver-k8s-master              1/1     Running   0          164m
kube-controller-manager-k8s-master     1/1     Running   0          164m
kube-flannel-ds-amd64-dwpcr            1/1     Running   0          163m
kube-flannel-ds-amd64-h2tm9            1/1     Running   0          161m
kube-flannel-ds-amd64-ssh2d            1/1     Running   0          161m
kube-proxy-4lr8q                       1/1     Running   0          165m
kube-proxy-7b22t                       1/1     Running   0          161m
kube-proxy-j6qkx                       1/1     Running   0          161m
kube-scheduler-k8s-master              1/1     Running   0          164m
kubernetes-dashboard-98b7c88bb-lqggg   1/1     Running   0          150m

10.3、查看 dashboard的外网暴露端口

[root@K8s-master ~/kubernetes.1.30_CN]# kubectl get service --namespace=kube-system
NAME                   TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)         AGE
kube-dns               ClusterIP   10.96.0.10     <none>        53/UDP,53/TCP   167m
kubernetes-dashboard   NodePort    10.106.23.70   <none>        443:31280/TCP   151m

10.4、生成私钥和证书签名

[root@K8s-master ~/kubernetes.1.30_CN]# openssl genrsa -des3 -passout pass:x -out dashboard.pass.key 2048
Generating RSA private key, 2048 bit long modulus
.....+++
.......................................................................................................................................................................................................................................................................................................+++
e is 65537 (0x10001)

[root@K8s-master ~/kubernetes.1.30_CN]# openssl rsa -passin pass:x -in dashboard.pass.key -out dashboard.key
writing RSA key

[root@K8s-master ~/kubernetes.1.30_CN]# rm -f dashboard.pass.key

#一路按回车即可
[root@K8s-master ~/kubernetes.1.30_CN]# openssl req -new -key dashboard.key -out dashboard.csr
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:
State or Province Name (full name) []:
Locality Name (eg, city) [Default City]:
Organization Name (eg, company) [Default Company Ltd]:
Organizational Unit Name (eg, section) []:
Common Name (eg, your name or your server's hostname) []:
Email Address []:

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:

10.5、 生成SSL证书,并移动到 /etc/kubernetes/pki

将生成的 dashboard.key 和 dashboard.crt置于路径 /etc/kubernetes/pki下, 路径在dashboard.yaml 里面已经定义好

#生成证书
[root@K8s-master ~/kubernetes.1.30_CN]# openssl x509 -req -sha256 -days 365 -in dashboard.csr -signkey dashboard.key -out dashboard.crt
Signature ok
subject=/C=XX/L=Default City/O=Default Company Ltd
Getting Private key

#移动到/etc/kubernetes/pki
[root@K8s-master ~/kubernetes.1.30_CN]# cp dashboard.c* /etc/kubernetes/pki/

#确认移动
[root@K8s-master ~/kubernetes.1.30_CN]# ll /etc/kubernetes/pki/dashboard.*
-rw-r--r--. 1 root root 1103 1月  16 14:08 /etc/kubernetes/pki/dashboard.crt
-rw-r--r--. 1 root root  952 1月  16 14:08 /etc/kubernetes/pki/dashboard.csr

10.6、创建 dashboard用户

[root@K8s-master ~/kubernetes.1.30_CN]# kubectl create -f dashboard-user-role.yaml
clusterrolebinding.rbac.authorization.k8s.io/admin created
serviceaccount/admin created

10.7、获取登陆token

[root@K8s-master ~/kubernetes.1.30_CN]# kubectl describe secret/$(kubectl get secret -nkube-system |grep admin|awk '{print $1}') -nkube-system
Name:         admin-token-q6v79
Namespace:    kube-system
Labels:       <none>
Annotations:  kubernetes.io/service-account.name: admin
              kubernetes.io/service-account.uid: 9ef5dbe7-1955-11e9-a360-000c29b4a7f7

Type:  kubernetes.io/service-account-token

Data
====
ca.crt:     1025 bytes
namespace:  11 bytes
token:      eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJhZG1pbi10b2tlbi1xNnY3OSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJhZG1pbiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6IjllZjVkYmU3LTE5NTUtMTFlOS1hMzYwLTAwMGMyOWI0YTdmNyIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlLXN5c3RlbTphZG1pbiJ9.BS93rVuy_61e5NKU20ZbXMybtr239tLgv9cmfL1knu9YZf66GBwQvERUtnHcYACt8vaD55RNLVk_9uHAKESSo0iJMv1-doKbAPZDrL-PT7XomrSgHleVzSyPHMixFRZVcQXpi5l1DcBC2QdNdfZL7h5SAnrs2NFuoGRv5IQXMMnlRVvbWFhBXIbVqRU7lEJo7VXglOYFjNPOC8JkTxxk2GsWJmp1zT-8ZRpajhfhe9VFxi-JLcKgMgv4d5IYGXr1CGcwMIChJz7jnPg7itSTpyYLGGTinZx0HhBivMw9hRm6RqAQgsr4g9sgGCeBRrFcZMbjKsvaQ3dnb7Dnupyyag

访问https://IP:port
port 在dashboard.yaml 中的nodeport定义

10.8、访问UI页面

查看暴露端口

[root@K8s-master ~/kubernetes.1.30_CN]# kubectl get svc --all-namespaces
NAMESPACE     NAME                   TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)         AGE
default       kubernetes             ClusterIP   10.96.0.1      <none>        443/TCP         5h27m
kube-system   kube-dns               ClusterIP   10.96.0.10     <none>        53/UDP,53/TCP   5h27m
kube-system   kubernetes-dashboard   NodePort    10.106.23.70   <none>        443:31280/TCP   5h12m

如上所示,Dashboard已经在31280端口上公开,现在可以在外部使用https://<cluster-ip>:31280进行访问。

需要注意的是,在多节点的集群中,必须找到运行Dashboard节点的IP来访问,而不是Master节点的IP

MatserIP为192.168.146.10,ClusterIP为192.168.146.20,192.168.146.21

直接访问master 的可能会出现如下 :

 

我们使用kubectl describe 来看下实际上是哪个node在运行dashboard

[root@K8s-master ~/kubernetes.1.30_CN]# kubectl describe po kubernetes-dashboard-98b7c88bb-lqggg  -n kube-system 
Name:               kubernetes-dashboard-98b7c88bb-lqggg
Namespace:          kube-system
Priority:           0
PriorityClassName:  <none>
Node:               k8s-node2/192.168.146.21
Start Time:         Wed, 16 Jan 2019 11:31:42 +0800
Labels:             k8s-app=kubernetes-dashboard
                    pod-template-hash=98b7c88bb
Annotations:        <none>
Status:             Running
IP:                 10.244.1.2
Controlled By:      ReplicaSet/kubernetes-dashboard-98b7c88bb
Containers:
  kubernetes-dashboard:
    Container ID:  docker://4fe074b5a90e86bcdb1d055d26204523c0cd72ecd5502af5a805d992ff77436f
    Image:         mirrorgooglecontainers/kubernetes-dashboard-amd64:v1.10.0
    Image ID:      docker-pullable://mirrorgooglecontainers/kubernetes-dashboard-amd64@sha256:e4b764fa9df0a30c467e7cec000920ea69dcc2ba8a9d0469ffbf1881a9614270
    Port:          8443/TCP
    Host Port:     0/TCP
    Args:
      --auto-generate-certificates
      --token-ttl=5400
    State:          Running
      Started:      Wed, 16 Jan 2019 11:35:47 +0800
    Ready:          True
    Restart Count:  0
    Liveness:       http-get https://:8443/ delay=30s timeout=30s period=10s #success=1 #failure=3
    Environment:    <none>
    Mounts:
      /certs from kubernetes-dashboard-certs (rw)
      /tmp from tmp-volume (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kubernetes-dashboard-token-zm8rg (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  kubernetes-dashboard-certs:
    Type:          HostPath (bare host directory volume)
    Path:          /etc/kubernetes/pki
    HostPathType:  Directory
  tmp-volume:
    Type:    EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:  
  kubernetes-dashboard-token-zm8rg:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  kubernetes-dashboard-token-zm8rg
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node-role.kubernetes.io/master:NoSchedule
                 node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:          <none>

可以看到实际上是node2在运行

我们再用node2的ip访问,此时就看到久违的页面了

选择令牌。把10.7步骤中找到的token粘贴进去,登录dashboard

由于在正式环境中,并不推荐使用NodePort的方式来访问Dashboard

 

11、集成Heapster

Heapster是容器集群监控和性能分析工具,天然的支持Kubernetes和CoreOS。

11.1、准备相应的yaml文件

Heapster支持多种储存方式,

  1. heapster
  2. influxdb
  3. grafana

Git clone

git clone https://github.com/Nedved-liao/kubernetes.1.30_CN/tree/master/heapster

官方 GitHub

https://github.com/kubernetes-retired/heapster

个人修改后GitHub

使用默认的yaml文件,   创建的pod日志一直报错如下:

E1028 07:39:05.011439       1 manager.go:101] Error in scraping containers from Kubelet:XX.XX.XX.XX:10255: failed to get all container stats from Kubelet URL "http://XX.XX.XX.XX:10255/stats/container/": Post http://XX.XX.XX.XX:10255/stats/container/: dial tcp XX.XX.XX.XX:10255:
 getsockopt: connection refused

经过googling后

https://brookbach.com/2018/10/29/Heapster-on-Kubernetes-1.11.3.html

参照里面的解析修改了部分官方给出的内容

heapster.yaml

- --source=kubernetes:https://kubernetes.default

- --source=kubernetes.summary_api:''?useServiceAccount=true&kubeletHttps=true&kubeletPort=10250&insecure=true

name: system:heapster

name: heapster

#并把k8s.gcr.io修改为国内镜像。

并新增了 heapster-role.yaml

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: heapster
rules:
- apiGroups:
  - ""
  resources:
  - pods
  - nodes
  - namespaces
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - extensions
  resources:
  - deployments
  verbs:
  - get
  - list
  - update
  - watch
- apiGroups:
  - ""
  resources:
  - nodes/stats
  verbs:
  - get

11.2、创建Pod

[root@K8s-master ~/kubernetes.1.30_CN/heapster]# ll
总用量 20
-rw-r--r-- 1 root root 2353 5月   1 2018 grafana.yaml
-rw-r--r-- 1 root root  269 1月  23 14:46 heapster-rbac.yaml
-rw-r--r-- 1 root root  368 1月  23 14:47 heapster-role.yaml
-rw-r--r-- 1 root root 1214 1月  23 14:45 heapster.yaml
-rw-r--r-- 1 root root 1005 5月   1 2018 influxdb.yaml
[root@K8s-master ~/kubernetes.1.30_CN/heapster]# kubectl create -f .

11.3、验证结果

稍等后,查看一下Pod的状态处于running:

[root@K8s-master ~/kubernetes.1.30_CN/heapster]# kubectl -n kube-system get po 
NAME                                   READY   STATUS    RESTARTS   AGE
coredns-86c58d9df4-xrz2j               1/1     Running   8          7d4h
coredns-86c58d9df4-z2lcj               1/1     Running   8          7d4h
etcd-k8s-master                        1/1     Running   3          7d4h
heapster-7c46fbf7cf-sffcg              1/1     Running   0          69m
kube-apiserver-k8s-master              1/1     Running   11         7d4h
kube-controller-manager-k8s-master     1/1     Running   14         7d4h
kube-flannel-ds-amd64-dwpcr            1/1     Running   12         7d4h
kube-flannel-ds-amd64-h2tm9            1/1     Running   10         7d4h
kube-flannel-ds-amd64-ssh2d            1/1     Running   10         7d4h
kube-proxy-4lr8q                       1/1     Running   3          7d4h
kube-proxy-7b22t                       1/1     Running   2          7d4h
kube-proxy-j6qkx                       1/1     Running   2          7d4h
kube-scheduler-k8s-master              1/1     Running   15         7d4h
kubernetes-dashboard-98b7c88bb-2fzlw   1/1     Running   0          20h
monitoring-grafana-5f5c6c4c4b-zmhkf    1/1     Running   0          69m
monitoring-influxdb-68df588786-zz9h5   1/1     Running   0          69m

再查看日志,确保没有问题

#查看heapster的日志,若有报错进行相对应的修复。influxdb,grafana日志此处省略
[root@K8s-master ~/kubernetes.1.30_CN/heapster]# kubectl -n kube-system logs --tail=200 heapster-7c46fbf7cf-sffcg
I0123 06:50:51.635860       1 heapster.go:72] /heapster --source=kubernetes.summary_api:''?useServiceAccount=true&kubeletHttps=true&kubeletPort=10250&insecure=true --sink=influxdb:http://monitoring-influxdb.kube-system.svc:8086
I0123 06:50:51.636028       1 heapster.go:73] Heapster version v1.4.2
I0123 06:50:51.637662       1 configs.go:61] Using Kubernetes client with master "https://10.96.0.1:443" and version v1
I0123 06:50:51.637859       1 configs.go:62] Using kubelet port 10250
I0123 06:50:52.102425       1 influxdb.go:278] created influxdb sink with options: host:monitoring-influxdb.kube-system.svc:8086 user:root db:k8s
I0123 06:50:52.103093       1 heapster.go:196] Starting with InfluxDB Sink
I0123 06:50:52.103158       1 heapster.go:196] Starting with Metric Sink
I0123 06:50:53.072648       1 heapster.go:106] Starting heapster on port 8082
I0123 06:51:12.521512       1 influxdb.go:241] Created database "k8s" on influxDB server at "monitoring-influxdb.kube-system.svc:8086"

确认无误后,刷新浏览器,查看到cpu使用率证明部署成功

 

 

12、集群拆卸

12.1、拆卸集群

首先处理各节点:

kubectl drain <node name> --delete-local-data --force --ignore-daemonsets
kubectl delete node <node name>

一旦节点移除之后,则可以执行如下命令来重置集群:
kubeadm reset

12.2、master处理节点

[root@K8s-master ~]# kubectl drain k8s-node1 --delete-local-data --force --ignore-daemonsets
node/k8s-node1 cordoned
WARNING: Ignoring DaemonSet-managed pods: kube-flannel-ds-amd64-r2qnb, kube-proxy-cvbbf
node/k8s-node1 drained
[root@K8s-master ~]# kubectl delete node k8s-node1
node "k8s-node1" deleted

12.3节点重置(node2省略)

[root@K8s-node1 ~]# kubeadm reset
[reset] WARNING: changes made to this host by 'kubeadm init' or 'kubeadm join' will be reverted.
[reset] are you sure you want to proceed? [y/N]: y
[preflight] running pre-flight checks
[reset] no etcd config found. Assuming external etcd
[reset] please manually reset etcd to prevent further issues
[reset] stopping the kubelet service
[reset] unmounting mounted directories in "/var/lib/kubelet"
[reset] deleting contents of stateful directories: [/var/lib/kubelet /etc/cni/net.d /var/lib/dockershim /var/run/kubernetes]
[reset] deleting contents of config directories: [/etc/kubernetes/manifests /etc/kubernetes/pki]
[reset] deleting files: [/etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/bootstrap-kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf]

The reset process does not reset or clean up iptables rules or IPVS tables.
If you wish to reset iptables, you must do so manually.
For example: 
iptables -F && iptables -t nat -F && iptables -t mangle -F && iptables -X

If your cluster was setup to utilize IPVS, run ipvsadm --clear (or similar)
to reset your system's IPVS tables.

[root@K8s-node1 ~]# iptables -F && iptables -t nat -F && iptables -t mangle -F && iptables -X

12.4、master重置

 

[root@K8s-master ~]# kubeadm reset
[reset] WARNING: changes made to this host by 'kubeadm init' or 'kubeadm join' will be reverted.
[reset] are you sure you want to proceed? [y/N]: y
[preflight] running pre-flight checks
[reset] no etcd config found. Assuming external etcd
[reset] please manually reset etcd to prevent further issues
[reset] stopping the kubelet service
[reset] unmounting mounted directories in "/var/lib/kubelet"
[reset] deleting contents of stateful directories: [/var/lib/kubelet /etc/cni/net.d /var/lib/dockershim /var/run/kubernetes]
[reset] deleting contents of config directories: [/etc/kubernetes/manifests /etc/kubernetes/pki]
[reset] deleting files: [/etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/bootstrap-kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf]

The reset process does not reset or clean up iptables rules or IPVS tables.
If you wish to reset iptables, you must do so manually.
For example: 
iptables -F && iptables -t nat -F && iptables -t mangle -F && iptables -X
[root@K8s-master ~]# iptables -F && iptables -t nat -F && iptables -t mangle -F && iptables -X

 

 

 

 

 

 

 

 

 

 

 

 

 

点赞

发表评论

电子邮件地址不会被公开。必填项已用 * 标注