Teach you the detailed installation steps of k8s cluster (the most complete big data tutorial in History)

Kubernetes It's a portable , Scalable open source platform , For managing containerized workloads and services , Promotes declarative configuration and Automation . It has a huge and rapidly growing ecosystem .Kubernetes Service for , Support and tools are widely used .

Google stay 2014 In open source Kubernetes project .Kubernetes Will be more than 15 Year of Google Experience in mass production workloads is combined with the best ideas and practices in the community .

1. Installation environment

Be careful ： The following steps are in root Permission to operate

1.1 Server preparation

Three systems are Centos7.5 Ali server

The following configuration of the machine is required

1.1.1 System environment

Ubuntu16.04 +
Debian9 +
CentOS 7
Red hat enterprise edition Linux（RHEL）7
HypriotOSv1.0.1 +
Fedora25 +
FlatcarContainer Linux（ Tested 2512.3.0）

3.2CPU Or more 2.2G Memory or more

4. Need to be able to connect to the Internet

5. Each shareholder needs to have a unique host name

6. Need to open some ports

7. Need to exchange functions

1.1.2 Open ports ：

Master

port	purpose
6443*	Kubernetes API The server
2379-2380	kubelet etcd Server client API
10250	kubelet API
10251	Kube-scheduler
10252	Kube-controller-manager

Worker node

port	purpose
10250	kubelet API
30000-32767	NodePort Services

2. install Docker

2.1 install Docker, Setup repository .

[[email protected] ~]# sudo yum install -y yum-utils
[[email protected] ~]# sudo yum-config-manager \
--add-repo \
https://download.docker.com/linux/centos/docker-ce.repo

2.2, Enable Repository , Turn off the parameter if necessary –enable Replace –disable

[[email protected] ~]# sudo yum-config-manager --enable docker-ce-nightly

2.3, Enable test channel , Turn off the parameter if necessary –enable Replace –disable

[[email protected] ~]# sudo yum-config-manager --enable docker-ce-test

2.4, install Docker engine

[[email protected] ~]# sudo yum install docker-ce docker-ce-cli containerd.io

2.5, start-up Docker

[[email protected] ~]# sudo systemctl start docker

2.6, testing Docker Is it installed correctly

[[email protected] ~]# sudo docker run hello-world
Unableto find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
0e03bdcc26d7: Pull complete
Digest:sha256: 1a523af650137b8accdaed439c17d684df61ee4d74feac151b5b337bd29e7eec
Status: Downloaded newer image for hello-world:latest
Hello from Docker!
This message shows that your installation appears to be working correctly.

Docker Installation website ：

https://docs.docker.com/engine/install/centos/#install-using-the-repository

3. Use tools to install Kubernetes

install Kubernetes There are three tools that clusters can use, namely Kubeadm、Kops、Kubespray. Here we use Kubeadm To install .

3.1 install Master

3.1.1 close swap

[[email protected] ~]# swapoff -a

3.1.2 To configure yum Source

[[email protected] ~]# cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpghttps://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

3.1.3 install kubeadm Related tools

[[email protected] ~]# yum install -y kubelet kubeadm kubectl

3.1.4 After go to init.default Initialization file

[[email protected] ~]# kubeadm config print init-defaults >init.default.yaml

3.1.5 edit init.default file , Modify the warehouse , modify pod Address range of

[[email protected] ~]# vim init.default.yaml
imageRepository:registry.aliyuncs.com/google_containers
kind:ClusterConfiguration
kubernetesVersion:v1.20.0
networking:
podSubnet:"192.168.0.0/16"
dnsDomain: cluster.local
serviceSubnet: 10.96.0.0/12
scheduler:{
}

3.1.6 download Kubernetes The relevant image of

[[email protected] ~]# kubeadm config images pull --config=init.default.yaml
[config/images]Pulled registry.aliyuncs.com/google_containers/kube-apiserver:v1.20.0
[config/images]Pulled registry.aliyuncs.com/google_containers/kube-controller-manager:v1.20.0
[config/images]Pulled registry.aliyuncs.com/google_containers/kube-scheduler:v1.20.0
[config/images]Pulled registry.aliyuncs.com/google_containers/kube-proxy:v1.20.0
[config/images]Pulled registry.aliyuncs.com/google_containers/pause:3.2
[config/images]Pulled registry.aliyuncs.com/google_containers/etcd:3.4.13-0
[config/images]Pulled registry.aliyuncs.com/google_containers/coredns:1.7.0

3.1.7 start-up Kubelet, And set the power on self start , Set up cgroupDriver

[[email protected] ~]# vim /etc/docker/daemon.json
{

"exec-opts":["native.cgroupdriver=systemd"]
}
[[email protected] ~]# vim /usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf
Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf--kubeconfig=/etc/kubernetes/kubelet.conf --cgroup-driver=systemd"
[[email protected] ~]# systemctl daemon-reload
[[email protected] ~]# systemctl restart docker
[[email protected] ~]# systemctl enable docker
[[email protected] ~]# systemctl enable kubelet

3.1.8 Kubeadm init Command to initialize the cluster set , Cluster settings first –pod-network-

cidr=192.168.0.0/16 Parameters , Turn off network functions
[[email protected] ~]#kubeadm init --image-repositoryregistry.aliyuncs.com/google_containers --kubernetes-version=v1.20.0--pod-network-cidr=192.168.0.0/16

3.1.9 Carry out orders , Finally, the installation is successful , Write down the token

kubeadmjoin 172.26.64.121:6443 --token c4r8zo.38zrpieopx6l51re \
--discovery-token-ca-cert-hashsha256:5fae9d62bf7d6e7a7759784aa8585103b82e5a2368ab5e11e2bca8ede6187c8a

3.1.10 Follow the picture tips , establish k8s user , Copy the configuration file to ordinary users . If it is root Then export

[[email protected] software]# useradd k8s
[[email protected] software]# passwd k8s
[[email protected] ~]# usermod -aG docker k8s
[[email protected] ~]# vim /etc/sudoers
k8s ALL=(ALL) ALL
[[email protected] ~]# su k8s
[[email protected] ~]$ mkdir -p $HOME/.kube
[[email protected] ~]$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
[[email protected] ~]$ sudo chown $(id -u):$(id -g) $HOME/.kube/config
[[email protected] ~]$ exit
[[email protected] ~]# export KUBECONFIG=/etc/kubernetes/admin.conf

3.1.11 This is the time Master Installed , But the lack of NODE, And there is no container network function , Verify that the installation was successful ：

[[email protected] ~]# kubectl get -n kube-system configmap
NAME DATA AGE
coredns 1 22m
extension-apiserver-authentication 6 22m
kube-proxy 2 22m
kube-root-ca.crt 1 22m
kubeadm-config 2 22m
kubelet-config-1.20 1 22m

3.2 install Node To join the cluster

3.2.1 install Node Front work and installation Master The same should be installed Docker, Set power on self start

3.2.2 installed Docker Then install the same kubeadm Related tools

[[email protected] ~]#cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpghttps://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
[[email protected] ~]# yum install -y kubelet kubeadmkubectl
[[email protected] ~]#cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpghttps://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
[[email protected] ~]# yum install -y kubelet kubeadmkubectl
[[email protected] ~]# vim /etc/docker/daemon.json
{

"exec-opts":["native.cgroupdriver=systemd"]
}
[[email protected] ~]# vim /usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf
Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf--kubeconfig=/etc/kubernetes/kubelet.conf --cgroup-driver=systemd"
[[email protected] ~]# systemctl daemon-reload
[[email protected] ~]# systemctl enable docker.service
[[email protected] ~]# systemctl restart docker
[[email protected] ~]# systemctl enable kubelet
[[email protected] ~]# vim /etc/docker/daemon.json
{

"exec-opts":["native.cgroupdriver=systemd"]
}
[[email protected] ~]# vim /usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf
Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf--kubeconfig=/etc/kubernetes/kubelet.conf --cgroup-driver=systemd"
[[email protected] ~]# systemctl daemon-reload
[[email protected] ~]# systemctl enable docker.service
[[email protected] ~]# systemctl restart docker
[[email protected] ~]# systemctl enable kubelet

3.2.3 Join in Master, establish join-config.ymal.apiServer by Master Address ,token For the above Master Generate the record after creation token

[[email protected] ~]# vim join-config.ymal
apiVersion:kubeadm.k8s.io/v1beta2
kind:JoinConfiguration
discovery:
bootstrapToken:
apiServerEndpoint: 172.26.64.121:6443
token: c4r8zo.38zrpieopx6l51re
unsafeSkipCAVerification: true
tlsBootstrapToken: c4r8zo.38zrpieopx6l51re
[[email protected] ~]# kubeadm join --config join-config.ymal

3.2.4 Prompt success ,103 Do the same , distribution join-config.ymal, perform join command

[[email protected] ~]# scp join-config.ymal 172.26.64.120:/root/
[[email protected] ~]# kubeadm join --config join-config.ymal

3.3 Install network plug-ins

3.3.1 Check the status , At this point, you can see that the nodes already have , But it's all NotReady state , The reason is that... Is not installed CNI The network plugin

[[email protected] ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s101 NotReady control-plane,master 78m v1.20.1
k8s102 NotReady <none> 5m v1.20.1
k8s103 NotReady <none> 2m38s v1.20.1

3.3.2 install CNI The network plugin , choice weave plug-in unit

[[email protected] ~]# docker pull quay.io/coreos/flannel:v0.9.1-amd64
[[email protected] ~]# mkdir -p /etc/cni/net.d/
[[email protected] ~]# cat <<EOF> /etc/cni/net.d/10-flannel.conf
{
"name":"cbr0","type":"flannel","delegate":{
"isDefaultGateway": true}}
EOF
[[email protected] ~]# mkdir /usr/share/oci-umount/oci-umount.d -p
[[email protected] ~]# mkdir /run/flannel/
[[email protected] ~]# kubectl apply -f"https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64| tr -d '\n')"

3.3.3 Verify that the cluster is installed successfully . install weave It has something to do with node network speed , It could be very slow , Need to wait

[[email protected] ~]# kubectl get pods --all-namespaces

Cluster installation succeeded . If the installation fails halfway, you can use... If you want to reinstall kubeadm reset Command reset , Install again .

4 To configure Docker Mirror to accelerate

4.1 First log in to Alibaba cloud services , Search container image service

4.2 After entering the image service, click image Accelerator

4.3 Follow the website tips , Configure every machine with docker Mirror to accelerate

[[email protected] root]$ vim /etc/docker/daemon.json
{

"registry-mirrors":["https://bl562v6z.mirror.aliyuncs.com"],
"exec-opts":["native.cgroupdriver=systemd"]
}
[[email protected] root]$ sudo systemctl daemon-reload
[[email protected] root]$ sudo systemctl restart docker
[[email protected] ~]# vim /etc/docker/daemon.json
{

"registry-mirrors":["https://bl562v6z.mirror.aliyuncs.com"],
"exec-opts":["native.cgroupdriver=systemd"]
}
[[email protected] root]$ sudo systemctl daemon-reload
[[email protected] root]$ sudo systemctl restart docker
{

"registry-mirrors":["https://bl562v6z.mirror.aliyuncs.com"],
"exec-opts":["native.cgroupdriver=systemd"]
}
[[email protected] root]$ sudo systemctl daemon-reload
[[email protected] root]$ sudo systemctl restart docker

5 To configure k8s Mirror pull

5.1 According to Alibaba cloud account docker（ Your Alibaba cloud account ）, If the login is successful, a message will be generated in the root directory of the user .docker Contents and config.json Authentication key

[[email protected] ~]# docker login --username=lzt_otzregistry.cn-zhangjiakou.aliyuncs.com
[[email protected] ~]# docker login --username=lzt_otzregistry.cn-zhangjiakou.aliyuncs.com
[[email protected] ~]# docker login --username=lzt_otzregistry.cn-zhangjiakou.aliyuncs.com

5.2 k8s Pulling an image does not read by default docker The authentication key needs to be configured , According to the tips on the official website https://kubernetes.io/docs/concepts/containers/images/#using-a-private-regist. take docker Copy your authentication key to k8s Next

[[email protected] ~]# cd ~
[[email protected] ~]# cp .docker/config.json /var/lib/kubelet/
[[email protected] ~]# systemctl restart kubelet
[[email protected] ~]# cp .docker/config.json /var/lib/kubelet/
[[email protected] ~]# systemctl restart kubelet
[[email protected] ~]# cp .docker/config.json /var/lib/kubelet/
[[email protected] ~]# systemctl restart kubelet

6、 Simple tasks try

Use K8s function MySql

6.1 To write MySql RC（Replication Controller） file , Pay attention to the space alignment （ Very important ）

[[email protected]~]# su k8s
[[email protected] root]$ cd ~
[[email protected] ~]$ vim mysql-rc.yaml
apiVersion:v1
kind:ReplicationController # Copy controller RC
metadata:
name: mysql #RC The name of , Globally unique
spec:
replicas: 1 #Pod Expected number of copies
selector:
app: mysql # Fit for purpose Pod Have this tag
template: # Create... From template Pod Copy of （ example ）
metadata:
labels:
app: mysql #Pod Copy owned tags , Corresponding RC Of Selector
spec:
containers: #Pod The definition part of the content container
- name: mysql # Container name
image: docker.io/library/mysql:5.7 # The container corresponds to Docker Image
ports:
- containerPort: 3306 # Port number the container application listens on
env:
- name: MYSQL_ROOT_PASSWORD
value: "123456"

6.2 Publish to Kubernetes In the cluster

[[email protected] ~]$ kubectl create -f mysql-rc.yaml
replicationcontroller/mysqlcreated

6.3 Check out the RC

[[email protected] ~]$ kubectl get rc
NAME DESIRED CURRENT READY AGE
mysql 1 1 0 76s

6.4 see Pod The creation of

[[email protected] ~]$ kubectl get pods
NAME READY STATUS RESTARTS AGE
mysql-82pvs 1/1 Running 0 62s

6.5 Check the details , You can see that the container is created in k8s102 On the machine , therefore 102 Upper docker Image acceleration must be configured , otherwise mysql The mirror image will not be pulled down .

[[email protected] ~]$ kubectl describe pod mysql-82pvs

6.6 Came to k8s102 machine , Check container details , There will be two mysql Related containers

[[email protected] ~]# docker ps |grep mysql

6.7 Create associated Kubernets Service File associations MySql.

[[email protected] ~]$ vim mysql-svc.yaml
apiVersion:v1
kind:Service # Table name is Kubernetes Service
metadata:
name: mysql #Service Globally unique name of
spec:
type: NodePort
ports:
- port: 3306 #Service Provide the port number of the server
nodePort: 30001 # Out of pile exposed port
selector: #Service Corresponding Pod Have the tags defined here
app: mysql
[[email protected] ~]$ kubectl create -f mysql-svc.yaml
service/mysqlcreated

6.8 See created Service

[[email protected] ~]$ kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 24h
mysql NodePort 10.102.49.161 <none> 3306:30001/TCP 7s

6.9 Access... Through client tools k8s101 30001 port , visit Mysql. This way k8s Deploy MySql It's done.

7、 Dynamic storage management NFS

[[email protected] ~]# yum -y install nfs-utils rpcbind
[[email protected] ~]# yum -y install nfs-utils rpcbind
[[email protected] ~]# yum -y install nfs-utils rpcbind
[[email protected] ~]# systemctl start rpcbind.service
[[email protected] ~]# systemctl start nfs
[[email protected] ~]# systemctl enable rpcbind.service
[[email protected] ~]# systemctl enable nfs
[[email protected] ~]# systemctl start rpcbind.service
[[email protected] ~]# systemctl start nfs
[[email protected] ~]# systemctl enable rpcbind.service
[[email protected] ~]# systemctl enable nfs
[[email protected] ~]# systemctl start rpcbind.service
[[email protected] ~]# systemctl start nfs
[[email protected] ~]# systemctl enable rpcbind.service
[[email protected] ~]# systemctl enable nfs
[[email protected] ~]# mkdir /data/nfs -p
[[email protected] ~]# chown nfsnobody.nfsnobody /data/nfs
[[email protected] ~]# mkdir /data/nfs -p
[[email protected] ~]# chown nfsnobody.nfsnobody /data/nfs
[[email protected] ~]# mkdir /data/nfs -p
[[email protected] ~]# chown nfsnobody.nfsnobody /data/nfs

Master Address

[[email protected] ~]# cat>>/etc/exports<<EOF
/data/nfs172.26.64.121/20(rw,sync,no_root_squash,no_all_squash)
EOF
[[email protected] ~]# mkdir nfs
[[email protected] ~]# cd nfs/
[[email protected]]# wget https://github.com/kubernetes-retired/external-storage/tree/master/nfs-client/deploy/rbac.yaml
[[email protected] nfs]# wgethttps://github.com/kubernetes-retired/external-storage/tree/master/nfs-client/deploy/class.yaml
[[email protected] nfs]# wget https://github.com/kubernetes-retired/external-storage/tree/master/nfs-client/deploy/deployment.yaml
[[email protected] nfs]# su k8s
[[email protected] nfs]$ kubectl apply -f class.yaml
[[email protected] nfs]$ kubectl apply -f rbac.yaml

7.1 Modify the download deoplyment.yaml

[[email protected] nfs]$ vim deployment.yaml
apiVersion:apps/v1
kind:Deployment
metadata:
name: nfs-client-provisioner
labels:
app: nfs-client-provisioner
# replace with namespace where provisioner isdeployed
namespace: default
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app: nfs-client-provisioner
template:
metadata:
labels:
app: nfs-client-provisioner
spec:
serviceAccountName:nfs-client-provisioner
containers:
- name: nfs-client-provisioner
image: registry.cn-zhangjiakou.aliyuncs.com/my-bonc/nfs-client-provisioner:latest
volumeMounts:
- name: nfs-client-root
mountPath: /persistentvolumes
env:
- name: PROVISIONER_NAME
value: fuseim.pri/ifs
- name: NFS_SERVER
value:172.26.64.121 #nfs Service IP K8S101 private ip
- name: NFS_PATH
value: /data/nfs #nfs Service mount Directory
volumes:
- name: nfs-client-root
nfs:
server: 172.26.64.121 #nfs Service IP k8s101
path: /data/nfs #nfs Service mount Directory

7.2 Import deploy.yaml

[[email protected] nfs]$ kubectl create -f deployment.yaml

see

[[email protected] nfs]$ kubectl get sc
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
managed-nfs-storage fuseim.pri/ifs Delete Immediate false 5m4s
[[email protected] nfs]$ kubectl get pods
NAME READY STATUS RESTARTS AGE
nfs-client-provisioner-7859c747f5-p82js 1/1 Running 0 31s
[[email protected] nfs]$ sudo vim /etc/kubernetes/manifests/kube-apiserver.yaml
- --feature-gates=RemoveSelfLink=false # stay command Add this content to

7.3 Test creation pvc

[[email protected] nfs]$ vim test-pvc.yaml
kind:PersistentVolumeClaim
apiVersion:v1
metadata:
name: test-claim
annotations:
volume.beta.kubernetes.io/storage-class:"managed-nfs-storage"
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Mi

7.4 Import

[[email protected] nfs]$ kubectl create -f test-pvc.yaml

7.5 see , Automatically created pvcpv

[[email protected] nfs]$ kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
test-claim Bound pvc-a22d6cad-f7e1-4b38-bcc3-7099d7a964b8 1Mi RWX managed-nfs-storage 29s
[[email protected] nfs]$ kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pvc-a22d6cad-f7e1-4b38-bcc3-7099d7a964b8 1Mi RWX Delete Bound default/test-claim managed-nfs-storage 83s