Kubernetes集群搭建

献给在墙内尝试搭建最新版kubernetes集群的家伙

1. Overview

cluster_topology

2. 组件

OS:CentOS7

2.1 Master

Apiproxy (必要)

Scheduler (必要)

Controller Manager (必要)

Kubectl (非必要)

2.2 Node

Docker(必要)

Kubelet (必要)

Flannel(非必要)

Kube-Proxy (非必要)

Etcd (非必要)

3. Start Trip

3.1 下载Kubernetes(所有节点)

  1. 创建并进入目录
    mkdir ~/kubernetes && cd ~/kubernetes
  2. 下载Kubernetes.tar.gz (LINK)
    wget https://github.com/kubernetes/kubernetes/releases/download/v1.7.6/kubernetes.tar.gz
  3. 解压
    tar zxvf kubernetes.tar.gz
  4. 下载二进制包
    ./kubernetes/cluster/get-kube-binaries.sh
  5. 解压
    tar zxvf ./kubernetes/server/kubernetes-server-linux-amd64.tar.gz
  6. 添加环境变量
    cd ./kubernetes/server/bin
    export PATH=`pwd`:$PATH
  7. 检查兼容的etcd版本
    [root@localhost ~]# grep TAGS kubernetes/cluster/images/etcd/Makefile
    # 	[TAGS=2.2.1 2.3.7 3.0.17] [REGISTRY=gcr.io/google_containers] [ARCH=amd64] [BASEIMAGE=busybox] make (build|push)

    TAGS的值为兼容的版本,kubernetes1.7.6兼容etcd2.2.1,于是选择etcd2.2.1

3.2 搭建etcd集群

  1. 下载etcd2.2.1 (LINK)
    curl -L  https://github.com/coreos/etcd/releases/download/v2.2.1/etcd-v2.2.1-darwin-amd64.zip -o etcd-v2.2.1-darwin-amd64.zip
    unzip etcd-v2.2.1-darwin-amd64.zip
    cd etcd-v2.2.1-darwin-amd64
  2. 搭建集群
    ###将export中的ip改为对应机器的ip###
    #host1 ip 为192.168.46.31,执行以下命令
    export HostIP="192.168.46.31"
    nohup \
    ./etcd \
        -name=etcd0 \
        -listen-client-urls=http://0.0.0.0:2379 \
        -advertise-client-urls=http://${HostIP}:2379 \
        -listen-peer-urls=http://0.0.0.0:2380 \
        -initial-advertise-peer-urls=http://${HostIP}:2380 \
        -initial-cluster-token etcd-cluster-1 \
         -initial-cluster etcd0=http://192.168.46.31:2380,etcd1=http://192.168.46.32:2380,etcd2=http://192.168.46.33:2380 \
         -initial-cluster-state new \
    &
    
    #host2 ip为192.168.46.32,执行以下命令
    export HostIP="192.168.46.32"
    nohup \
    ./etcd \
        -name=etcd1 \
        -listen-client-urls=http://0.0.0.0:2379 \
        -advertise-client-urls=http://${HostIP}:2379 \
        -listen-peer-urls=http://0.0.0.0:2380 \
        -initial-advertise-peer-urls=http://${HostIP}:2380 \
        -initial-cluster-token etcd-cluster-1 \
         -initial-cluster etcd0=http://192.168.46.31:2380,etcd1=http://192.168.46.32:2380,etcd2=http://192.168.46.33:2380 \
         -initial-cluster-state new \
    &
    
    #host3 ip为192.168.46.33,执行以下命令
    export HostIP="192.168.46.33"
    nohup \
    ./etcd \
        -name=etcd2 \
        -listen-client-urls=http://0.0.0.0:2379 \
        -advertise-client-urls=http://${HostIP}:2379 \
        -listen-peer-urls=http://0.0.0.0:2380 \
        -initial-advertise-peer-urls=http://${HostIP}:2380 \
        -initial-cluster-token etcd-cluster-1 \
         -initial-cluster etcd0=http://192.168.46.31:2380,etcd1=http://192.168.46.32:2380,etcd2=http://192.168.46.33:2380 \
         -initial-cluster-state new \
    &
    
  3. 检查集群状态
    etcdctl member list
    etcdctl cluster-health
  4. 设置flannel配置
    etcdctl set /coreos.com/network/config '{"Network": "10.1.0.0/16","SubnetLen": 24,"SubnetMin": "10.1.0.0","SubnetMax": "10.1.9.255","Backend": {"Type":"vxlan"}}'

3.3 安装Flannel(每个node)

  1. 安装flannel
    wget https://github.com/coreos/flannel/releases/download/v0.8.0/flanneld-amd64 && chmod +x flanneld-amd64
  2. 启动flannel
    ./flanneld-amd64 --etcd-endpoints=http://192.168.46.31:2379
  3. 检查flannel运行状态
    [root@localhost ~]# cat /var/run/flannel/subnet.env
    FLANNEL_NETWORK=10.1.0.0/16
    FLANNEL_SUBNET=10.1.5.1/24
    FLANNEL_MTU=1450
    FLANNEL_IPMASQ=false

3.4 安装Docker(每个node)

  1. 安装Docker
    yum install docker -y
  2. 配置docker网络
    #修改/etc/systemd/system/docker.service文件信息如下
    #其实就是在docker的启动命令中添加--bip和--mtu两个参数,参数信息来自flannel获取的配置
    ...
    [Service]
    ...
    EnvironmentFile=-/run/flannel/subnet.env (新增此行)
    ...
    ExecStart=/usr/bin/dockerd ... \
        ... \
        --bip=${FLANNEL_SUBNET} --mtu=${FLANNEL_MTU} \ (新增此行)
        ...
    
  3. 重启docker并确认网段
    systemctl restart docker
    ip a
    #输出信息docker0的网段应当落入flannel归属的网段,即配置文件中/var/run/flannel/subnet.env的FLANNEL_SUBNET
  4. 加载gcr必要镜像
    wget https://github.com/zman2013/kubernetes-install/raw/master/docker-images/pause-amd64.tar
    docker load -i pause-amd64.tar
    #Loaded image: gcr.io/google_containers/pause-amd64:3.0

3.5 启动Apiserver(master节点)

nohup hyperkube apiserver \
    --etcd-servers=http://192.168.46.31:2379 \  #etcd集群地址
    --service-cluster-ip-range=10.1.0.0/16 \
    --storage-backend=etcd2 \
    --insecure-bind-address=0.0.0.0 \
    --admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,ResourceQuota \
    --v=4 > apiserver.log &

3.6 启动Scheduler(master节点)

nohup hyperkube scheduler --master=127.0.0.1:8080 > scheduler.log &  #apiserver的地址

3.7 启动Controller Manager(master节点)

nohup hyperkube controller-manager --master=127.0.0.1:8080 > controller-manager.log & #apiserver的地址

 3.8 启动Kubelet(每个Node)

nohup hyperkube kubelet \
    --api-servers=http://192.168.46.31:8080 \
    --hostname-override=192.168.46.32 \       #本机ip
    --cgroup-driver=systemd \
    --v=4  > kubelet.log &

 3.9 启动Kube-Proxy(每个Node)

nohup hyperkube kube-proxy --master=http://192.168.46.31:8080 > kube-proxy.log &

 

Q&A

1. Q: Error syncing pod 39d2bd89-a174-11e7-b855-0050568723c9, skipping: failed to “StartContainer” for “POD” with ErrImagePull: “image pull failed for gcr.io/google_containers/pause-amd64:3.0, this may be because there are no credentials on this request

A: 获取pause-amd64镜像失败。参考步骤:3.4中4加载gcr必要镜像。

 

LINKS

  1. Flannel
  2. Flanneld配置
  3. Etcd
  4. Kubernetes从零创建
  5. Kubernetes启动参数