CentOS 7.4
1 2 3 4 |
192.168.61.11 node1 192.168.61.12 node2 192.168.61.13 node3 |
TLS密钥和证书
这里部署的etcd集群使用TLS证书对证书通信进行加密,并开启基于CA根证书签名的双向数字证书认证。
下面介绍使用cfssl生成所需要的私钥和证书.
centos-mini版本的linux需要安装开发包:
yum groupinstall “Development Tools”
安装cfssl
cfssl是使用Go语言开发的工具,如果系统中安装了Go,可以使用直接go get
安装cfssl:
项目地址:https://github.com/cloudflare/cfssl
1 2 |
go get -u <a class="vglnk" href="http://github.com/cloudflare/cfssl/cmd/" rel="nofollow">github.com/cloudflare/cfssl/cmd/</a>... |
会在$GOPATH/bin
下安装cfssl, cfssjosn, mkbundle等工具。
CA证书和私钥
创建ca-config.json:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
<span class="p">{</span> <span class="nt">"signing"</span><span class="p">:</span> <span class="p">{</span> <span class="nt">"default"</span><span class="p">:</span> <span class="p">{</span> <span class="nt">"expiry"</span><span class="p">:</span> <span class="s2">"87600h"</span> <span class="p">},</span> <span class="nt">"profiles"</span><span class="p">:</span> <span class="p">{</span> <span class="nt">"frognew"</span><span class="p">:</span> <span class="p">{</span> <span class="nt">"usages"</span><span class="p">:</span> <span class="p">[</span> <span class="s2">"signing"</span><span class="p">,</span> <span class="s2">"key encipherment"</span><span class="p">,</span> <span class="s2">"server auth"</span><span class="p">,</span> <span class="s2">"client auth"</span> <span class="p">],</span> <span class="nt">"expiry"</span><span class="p">:</span> <span class="s2">"87600h"</span> <span class="p">}</span> <span class="p">}</span> <span class="p">}</span> <span class="p">}</span> |
ca-config.json中可以定义多个profile,分别设置不同的expiry和usages等参数。如上面的ca-config.json中定义了名称为frognew的profile,这个profile的expiry 87600h为10年,useages中:
signing
表示此CA证书可以用于签名其他证书,ca.pem中的CA=TRUE
server auth
表示TLS Server Authenticationclient auth
表示TLS Client Authentication
创建CA证书签名请求配置ca-csr.json:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
<span class="p">{</span> <span class="nt">"CN"</span><span class="p">:</span> <span class="s2">"frognew"</span><span class="p">,</span> <span class="nt">"key"</span><span class="p">:</span> <span class="p">{</span> <span class="nt">"algo"</span><span class="p">:</span> <span class="s2">"rsa"</span><span class="p">,</span> <span class="nt">"size"</span><span class="p">:</span> <span class="mi">2048</span> <span class="p">},</span> <span class="nt">"names"</span><span class="p">:</span> <span class="p">[</span> <span class="p">{</span> <span class="nt">"C"</span><span class="p">:</span> <span class="s2">"CN"</span><span class="p">,</span> <span class="nt">"ST"</span><span class="p">:</span> <span class="s2">"BeiJing"</span><span class="p">,</span> <span class="nt">"L"</span><span class="p">:</span> <span class="s2">"BeiJing"</span><span class="p">,</span> <span class="nt">"O"</span><span class="p">:</span> <span class="s2">"frognew"</span><span class="p">,</span> <span class="nt">"OU"</span><span class="p">:</span> <span class="s2">"cloudnative"</span> <span class="p">}</span> <span class="p">]</span> <span class="p">}</span> |
下面使用cfss生成CA证书和私钥:
1 2 |
cfssl gencert -initca ca-csr.json | cfssljson -bare ca |
1 2 3 |
ls ca-config.json ca.csr ca-csr.json ca-key.pem ca.pem |
ca-key.pem和ca.pem需要保存在一个安全的地方,后边会用到。
etcd证书和私钥
创建etcd证书签名请求配置etcd-csr.json:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
<span class="p">{</span> <span class="nt">"CN"</span><span class="p">:</span> <span class="s2">"frognew"</span><span class="p">,</span> <span class="nt">"hosts"</span><span class="p">:</span> <span class="p">[</span> <span class="s2">"127.0.0.1"</span><span class="p">,</span> <span class="s2">"192.168.61.11"</span><span class="p">,</span> <span class="s2">"192.168.61.12"</span><span class="p">,</span> <span class="s2">"192.168.61.13"</span><span class="p">,</span> <span class="s2">"node1"</span><span class="p">,</span> <span class="s2">"node2"</span><span class="p">,</span> <span class="s2">"node3"</span> <span class="p">],</span> <span class="nt">"key"</span><span class="p">:</span> <span class="p">{</span> <span class="nt">"algo"</span><span class="p">:</span> <span class="s2">"rsa"</span><span class="p">,</span> <span class="nt">"size"</span><span class="p">:</span> <span class="mi">2048</span> <span class="p">},</span> <span class="nt">"names"</span><span class="p">:</span> <span class="p">[</span> <span class="p">{</span> <span class="nt">"C"</span><span class="p">:</span> <span class="s2">"CN"</span><span class="p">,</span> <span class="nt">"ST"</span><span class="p">:</span> <span class="s2">"BeiJing"</span><span class="p">,</span> <span class="nt">"L"</span><span class="p">:</span> <span class="s2">"BeiJing"</span><span class="p">,</span> <span class="nt">"O"</span><span class="p">:</span> <span class="s2">"frognew"</span><span class="p">,</span> <span class="nt">"OU"</span><span class="p">:</span> <span class="s2">"cloudnative"</span> <span class="p">}</span> <span class="p">]</span> <span class="p">}</span> |
注意上面配置hosts字段中制定授权使用该证书的IP和域名列表,因为现在要生成的证书需要被etcd集群各个节点使用,所以这里指定了各个节点的IP和hostname。
下面生成etcd的证书和私钥:
1 2 3 4 5 |
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=frognew etcd-csr.json | cfssljson -bare etcd ls etcd* etcd.csr etcd-csr.json etcd-key.pem etcd.pem |
对生成的证书可以使用cfssl或openssl查看:
1 2 3 4 |
cfssl-certinfo -cert etcd.pem openssl x509 -noout -text -in etcd.pem |
安装etcd
将CA证书ca.pem, etcd秘钥etcd-key.pem, etcd证书etcd.pem拷贝到各节点的/etc/etcd/ssl目录中。
下载etcd二进制文件包:
1 2 |
wget <a class="vglnk" href="https://github.com/coreos/etcd/releases/download/v3.1.6/etcd-v3.1.6-linux-amd64.tar.gz" rel="nofollow">https://github.com/coreos/etcd/releases/download/v3.1.6/etcd-v3.1.6-linux-amd64.tar.gz</a> |
解压缩etcd-v3.1.6-linux-amd64.tar.gz,将其中的etcd和etcdctl两个可执行文件复制到各节点的/usr/bin目录。
在各节点创建etcd的数据目录:
1 2 |
mkdir -p /var/lib/etcd |
在每个节点上创建etcd的systemd unit文件/usr/lib/systemd/system/etcd.service,注意替换ETCD_NAME
和INTERNAL_IP
变量的值:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
export ETCD_NAME=node1 export INTERNAL_IP=192.168.61.11 cat > /usr/lib/systemd/system/etcd.service <<EOF [Unit] Description=etcd server After=network.target After=network-online.target Wants=network-online.target [Service] Type=notify WorkingDirectory=/var/lib/etcd/ EnvironmentFile=-/etc/etcd/etcd.conf ExecStart=/usr/bin/etcd \ --name ${ETCD_NAME} \ --cert-file=/etc/etcd/ssl/etcd.pem \ --key-file=/etc/etcd/ssl/etcd-key.pem \ --peer-cert-file=/etc/etcd/ssl/etcd.pem \ --peer-key-file=/etc/etcd/ssl/etcd-key.pem \ --trusted-ca-file=/etc/etcd/ssl/ca.pem \ --peer-trusted-ca-file=/etc/etcd/ssl/ca.pem \ --initial-advertise-peer-urls <a class="vglnk" href="https://%24%7Binternal_ip%7D:2380/" rel="nofollow">https://${INTERNAL_IP}:2380</a> \ --listen-peer-urls <a class="vglnk" href="https://%24%7Binternal_ip%7D:2380/" rel="nofollow">https://${INTERNAL_IP}:2380</a> \ --listen-client-urls <a class="vglnk" href="https://%24%7Binternal_ip%7D:2379%2Chttps//127.0.0.1:2379" rel="nofollow">https://${INTERNAL_IP}:2379,https://127.0.0.1:2379</a> \ --advertise-client-urls <a class="vglnk" href="https://%24%7Binternal_ip%7D:2379/" rel="nofollow">https://${INTERNAL_IP}:2379</a> \ --initial-cluster-token etcd-cluster-1 \ --initial-cluster node1=<a class="vglnk" href="https://192.168.61.11:2380%2Cnode2%3Dhttps//192.168.61.12:2380,node3=https://192.168.61.13:2380" rel="nofollow">https://192.168.61.11:2380,node2=https://192.168.61.12:2380,node3=https://192.168.61.13:2380</a> \ --initial-cluster-state new \ --data-dir=/var/lib/etcd Restart=on-failure RestartSec=5 LimitNOFILE=65536 [Install] WantedBy=multi-user.target EOF |
- 上面在启动参数中指定了etcd的工作目录和数据目录是/var/lib/etcd
--cert-file
和--key-file
分别指定etcd的公钥证书和私钥--peer-cert-file
和--peer-key-file
分别指定了etcd的Peers通信的公钥证书和私钥。--trusted-ca-file
指定了客户端的CA证书--peer-trusted-ca-file
指定了Peers的CA证书--initial-cluster-state new
表示这是新初始化集群,--name
指定的参数值必须在--initial-cluster
中
启动etcd
在各节点上启动etcd:
1 2 3 4 5 |
systemctl daemon-reload systemctl enable etcd systemctl start etcd systemctl status etcd |
检查集群是否健康,在任一节点执行:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
etcdctl \ --ca-file=/etc/etcd/ssl/ca.pem \ --cert-file=/etc/etcd/ssl/etcd.pem \ --key-file=/etc/etcd/ssl/etcd-key.pem \ --endpoints=<a class="vglnk" href="https://node1:2379%2Chttps//node2:2379,https://node3:2379" rel="nofollow">https://node1:2379,https://node2:2379,https://node3:2379</a> \ cluster-health 2017-04-24 19:53:40.545148 I | warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated 2017-04-24 19:53:40.546127 I | warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated member 4f2f99d70000fc19 is healthy: got healthy result from <a class="vglnk" href="https://192.168.61.12:2379/" rel="nofollow">https://192.168.61.12:2379</a> member 99a756f799eb4163 is healthy: got healthy result from <a class="vglnk" href="https://192.168.61.11:2379/" rel="nofollow">https://192.168.61.11:2379</a> member a9aff19397de2e4e is healthy: got healthy result from <a class="vglnk" href="https://192.168.61.13:2379/" rel="nofollow">https://192.168.61.13:2379</a> cluster is healthy |
确保输出cluster is healthy
的信息。
0 Comments