kubeadm join命令无法加入HA k8s主群集

Question

我正在AWS上设置kubernetes 1.14 HA。

我正在使用Stacked etcd拓扑结构，包含3个主节点和5个工作节点。我能够在第一个主节点上运行kubeadm init命令，并在第二个主节点上运行kubeadm join命令。我看到两者都成功并能够使用kubectl get nodes命令列出。

但是，相同的kubeadm join命令在第3个主节点上失败，命令失败。

[mark-control-plane] Marking the node ip-10-169-50-168 as control-plane by adding the label "node-role.kubernetes.io/master=''"
[mark-control-plane] Marking the node ip-10-169-50-168 as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[kubelet-check] Initial timeout of 40s passed.
error execution phase control-plane-join/mark-control-plane: error applying control-plane label and taints: timed out waiting for the condition

NAME                           STATUS     ROLES    AGE     VERSION
ip-XX-XX-XX-XX                 Ready      master   74m     v1.14.0
ip-XX-XX-XX-XX                 Ready      master   70m     v1.14.0

当我检查docker日志时，我看到第三个节点上的etcd能够加入集群，之后连接被拒绝。以下是日志。

2019-04-10 17:44:59.409307 I | etcdserver/membership: added member 8ee1c831d170ef7f [https://XX.XX.XX.XX:2380] to cluster bedd10c18e149ae2
2019-04-10 17:44:59.409447 N | etcdserver/membership: set the initial cluster version to 3.3
2019-04-10 17:44:59.409506 I | etcdserver/api: enabled capabilities for version 3.3
2019-04-10 17:44:59.414195 I | rafthttp: established a TCP streaming connection with peer aa2e639fdfb57216 (stream Message reader)
2019-04-10 17:44:59.426797 I | etcdserver/membership: added member aa2e639fdfb57216 [https://XX.XX.XX.XX:2380] to cluster bedd10c18e149ae2
2019-04-10 17:44:59.428027 I | etcdserver/membership: added member 4d402309132b25d3 [https://XX.XX.XX.XX:2380] to cluster bedd10c18e149ae2
2019-04-10 17:44:59.436291 I | etcdserver: 4d402309132b25d3 initialzed peer connection; fast-forwarding 8 ticks (election ticks 10) with 2 active peer(s)
2019-04-10 17:44:59.448880 I | etcdserver: published {Name:ip-10-169-50-178 ClientURLs:[https://XX.XX.XX.XX:2379]} to cluster bedd10c18e149ae2
2019-04-10 17:44:59.448959 I | embed: ready to serve client requests
2019-04-10 17:44:59.449247 I | embed: ready to serve client requests
2019-04-10 17:44:59.450469 I | embed: serving client requests on XX.XX.XX.XX:2379
2019-04-10 17:44:59.450817 I | embed: serving client requests on 127.0.0.1:2379
2019-04-10 17:45:01.533145 I | embed: rejected connection from "127.0.0.1:46992" (error "EOF", ServerName "")
2019-04-10 17:45:03.146800 I | embed: rejected connection from "XX.XX.XX.XX:48888" (error "remote error: tls: bad certificate", ServerName "")
2019-04-10 17:45:03.788293 I | raft: 4d402309132b25d3 [logterm: 8, index: 892, vote: 0] ignored MsgVote from 8ee1c831d170ef7f [logterm: 8, index: 892] at term 8: lease is not expired (remaining ticks: 10)
2019-04-10 17:45:04.312725 W | wal: sync duration of 1.985619012s, expected less than 1s
2019-04-10 17:45:05.588410 I | raft: 4d402309132b25d3 [logterm: 8, index: 892, vote: 0] ignored MsgVote from 8ee1c831d170ef7f [logterm: 8, index: 892] at term 8: lease is not expired (remaining ticks: 3)
2019-04-10 17:45:05.589745 I | raft: 4d402309132b25d3 [term: 8] received a MsgApp message with higher term from 8ee1c831d170ef7f [term: 10]
2019-04-10 17:45:05.589762 I | raft: 4d402309132b25d3 became follower at term 10
2019-04-10 17:45:05.589781 I | raft: raft.node: 4d402309132b25d3 changed leader from aa2e639fdfb57216 to 8ee1c831d170ef7f at term 10
proto: no coders for int
proto: no encoder for ValueSize int [GetProperties]
2019-04-10 17:50:43.108887 I | mvcc: store.index: compact 978
2019-04-10 17:50:43.110211 I | mvcc: finished scheduled compaction at 978 (took 960.176µs)

你可以分享一些指示，来解决这个问题吗？

Answer 1

我能够解决这个问题。我必须通过apiServer.authorization-mode作为Node,RBAC。

Answer 2

拒绝来自“XX.XX.XX.XX：48888”的连接（错误“远程错误：tls：错误证书”

我开始研究第三位大师的证书。将它与来自第二个主服务器的证书进行比较，第二个主服务器是成功引导到集群中的证书 最终你应该有这个文件列表：

希望你没有错过这个instruction的一步