KUBERNETES
Ingress-Controller Troubleshooting
下午有个集群要做升级验证， 同事把developer-center的namespaces下的应用都删除重建了，后来发现paas平台打不开，查询nginx日志，应该是nginx-ingress出了问题 首先查看ingress-nginx-cotroller日志：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
[root@dciuap1 nginx]# kubectl logs -n ingress-nginx nginx-ingress-controller-qdfq5 ------------------------------------------------------------------------------- NGINX Ingress controller Release: 0.26.1 Build: git-2de5a893a Repository: https://github.com/kubernetes/ingress-nginx nginx version: openresty/1.15.8.2 。。。 。。。 W0325 00:28:50.153058 6 controller.go:811] Error obtaining Endpoints for Service "developer-center/k8s-api": no object matching key "developer-ce W0325 00:28:50.153070 6 controller.go:811] Error obtaining Endpoints for Service "developer-center/servmeta": no object matching key "developer-c W0325 00:28:50.153080 6 controller.go:811] Error obtaining Endpoints for Service "developer-center/runtime-log": no object matching key "develope W0325 00:28:50.153095 6 controller.go:811] Error obtaining Endpoints for Service "developer-center/res-remote-manager": no object matching key "d local store W0325 00:28:50.153108 6 controller.go:811] Error obtaining Endpoints for Service "developer-center/res-pool-manager": no object matching key "deval store I0325 00:28:50.365637 6 controller.go:150] Backend successfully reloaded. W0325 00:28:53.486108 6 controller.go:920] Service "developer-center/app-apply" does not have any active Endpoint. W0325 00:28:53.486154 6 controller.go:920] Service "developer-center/aiops" does not have any active Endpoint. W0325 00:28:53.486175 6 controller.go:920] Service "developer-center/confcenter" does not have any active Endpoint. W0325 00:28:53.486189 6 controller.go:920] Service "developer-center/collect" does not have any active Endpoint. |
可以修改deployment，使用 flag –v=XX来修改日志级别
|
1 2 3 4 5 6 7 |
$ kubectl get deploy -n <namespace-of-ingress-controller> NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE default-http-backend 1 1 1 1 35m nginx-ingress-controller 1 1 1 1 35m $ kubectl edit deploy -n <namespace-of-ingress-controller> nginx-ingress-controller # Add --v=X to "- args", where X is an integer |
–v=2 shows details using diff about the changes in the configuration in nginx –v=3 shows details about the service, Ingress rule, endpoint changes and it dumps the nginx configuration in JSON format –v=5 configures NGINX in debug mode 到api server的认证逻辑：
|
1 2 3 4 5 |
+-------------+ service +------------+ | | authentication | | + apiserver +<-------------------+ ingress | | | | controller | +-------------+ +------------+ |
得确定下看是否是服务认证或kubeconfig导致的问题 ingress需要从 apiserver获得信息，是需要认证的，有两种认证方式： Service Account: This Read more…