前言
根据 kubernetes 中 Service 的定义和了解我们知道,如果 kubernetes 中的 pod 中容器想要对外部用户提供服务就需要将 pod 服务暴露至外部,让用户可以访问,而 Service 的表现形式为 ip 地址和端口号(ClusterIP:port)&(NodePort)即工作在四层 TCP/IP 层只能够通过 ip + 端口的方式访问,而在 ingress 当中可以使用七层协议(HTTP/HTTPS)的服务方式来实现负载均衡以及 k8s 集群内部服务和客户端之间的通信(既可以通过域名和主机名的方式),接下来将详细讲解 ingress-nginx 工作机制及概念;
一、ingress 是什么?
ingress 在 kubernetes 中是一个具有七层协议(HTTP/HTTPS)的路由转发机制,通过策略以及结合具体提供转发服务的 Ingress Controller 来实现基于灵活的 ingress 策略定义的服务路由功能;
二、ingress-nginx 概念
在 k8s 集群当中可以通过 Service 来对外暴露 k8s 集群当中的 pod 服务,从而使得外部请求能够访问 k8s 集群当中的 pod,而 Service 有多种暴露方式,常用的有 ClusterIP、NodePort.. ,如下通过 Service 和 ingress 做对比来更加了解 ingress-nginx 的概念:
第一种暴露方式 clusterIP 针对集群内部的端口访问,通过标签的方式将 pod 和 svc 进行关联,访问 svc 时 svc 通过策略来将请求发送至各个 pod ,只限于集群内部访问;
第二种暴露方式就是 NodePort ,该方式是基于 ClusterIP 暴露端口方式之上对集群外部进行暴露访问的一种方法,默认端口是从 30000 开始,此时集群外部就可以通过 master节点ip + 端口的方式对集群内部的 pod 服务进行访问,请求会先到达 NodePort 暴露的 ip + 端口,随后 NodePort 会和集群内的 ClusterIP 进行映射从而使得外部服务访问集群内的 pod 资源;
第三种暴露方式就是 ingress 方式,该访问方式也是基于 ClusterIP 之上建立一个暴露方式, nodeport 只能够暴露 ip+端口(四层代理) 而 ingress-nginx 可以使用域名(七层代理)的方式对外暴露服务,ingress-nginx 就相当于是一个 nginx 服务的反向代理,当创建 ingress-nginx 之后会创建一个 nginx 服务(ingress-nginx 的 pod 容器),其中 Pod 中容器 nginx.conf 配置文件则记录的是对外暴露的域名以及监听端口是都是通过 ingress 的配置自动添加的,使用 location 中的 proxy-pass 来代理 k8s 集群中的 cluster 暴露的 ip,当外部请求访问到 ingress 暴露的域名时会反向代理到对应匹配的 svc 从而实现服务的访问;
三、 ingress-nginx 工作流程
如下图,如果管理员修改了 ingress 配置则 ingress 会通过 kubernetes API Server 来监听修改的配置动作,API 会和 Store(协程)建立连接,由 Store 来监听资源变化的时间(informer 模式),最终会判断资源是否发生变化,当发生变化之后会将变更的资源写入至更新事件当中(updateChannel) 之后主程(NginxController) 会读取 updateChannel 当中的更新事件加载到主程(NginxController)并向同步队列(syncQueue)追加同步任务,之后同步队列协程(syncQueue)会拉取同步队列的信息,还有一种情况就是获取最新的 ingress/service 配置 nginx 服务必须的信息,也会加载到协程(syncQueue)当中,之后会对这些收集的信息资源进行判断是否需要 reload ,当有新配置时会触发 reload 重写 nginx 配置文件并构造 POST 数据,向 Lua Server 发送更新请求,如果判断不需要重载则直接构造 POST 数据,并向 LuaServer 发送更新请求;
四、Ingress-nginx 部署案例
1、创建 IngressContainer
[root@kube-master~][root@kube-master~][root@kube-masteringress]---apiVersion: v1kind: Namespacemetadata:name: nginx-ingress
---apiVersion: v1kind: ServiceAccountmetadata:name: nginx-ingressnamespace: nginx-ingress
---kind: ClusterRoleapiVersion: rbac.authorization.k8s.io/v1metadata:name: nginx-ingressrules:-apiGroups:-""resources:-services-endpointsverbs:-get-list-watch-apiGroups:-""resources:-secretsverbs:-get-list-watch-apiGroups:-""resources:-configmapsverbs:-get-list-watch-update-create-apiGroups:-""resources:-podsverbs:-list-watch-apiGroups:-""resources:-eventsverbs:-create-patch-list-apiGroups:-extensionsresources:-ingressesverbs:-list-watch-get-apiGroups:-"extensions"resources:-ingresses/statusverbs:-update-apiGroups:-k8s.nginx.orgresources:-virtualservers-virtualserverroutes-globalconfigurations-transportservers-policiesverbs:-list-watch-get-apiGroups:-k8s.nginx.orgresources:-virtualservers/status-virtualserverroutes/statusverbs:-update---kind: ClusterRoleBindingapiVersion: rbac.authorization.k8s.io/v1metadata:name: nginx-ingresssubjects:-kind: ServiceAccountname: nginx-ingressnamespace: nginx-ingressroleRef:kind: ClusterRolename: nginx-ingressapiGroup: rbac.authorization.k8s.io
---apiVersion: v1kind: Secretmetadata:name: default-server-secretnamespace: nginx-ingresstype: Opaquedata:tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN2akNDQWFZQ0NRREFPRjl0THNhWFhEQU5CZ2txaGtpRzl3MEJBUXNGQURBaE1SOHdIUVlEVlFRRERCWk8KUjBsT1dFbHVaM0psYzNORGIyNTBjbTlzYkdWeU1CNFhEVEU0TURreE1qRTRNRE16TlZvWERUSXpNRGt4TVRFNApNRE16TlZvd0lURWZNQjBHQTFVRUF3d1dUa2RKVGxoSmJtZHlaWE56UTI5dWRISnZiR3hsY2pDQ0FTSXdEUVlKCktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQUwvN2hIUEtFWGRMdjNyaUM3QlBrMTNpWkt5eTlyQ08KR2xZUXYyK2EzUDF0azIrS3YwVGF5aGRCbDRrcnNUcTZzZm8vWUk1Y2Vhbkw4WGM3U1pyQkVRYm9EN2REbWs1Qgo4eDZLS2xHWU5IWlg0Rm5UZ0VPaStlM2ptTFFxRlBSY1kzVnNPazFFeUZBL0JnWlJVbkNHZUtGeERSN0tQdGhyCmtqSXVuektURXUyaDU4Tlp0S21ScUJHdDEwcTNRYzhZT3ExM2FnbmovUWRjc0ZYYTJnMjB1K1lYZDdoZ3krZksKWk4vVUkxQUQ0YzZyM1lma1ZWUmVHd1lxQVp1WXN2V0RKbW1GNWRwdEMzN011cDBPRUxVTExSakZJOTZXNXIwSAo1TmdPc25NWFJNV1hYVlpiNWRxT3R0SmRtS3FhZ25TZ1JQQVpQN2MwQjFQU2FqYzZjNGZRVXpNQ0F3RUFBVEFOCkJna3Foa2lHOXcwQkFRc0ZBQU9DQVFFQWpLb2tRdGRPcEsrTzhibWVPc3lySmdJSXJycVFVY2ZOUitjb0hZVUoKdGhrYnhITFMzR3VBTWI5dm15VExPY2xxeC9aYzJPblEwMEJCLzlTb0swcitFZ1U2UlVrRWtWcitTTFA3NTdUWgozZWI4dmdPdEduMS9ienM3bzNBaS9kclkrcUI5Q2k1S3lPc3FHTG1US2xFaUtOYkcyR1ZyTWxjS0ZYQU80YTY3Cklnc1hzYktNbTQwV1U3cG9mcGltU1ZmaXFSdkV5YmN3N0NYODF6cFErUyt1eHRYK2VBZ3V0NHh3VlI5d2IyVXYKelhuZk9HbWhWNThDd1dIQnNKa0kxNXhaa2VUWXdSN0diaEFMSkZUUkk3dkhvQXprTWIzbjAxQjQyWjNrN3RXNQpJUDFmTlpIOFUvOWxiUHNoT21FRFZkdjF5ZytVRVJxbStGSis2R0oxeFJGcGZnPT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
tls.key: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcEFJQkFBS0NBUUVBdi91RWM4b1JkMHUvZXVJTHNFK1RYZUprckxMMnNJNGFWaEMvYjVyYy9XMlRiNHEvClJOcktGMEdYaVN1eE9ycXgrajlnamx4NXFjdnhkenRKbXNFUkJ1Z1B0ME9hVGtIekhvb3FVWmcwZGxmZ1dkT0EKUTZMNTdlT1l0Q29VOUZ4amRXdzZUVVRJVUQ4R0JsRlNjSVo0b1hFTkhzbysyR3VTTWk2Zk1wTVM3YUhudzFtMApxWkdvRWEzWFNyZEJ6eGc2clhkcUNlUDlCMXl3VmRyYURiUzc1aGQzdUdETDU4cGszOVFqVUFQaHpxdmRoK1JWClZGNGJCaW9CbTVpeTlZTW1hWVhsMm0wTGZzeTZuUTRRdFFzdEdNVWozcGJtdlFmazJBNnljeGRFeFpkZFZsdmwKMm82MjBsMllxcHFDZEtCRThCay90elFIVTlKcU56cHpoOUJUTXdJREFRQUJBb0lCQVFDZklHbXowOHhRVmorNwpLZnZJUXQwQ0YzR2MxNld6eDhVNml4MHg4Mm15d1kxUUNlL3BzWE9LZlRxT1h1SENyUlp5TnUvZ2IvUUQ4bUFOCmxOMjRZTWl0TWRJODg5TEZoTkp3QU5OODJDeTczckM5bzVvUDlkazAvYzRIbjAzSkVYNzZ5QjgzQm9rR1FvYksKMjhMNk0rdHUzUmFqNjd6Vmc2d2szaEhrU0pXSzBwV1YrSjdrUkRWYmhDYUZhNk5nMUZNRWxhTlozVDhhUUtyQgpDUDNDeEFTdjYxWTk5TEI4KzNXWVFIK3NYaTVGM01pYVNBZ1BkQUk3WEh1dXFET1lvMU5PL0JoSGt1aVg2QnRtCnorNTZud2pZMy8yUytSRmNBc3JMTnIwMDJZZi9oY0IraVlDNzVWYmcydVd6WTY3TWdOTGQ5VW9RU3BDRkYrVm4KM0cyUnhybnhBb0dCQU40U3M0ZVlPU2huMVpQQjdhTUZsY0k2RHR2S2ErTGZTTXFyY2pOZjJlSEpZNnhubmxKdgpGenpGL2RiVWVTbWxSekR0WkdlcXZXaHFISy9iTjIyeWJhOU1WMDlRQ0JFTk5jNmtWajJTVHpUWkJVbEx4QzYrCk93Z0wyZHhKendWelU0VC84ajdHalRUN05BZVpFS2FvRHFyRG5BYWkyaW5oZU1JVWZHRXFGKzJyQW9HQkFOMVAKK0tZL0lsS3RWRzRKSklQNzBjUis3RmpyeXJpY05iWCtQVzUvOXFHaWxnY2grZ3l4b25BWlBpd2NpeDN3QVpGdwpaZC96ZFB2aTBkWEppc1BSZjRMazg5b2pCUmpiRmRmc2l5UmJYbyt3TFU4NUhRU2NGMnN5aUFPaTVBRHdVU0FkCm45YWFweUNweEFkREtERHdObit3ZFhtaTZ0OHRpSFRkK3RoVDhkaVpBb0dCQUt6Wis1bG9OOTBtYlF4VVh5YUwKMjFSUm9tMGJjcndsTmVCaWNFSmlzaEhYa2xpSVVxZ3hSZklNM2hhUVRUcklKZENFaHFsV01aV0xPb2I2NTNyZgo3aFlMSXM1ZUtka3o0aFRVdnpldm9TMHVXcm9CV2xOVHlGanIrSWhKZnZUc0hpOGdsU3FkbXgySkJhZUFVWUNXCndNdlQ4NmNLclNyNkQrZG8wS05FZzFsL0FvR0FlMkFVdHVFbFNqLzBmRzgrV3hHc1RFV1JqclRNUzRSUjhRWXQKeXdjdFA4aDZxTGxKUTRCWGxQU05rMXZLTmtOUkxIb2pZT2pCQTViYjhibXNVU1BlV09NNENoaFJ4QnlHbmR2eAphYkJDRkFwY0IvbEg4d1R0alVZYlN5T294ZGt5OEp0ek90ajJhS0FiZHd6NlArWDZDODhjZmxYVFo5MWpYL3RMCjF3TmRKS2tDZ1lCbyt0UzB5TzJ2SWFmK2UwSkN5TGhzVDQ5cTN3Zis2QWVqWGx2WDJ1VnRYejN5QTZnbXo5aCsKcDNlK2JMRUxwb3B0WFhNdUFRR0xhUkcrYlNNcjR5dERYbE5ZSndUeThXczNKY3dlSTdqZVp2b0ZpbmNvVlVIMwphdmxoTUVCRGYxSjltSDB5cDBwWUNaS2ROdHNvZEZtQktzVEtQMjJhTmtsVVhCS3gyZzR6cFE9PQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo=
---kind: ConfigMapapiVersion: v1metadata:name: nginx-confignamespace: nginx-ingressdata:
---apiVersion: apps/v1kind: Deploymentmetadata:name: nginx-ingressnamespace: nginx-ingressspec:replicas: 1selector:matchLabels:app: nginx-ingresstemplate:metadata:labels:app: nginx-ingressspec:serviceAccountName: nginx-ingress # 引入 SA 用于 pod 和 kube-apiServer 进行加密认证containers:-image: nginx/nginx-ingress:1.7.2imagePullPolicy: IfNotPresentname: nginx-ingressports:-name: httpcontainerPort: 80hostPort: 80-name: httpscontainerPort: 443hostPort: 443env: # 定义变量-name: POD_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespace-name: POD_NAMEvalueFrom:fieldRef:fieldPath: metadata.nameargs: # 通过变量引入 secret 和 configMap--nginx-configmaps=$(POD_NAMESPACE)/nginx-config--default-server-tls-secret=$(POD_NAMESPACE)/default-server-secret
[root@kube-masteringress]NAMEDATA AGEconfigmap/kube-root-ca.crt1 77mconfigmap/nginx-config0 77m
NAMESECRETS AGEserviceaccount/default1 77mserviceaccount/nginx-ingress1 77m
NAMEREADY UP-TO-DATE AVAILABLE AGEdeployment.apps/nginx-ingress1/1 1 1 77m
NAMETYPE DATA AGEsecret/default-server-secretOpaque 2 77msecret/default-token-2rhp2kubernetes.io/service-account-token 3 77msecret/nginx-ingress-token-9k4tpkubernetes.io/service-account-token 3 77m
2、创建 Service 和 Deployment(服务)
[root@kube-masteringress]apiVersion: apps/v1kind: Deploymentmetadata:name: nginxspec:replicas: 3selector:matchLabels:app: nginx
template:metadata:labels:app: nginxspec:containers:-name: mywebimage: nginximagePullPolicy: IfNotPresentports:-containerPort: 80
---apiVersion: v1kind: Servicemetadata:name: nginx-svcspec:selector:app: nginxports:-protocol: TCPport: 80targetPort: 80
[root@kube-masteringress]NAMEREADY STATUS RESTARTS AGEpod/nginx-f8f495-8l9sd1/1 Running 0 43mpod/nginx-f8f495-hs49c1/1 Running 0 43mpod/nginx-f8f495-zhbvn1/1 Running 0 43m
NAMETYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEservice/kubernetesClusterIP 10.96.0.1 443/TCP 16dservice/nginx-svcClusterIP 10.96.74.181 80/TCP 43m
3、创建 Ingress 对外进行域名代理
[root@kube-master ingress]apiVersion: networking.k8s.io/v1kind: Ingressmetadata: name: nginx-ingress-test spec: rules: - host: www.zkj.com http: paths: - path: /backend pathType: ImplementationSpecific backend: service: name: nginx-svc port: number: 80
[root@kube-master ingress]Name: nginx-ingress-testNamespace: defaultAddress: Default backend: default-http-backend:80 ("default-http-backend" not found>)Rules: Host Path Backends ---- ---- -------- www.zkj.com /backend nginx-svc:80 (10.244.1.4:80,10.244.1.5:80,10.244.2.3:80)Annotations: Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal AddedOrUpdated 35s (x2 over 38m) nginx-ingress-controller Configuration for default/nginx-ingress-test was added or updated
4、查看 IngressContainer 容器中的 nginx 配置文件
[root@kube-masteringress]NAMEREADY STATUS RESTARTS AGEnginx-ingress-78c4c6db55-tgl4w1/1 Running 0 3h18m
[root@kube-master~]
nginx@nginx-ingress-78c4c6db55-tgl4w:/$ cd etc/nginx/conf.d/
nginx@nginx-ingress-78c4c6db55-tgl4w:/etc/nginx/conf.d$ cat default-nginx-ingress-test.conf
# 负载均衡 pod ip 地址upstreamdefault-nginx-ingress-test-www.zkj.com-nginx-svc-8080 {zonedefault-nginx-ingress-test-www.zkj.com-nginx-svc-8080 256k;randomtwo least_conn;
server10.244.1.10:80 max_fails=1 fail_timeout=10s max_conns=0;
}
server{
listen80;server_tokenson;server_namewww.zkj.com; # 域名为 hosts 字段定义的域名
location/backend {
proxy_http_version1.1;
proxy_connect_timeout60s;proxy_read_timeout60s;proxy_send_timeout60s;client_max_body_size1m;proxy_set_headerHost $host;proxy_set_headerX-Real-IP $remote_addr;proxy_set_headerX-Forwarded-For $proxy_add_x_forwarded_for;proxy_set_headerX-Forwarded-Host $host;proxy_set_headerX-Forwarded-Port $server_port;proxy_set_headerX-Forwarded-Proto $scheme;proxy_bufferingon;
proxy_passhttp://default-nginx-ingress-test-www.zkj.com-nginx-svc-8080;}}
欢迎👏关注《院长技术》,只做原创,干货~~~

院长技术官网:https://deanit.cn
院长技术博客:https://blog.deanit.cn
更多技术干货持续输出中......

欢迎加入微信群:《有效期至:11.21日》

Tips:二维码过期后,可通过公众号私信院长,邀请微信群
欢迎加入QQ-2群:【
1群已满
】
