K8S ingress: Deploy internet facing kubernetes ingress with NodePort

6 min readSep 4, 2022

There are 3 main service types in a kubernetes deployment- ClusterIP NodePort LoadBalancer. To expose services in the internet, using ClusterIP and NodePort is not a good practice. LoadBalancer can be used in this case. But using LoadBalancer service type in can be problematic in many cases, because many of the cloud providers out there don’t provide load balancer service. This is also true for on prem cloud. We will explore around such a case in this blog post. Here what we gonna do is, we will deploy a kubernetes service with NGINX-INGRESS where its service type will be NodePort and we will expose our deployment in the internet following the production standards.

Architecture

Here is the simplified architecture of the solution to the problem. To summarize, we will create an ingress which will route traffic to our pods in the kubernetes cluster. The ingress will be accessed from a separate reverse proxy which is publicly accessible in a certain port and binded by a domain name. Our cluster and reverse proxy vm is in the same subnet and no firewall within. Cluster is blocked from the internet, but the VM is accessible in port 80.

Preparation

In the first place, we will deploy a Nginx image in service type ClusterIP. This is a basic nginx welcome page.

# nginx.ymlapiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 2
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80---apiVersion: v1
kind: Service
metadata:
  name: nginx-svc
spec:
  ports:
  - port: 80
    targetPort: 80
  selector:
    app: nginx

So, let’s apply it-

kubectl apply -f nginx.yml

So it launches 2 pods and a service of type ClusterIP. We can check it using-

We can check the content served using cluster ip:

Nginx Ingresss

Installation & Setup Ingress Controller

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.3.0/deploy/static/provider/cloud/deploy.yaml

This will create a new namespace ingress-nginx. It will create 3 pods and 2 services. If we check the service now-

we will see the external IP is in <pending> state. This is due to our cloud provider doesn't provide load balancer. Now, we will change the service type of ingress-nginx-controller pod to NodePort-

$ kubectl edit svc -n ingress-nginx ingress-nginx-controller

Change the type and save. If we check service now, we will see the external ip is now in <none> stage from <pending>.

Our nginx controller configuration is done. Now it’s time to create the rules for ingress resource.

Ingress Resource

In our ingress resource, we will serve our nginx welcome page which we deployed earlier using our hostname, e.g. web.shafinhasnat.me, later we will make this public. Create the yml file below-

# ingress.yml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: minimal-ingress
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/use-regex: "true"
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - host: web.shafinhasnat.me
    http:
      paths:      
      - path: /
        pathType: Prefix
        backend:
          service:
            name: nginx-svc
            port:
              number: 80

Apply using-

kubectl apply -f ingress.yml

It will create an ingress called minimal-ingress. We can check it using-

Now, if we resolve the dns to the address, we will be able to see the content served by nginx pod-

We can check directly using the domain by resolving it to ip in /etc/hosts-

#add this line

10.18.0.66 web.shafinhasnat.me

So everything is working from the host machine. Now we will make this accessible from the internet.

Expose to Internet

Now, what we have to do is, reverse proxy incoming traffic to the ingress ip and port. Basically we will expose the reverse proxy service in the internet. We will use nginx in this case. Install nginx in a linux vm. Create a .conf file inside /etc/nginx/conf.d/ directory.

# /etc/nginx/conf.d/default.confserver {
  listen 80; server_name _;
  location / {
    proxy_set_header Host $host ;
    proxy_set_header X-Forwarded-For $remote_addr ;
    proxy_pass http://11.0.0.21:30243;
  }
}

Here the ip is of the worker node and the port is the nodeport of ingress controller which binds to port 80. Apply the changes-

sudo systemctl restart nginx

Make sure, this vm has internet connectivity and port 80 allowed in firewall. Now, if we check resolving dns to the ip of the vm, we will see our nginx welcome page form our kubernetes cluster is up and running.

To make it public, just simply add the public ip in the A records of your domain name provider manager dashboard. Your ingress is working fine from the Internet!!!

Setting up TLS

We have to create a kubernetes Secret with our .crt and .key file. And bind that Secret to our ingress file. To create the Secret-

kubectl create secret tls our-secret —-key <key_file.key> --cert <cert_file.crt>

A secret called our-secret is created in default namespace. Now make the required changes in ingress.yml file-

# ingress.yml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: minimal-ingress
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/use-regex: "true"
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  tls:
    - hosts:
      - web.shafinhasnat.me
  secretName: our-secret
  rules:
  - host: web.shafinhasnat.me
    http:
      paths:      
      - path: /
        pathType: Prefix
        backend:
          service:
            name: nginx-svc
            port:
              number: 80

Apply the changes with-

kubectl apply -f ingress.yml

In our reverse proxy server, we have to set up a nginx TLS pass through. Delete the .conf file from before steps and open /etc/nginx/nginx.conf with vi editor, and add make the changes like below-

...
http {
...
  server {
    listen 80;
    server_name web.shafinhasnat.me;
    return 301 https://$host$request_uri;
  }
}...
stream {upstream backend_servers {
  server 11.0.0.21:32414;
}
log_format basic '$remote_addr [$time_local] '
                 '$protocol $status $bytes_sent $bytes_received '
                 '$session_time "$upstream_addr" '
                 '"$upstream_bytes_sent" "$upstream_bytes_received"             "$upstream_connect_time"';
access_log /var/log/nginx/access.log basic;
error_log  /var/log/nginx/error.log;
server {
    listen 443;
    proxy_pass backend_servers;
    proxy_next_upstream on;
  }
}

Here the ip is the ip of cluster and the port is the nodeport of ingress controller which binds to port 443. Now, apply the changes-

sudo systemctl restart nginx

So, our project is now https enabled. We can now access our website from https://web.shafinhasnat.me (Obviously the domain name will be your personal one 😛). If we try with http, we will be automatically redirected to https!