Secure Kubernetes Services with Ingress, TLS and Let's Encrypt

By Adnan Abdulhussein, Vikram Vaswani, Sameer Naik
  • ingress
  • tls
  • security
  • let's encrypt

Introduction

Kubernetes gives you a lot of flexibility in defining how you want services to be exposed. You can configure your Service objects to ensure a group of pods are only accessible within the cluster, or enable access from outside the cluster. If your cloud provider of choice supports it, you can even request a load balanced IP address or hostname for your Service.

The Service object gives us a simple way to connect services running in pods. However, if you find the need for setting up basic routing rules to different services or want to configure things like TLS, the Ingress resource provides a much more flexible way to configure external access.

Ingress can be backed by different implementations through the use of different Ingress Controllers. The most popular of these is the NGINX Ingress Controller, however there are other options available such as Traefik or Rancher. Each controller should support a basic configuration, but can even expose other features (rewrite rules, authentication modes) via annotations.

In this guide, we'll take a look at setting up the NGINX Ingress Controller in our cluster and create Ingress routes for an example application. You'll learn how Ingress objects are defined, including how to configure TLS and basic authentication.

Prerequisites and Assumptions

This guide makes the following assumptions:

Step 1: Install the NGINX Ingress controller

The first step is to install the NGINX Ingress controller. The easiest way to get this running on any platform is using the Bitnami Helm chart.

helm repo add bitnami https://charts.bitnami.com/bitnami
helm install ingress bitnami/nginx-ingress-controller

Once the controller is created and running, you can access the NGINX server through the LoadBalancer IP address or NodePort of its Service. To obtain the LoadBalancer IP address, use the command below. For kubeadm, you can use the IP address of any node.

kubectl get svc ingress-nginx-ingress-controller -o jsonpath="{.status.loadBalancer.ingress[0].ip}"
Tip

It may take some time for the load balancer IP address to be assigned, so you may need to wait a few minutes before the previous command returns output.

External IP of Ingress controller

If everything is configured correctly, you should get a default NGINX welcome page when accessing the NGINX server using the load balancer IP address. This tells us that the controller doesn't know where to route the request to, so responds with the default page.

Step 2: Deploy the example application

Tip

Before proceeding, configure the DNS for your domain name by adding an A record pointing to the public IP address obtained in the previous step.

The Bitnami charts repository has a lot of easily deployable applications on offer. This example uses the Joomla! chart to get an instance of the Joomla! CMS running in the cluster.

Replace the DOMAIN placeholder with your domain name and update the password strings with different values:

helm install joomla bitnami/joomla \
      --set joomlaPassword=secretpassword \
      --set mariadb.root.password=secretpassword \
      --set service.type=ClusterIP \
      --set ingress.enabled=true \
      --set ingress.hosts[0].name=DOMAIN

The chart has built-in Ingress support, so when installed with the parameters shown above, it will automatically create an Ingress route. Review the complete list of available chart parameters.

Once the deployment completes, Joomla! should be accessible via the domain name. Note that the NGINX Ingress Controller forces a self-signed TLS certificate for wildcard routes. In the next sections, you will make this more secure by configuring a real TLS certificate for the Joomla! site.

Joomla! site behind a self-signed TLS certificate

Step 3: Configure TLS with Let's Encrypt certificates and cert-manager

Now that you have enabled external access to the Joomla! instance, the next step is to enable TLS. Let's Encrypt is a free TLS Certificate Authority (CA) and you can use it to automatically request and renew Let's Encrypt certificates for public domain names.

cert-manager is a Kubernetes tool that issues certificates from various certificate providers, including Let's Encrypt. The next step is to install cert-manager with Helm following the official instructions.

  • Begin by adding the repository and creating a namespace:

    helm repo add jetstack https://charts.jetstack.io
kubectl create namespace cert-manager

  • Install some required resources. For Kubernetes v1.14 and lower:

    kubectl apply --validate=false -f https://github.com/jetstack/cert-manager/releases/download/v0.14.1/cert-manager-legacy.crds.yaml
```bash

For Kubernetes v1.15 and higher:

```bash
kubectl apply --validate=false -f https://github.com/jetstack/cert-manager/releases/download/v0.14.1/cert-manager.crds.yaml
```bash
Warning

When executing these commands on Google Kubernetes Engine (GKE), you may encounter permission errors. Refer to the official cert-manager documentation for notes on how to elevate your permissions.

  • Create a ClusterIssuer resource for Let's Encrypt certificates. Create a file named letsencrypt-prod.yaml with the following content. Replace the EMAIL-ADDRESS placeholder with a valid email address.

    apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
  labels:
    name: letsencrypt-prod
spec:
  acme:
    email: EMAIL-ADDRESS
    privateKeySecretRef:
      name: letsencrypt-prod
    server: https://acme-v02.api.letsencrypt.org/directory
    solvers:
    - http01:
        ingress:
          class: nginx

  • Apply the changes to the cluster:

    kubectl apply -f letsencrypt-prod.yaml

  • Install cert-manager with Helm and configure Let's Encrypt as the default Certificate Authority (CA):

    helm install cert-manager --namespace cert-manager jetstack/cert-manager --version v0.14.1
    Warning

    Before proceeding to the next step, ensure that the intended public domain name for your application has an A record pointing to the external IP address of the NGINX service, as this is required for Let's Encrypt validation and successful certificate generation.

  • Use the command below and replace the DOMAIN placeholder with your domain name. Remember to also replace the password strings with the values specified at the time of installation. Review the complete list of available chart parameters.

    helm upgrade joomla bitnami/joomla \
  --set joomlaPassword=secretpassword \
  --set mariadb.root.password=secretpassword \
  --set service.type=ClusterIP \
  --set ingress.enabled=true \
  --set ingress.certManager=true \
  --set ingress.tls[0].secretName=joomla.local-tls \
  --set ingress.annotations."kubernetes\.io/ingress\.class"=nginx \
  --set ingress.annotations."cert-manager\.io/cluster-issuer"=letsencrypt-prod \
  --set ingress.tls[0].hosts[0]=DOMAIN \
  --set ingress.hosts[0].name=DOMAIN

Once the deployment has been upgraded, a visit to the domain in the browser should present the Joomla! site over a secure TLS connection.

Joomla! site behind a secure TLS certificate

Step 4: Set up a basic routing rule

So far, you have defined a simple route to just one application. However, Ingress objects allow you to define multiple routes and paths under multiple hostnames. For example, suppose that in addition to the Joomla! site, you want to configure a Ghost blog when visiting the blog subdomain.

To create an Ingress definition with multiple routes:

  • First, install the Ghost chart on the cluster. Make sure to replace DOMAIN with your own domain. This time, set the type of the Service when installing the chart so you don't need to change it later.

    helm install ghost bitnami/ghost --set ghostHost=blog.DOMAIN,serviceType=ClusterIP

  • Now, update the Ingress definition to add the new routing rule for the Ghost blog on the subdomain. The subdomain must also be added to the TLS section. Remember to change DOMAIN here too.

    apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: joomla-ingress
  annotations:
    kubernetes.io/ingress.class: nginx
    kubernetes.io/tls-acme: "true"
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
  rules:
  - host: DOMAIN
    http:
      paths:
      - backend:
          serviceName: joomla
          servicePort: http
        path: /
  - host: blog.DOMAIN
    http:
      paths:
      - backend:
          serviceName: ghost
          servicePort: 80
        path: /
  tls:
  - secretName: joomla-tls-cert
    hosts:
      - DOMAIN
      - blog.DOMAIN

  • Once again, save this to a file and apply the change.

    kubectl apply -f multi-app-ingress.yaml

With this change, cert-manager will pick up the change to the TLS section and request a new certificate that covers both domains, and the NGINX Ingress controller will regenerate the NGINX configuration with the new virtual host for the blog subdomain. Once this configuration is updated, the Joomla! site will be accessible on the root domain and the Ghost blog can be reached on the blog subdomain.

Ghost blog on subdomain

Step 5: Configure basic authentication and other Ingress options

The NGINX Ingress Controller exposes different options for configuring the NGINX server through annotations on the Ingress object. Here is an example of setting up HTTP-Basic authentication:

  • First, create the htpasswd file for storing the usernames and passwords. Create a Secret containing this file so the NGINX Ingress controller can use it.

    htpasswd -bc auth ingress-user ingress-password
kubectl create secret generic joomla-basic-auth --from-file=auth

  • Add the auth-type: basic and auth-secret: joomla-basic-auth annotations to the Ingress definition. This tells the NGINX Ingress controller to configure basic authentication for the virtual host and where to read the htpasswd file from. Remember to change the DOMAIN placeholder.

    apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: joomla-ingress
annotations:
  kubernetes.io/ingress.class: nginx
  kubernetes.io/tls-acme: "true"
  cert-manager.io/cluster-issuer: "letsencrypt-prod"
  nginx.ingress.kubernetes.io/auth-type: basic
  nginx.ingress.kubernetes.io/auth-secret: joomla-basic-auth
spec:
rules:
- host: DOMAIN
    http:
    paths:
    - path: /
        backend:
        serviceName: joomla
        servicePort: 80
- host: blog.DOMAIN
    http:
    paths:
    - path: /
        backend:
        serviceName: ghost
        servicePort: 80
tls:
- secretName: joomla-tls-cert
    hosts:
    - DOMAIN
    - blog.DOMAIN

  • Save the above definition to a file and apply the changes.

    kubectl apply -f basic-auth-ingress.yaml

Once the NGINX Ingress controller has picked up the change and configured the NGINX service, a login prompt should appear when refreshing the Joomla! or Ghost site in your browser. Logging in with ingress-user and ingress-password will grant access.

Basic authentication prompt
Tip

If, even after making the change above, you are still allowed access to the Joomla! or Ghost sites without an authentication prompt, reconfirm that you do not have existing active Ingress definitions which allow unprotected access.

Useful links

This guide walked through the Kubernetes Ingress object: what it is, how it's different from a Service and how it's configured. It looked at setting up a simple Ingress definition for an example Joomla! site, then extending it to secure with TLS encryption and adding a new rule to route to the Ghost blog. It also examined other options such as configuring HTTP-Basic authentication for the NGINX Ingress controller. The documentation for the controller is a great resource for seeing what other options are available.