Kubernetes GitOps Ingress Nginx with Lets Encrypt Certificate

This guide explains how to deploy Ingress-NGINX with dynamically (hostname-based) assigned Let’s Encrypt certificates using Flux GitOps. The steps are based on a working example and provide instructions for configuration, deployment, and testing.

Prerequisites

Flux Installed: Ensure Flux is installed and running in your Kubernetes cluster.
Let’s Encrypt Certificate: Provisioned for FQDN. Follow the instructions in the Let’s Encrypt guide.
Git Repository: A Git repository structured for Flux GitOps, e.g.:

.
├── apps/
└── ingress-nginx/
    └── base/
├── clusters/
│   └── production/
│       ├── flux-system/
│       │   └── sources/
│       └── apps/
├── infrastructure/
    └── networking/
        ├── metallb/
        └── ingress-nginx/

Kubernetes Cluster: A Kubernetes cluster with MetalLB-compatible networking.

1. Deploying Ingress-NGINX via Flux

Step 1: Create the Ingress-NGINX Namespace

Create a namespace for Ingress-NGINX in your Git repository:

File: clusters/production/flux-system/sources/ingress-nginx-namespace.yaml:

# File: clusters/production/flux-system/sources/ingress-nginx-namespace.yaml
---
apiVersion: v1
kind: Namespace
metadata:
  name: ingress-nginx

Step 2: Create the Ingress-NGINX HelmRepository

Define the Helm repository for Ingress-NGINX:

File: clusters/production/flux-system/sources/ingress-nginx-helmrepository.yaml:

# File: clusters/production/flux-system/sources/ingress-nginx-helmrepository.yaml
---
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: HelmRepository
metadata:
  name: ingress-nginx
  namespace: flux-system
spec:
  interval: 1h
  url: https://kubernetes.github.io/ingress-nginx

Step 3: Create the Ingress-NGINX HelmRelease

Define the HelmRelease to install Ingress-NGINX:

File: clusters/production/flux-system/sources/ingress-nginx-helmrelease.yaml:

# File: clusters/production/flux-system/sources/ingress-nginx-helmrelease.yaml
---
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
spec:
  releaseName: ingress-nginx
  chart:
    spec:
      chart: ingress-nginx
      version: 4.12.0
      sourceRef:
        kind: HelmRepository
        name: ingress-nginx
        namespace: flux-system
  interval: 10m
  values:
    controller:
      service:
        type: LoadBalancer
      admissionWebhooks:
        enabled: true

Add these files to your kustomization.yaml file:

File: clusters/production/flux-system/sources/kustomization.yaml:

# File: clusters/production/flux-system/sources/kustomization.yaml
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  # ...existing resources...
  - ingress-nginx-namespace.yaml
  - ingress-nginx-helmrepository.yaml
  - ingress-nginx-helmrelease.yaml

Step 4: Configure Ingress-NGINX

Create the Ingress-NGINX configuration:

File: infrastructure/networking/ingress-nginx/ingress-nginx-config.yaml:

# File: infrastructure/networking/ingress-nginx/ingress-nginx-config.yaml
---
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
  name: ingress-nginx
  namespace: flux-system
spec:
  interval: 10m
  sourceRef:
    kind: GitRepository
    name: flux-system
  path: ./infrastructure/networking/ingress-nginx
  prune: true
  healthChecks:
    - apiVersion: apps/v1
      kind: Deployment
      name: ingress-nginx-controller
      namespace: ingress-nginx
  dependsOn: []

Add this file to your kustomization.yaml:

File: infrastructure/networking/ingress-nginx/kustomization.yaml:

# File: infrastructure/networking/ingress-nginx/kustomization.yaml
---
resources:
  - ingress-nginx-config.yaml

Step 5: Add Ingress-NGINX in Flux

Add the configuration to your Flux kustomization.yaml:

File: clusters/production/flux-system/ingress-nginx.yaml:

# File: clusters/production/flux-system/ingress-nginx.yaml
---
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
  name: ingress-nginx
  namespace: flux-system
spec:
  interval: 10m
  sourceRef:
    kind: GitRepository
    name: flux-system
  path: ./infrastructure/networking/ingress-nginx
  prune: true
  healthChecks:
    - apiVersion: apps/v1
      kind: Deployment
      name: ingress-nginx-controller
      namespace: ingress-nginx

Add this entry to your clusters/production/flux-system/kustomization.yaml:

File: clusters/production/flux-system/kustomization.yaml:

# File: clusters/production/flux-system/kustomization.yaml
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  # ...existing entries...
  - ingress-nginx.yaml

Step 6: Create Test Ingress with Certificate

Create a test ingress resource. This test ensures that the NGINX controller is properly configured to serve the Let’s Encrypt certificate and validates that TLS is working correctly for the specified hostname.

File: apps/ingress-nginx/base/nginx-test-ingress.yaml:

# File: apps/ingress-nginx/base/nginx-test-ingress.yaml
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-ingress
  namespace: default
  annotations:
    kubernetes.io/ingress.class: nginx
    cert-manager.io/cluster-issuer: letsencrypt-production
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - <hostname FQDN>  # Set to your test hostname
    secretName: letsencrypt-production-cert-tls
  rules:
  - host: <hostname FQDN>  # Set to your test hostname
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: nginx-service
            port:
              number: 80

Add this file to the apps/ingress-nginx/base/kustomization.yaml:

File: apps/ingress-nginx/base/kustomization.yaml:

# File: apps/ingress-nginx/base/kustomization.yaml
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - nginx-test-ingress.yaml

Include the entire folder in clusters/production/kustomization.yaml:

File: clusters/production/kustomization.yaml:

# File: clusters/production/kustomization.yaml
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  # ...existing resources...
  - ../../../apps/ingress-nginx/base

2. Commit and Push the Changes

git add -A
git commit -m "Deploy Ingress-NGINX with Let's Encrypt"
git push

Reconcile Flux to apply the changes:

flux reconcile kustomization apps -n flux-system
flux reconcile kustomization ingress-nginx -n flux-system

3. Test the Deployment

Verify the Setup

Check all kustomizations are in a READY state:

kubectl get kustomizations -A
# Example output
# NAMESPACE      NAME            AGE     READY   STATUS
# cert-manager   cert-manager    6d19h   True    Applied revision: main@sha1:c830e277a93d0c73fb4941b6f2e5b6aea15f20cb
# flux-system    flux-system     8d      True    Applied revision: main@sha1:c830e277a93d0c73fb4941b6f2e5b6aea15f20cb
# flux-system    ingress-nginx   4h6m    True    Applied revision: main@sha1:c830e277a93d0c73fb4941b6f2e5b6aea15f20cb
# flux-system    metallb         2d20h   True    Applied revision: main@sha1:c830e277a93d0c73fb4941b6f2e5b6aea15f20cb
# flux-system    pull-secrets    8d      True    Applied revision: main@sha1:c830e277a93d0c73fb4941b6f2e5b6aea15f20cb

Check the namespace:

kubectl get namespace ingress-nginx
# Example output
# NAME              STATUS   AGE
# ingress-nginx     Active   4h8m

Verify the Helm repository:

kubectl get helmrepository ingress-nginx -n flux-system
# Example output
# NAME            URL                                          AGE     READY   STATUS
# ingress-nginx   https://kubernetes.github.io/ingress-nginx   4h9m    True    stored artifact: revision 'sha256:79f424dbbc344395f9cf19d4ade457e9de14af43a337bd76ee593fa1ad920330'

Verify the Helm release:

kubectl get helmrelease ingress-nginx -n ingress-nginx
# Example output
# NAME            AGE     READY   STATUS
# ingress-nginx   4h10m   True    Helm install succeeded for release ingress-nginx/ingress-nginx.v1 with chart [email protected]

Check the deployment and pods:

kubectl get deployments -n ingress-nginx
# Example output
# NAME                       READY   UP-TO-DATE   AVAILABLE   AGE
# ingress-nginx-controller   1/1     1            1           4h12m

kubectl get pods -n ingress-nginx
# Example output
# NAME                                        READY   STATUS    RESTARTS   AGE
# ingress-nginx-controller-6bf449f988-kns7n   1/1     Running   0          3h38m

Check for the external IP of the ingress-nginx-controller:

kubectl get svc -n ingress-nginx
# Example output
# NAME                                 TYPE           CLUSTER-IP      EXTERNAL-IP    PORT(S)                      AGE
# ingress-nginx-controller             LoadBalancer   10.109.225.4    xxx.xxx.xxx.91   80:32680/TCP,443:30597/TCP   4h13m
# ingress-nginx-controller-admission   ClusterIP      10.101.140.93   <none>         443/TCP                      4h13m

Test the Ingress

Ensure your hostname resolves to the external IP. If DNS is not configured, you can manually update your /etc/hosts file by adding an entry mapping the external IP to the hostname, e.g., xxx.xxx.xxx.91 <FQDN>. Once resolved, test using curl:

curl -v -k https://<FQDN> 2>&1 | grep issuer
# Example output
# *  issuer: C=US; O=Let's Encrypt; CN=R11
# *  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.

This completes the deployment of Ingress-NGINX with Let’s Encrypt certificates using Flux. If you encounter any issues, review the Flux logs running flux events and ensure all resources are correctly configured.

kubectl get namespace ingress-nginx
# Example output
# NAME              STATUS   AGE
# ingress-nginx     Active   4h8m

Verify the Helm repository:

kubectl get helmrepository ingress-nginx -n flux-system
# Example output
# NAME            URL                                          AGE     READY   STATUS
# ingress-nginx   https://kubernetes.github.io/ingress-nginx   4h9m    True    stored artifact: revision 'sha256:79f424dbbc344395f9cf19d4ade457e9de14af43a337bd76ee593fa1ad920330'

Verify the Helm release:

kubectl get helmrelease ingress-nginx -n ingress-nginx
# Example output
# NAME            AGE     READY   STATUS
# ingress-nginx   4h10m   True    Helm install succeeded for release ingress-nginx/ingress-nginx.v1 with chart [email protected]

Check the deployment and pods:

kubectl get deployments -n ingress-nginx
# Example output
# NAME                       READY   UP-TO-DATE   AVAILABLE   AGE
# ingress-nginx-controller   1/1     1            1           4h12m
#
kubectl get pods -n ingress-nginx
# Example output
# NAME                                        READY   STATUS    RESTARTS   AGE
# ingress-nginx-controller-6bf449f988-kns7n   1/1     Running   0          3h38m

Check for the external IP of the ingress-nginx-controller :

kubectl get svc -n ingress-nginx
# Example output
# NAME                                 TYPE           CLUSTER-IP      EXTERNAL-IP    PORT(S)                      AGE
# ingress-nginx-controller             LoadBalancer   10.109.225.4    xxx.xxx.xxx.91   80:32680/TCP,443:30597/TCP   4h13m
# ingress-nginx-controller-admission   ClusterIP      10.101.140.93   <none>         443/TCP                      4h13m

Test the Ingress:

Ensure your hostname resolves to the external IP. If DNS is not configured, you can manually update your /etc/hosts file by adding an entry mapping the external IP to the hostname, e.g., xxx.xxx.xxx.91 . Once resolved, test using curl:

curl -v -k https://<FQDN>  2>&1 | grep issuer
# Example output
# *  issuer: C=US; O=Let's Encrypt; CN=R11
# *  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.

This completes the deployment of Ingress-NGINX with Let’s Encrypt certificates using Flux. If you encounter any issues, review the Flux logs running flux events and ensure all resources are correctly configured.

Prerequisites#

1. Deploying Ingress-NGINX via Flux#

Step 1: Create the Ingress-NGINX Namespace#

Step 2: Create the Ingress-NGINX HelmRepository#

Step 3: Create the Ingress-NGINX HelmRelease#

Step 4: Configure Ingress-NGINX#

Step 5: Add Ingress-NGINX in Flux#

Step 6: Create Test Ingress with Certificate#

2. Commit and Push the Changes#

3. Test the Deployment#

Verify the Setup#

Test the Ingress#

Test the Ingress:#