OpenWebUI Deployment in FluxCD Kubernetes Cluster

Overview

Successfully deployed OpenWebUI in a production Kubernetes cluster managed by FluxCD GitOps. OpenWebUI provides a web interface for interacting with Large Language Models (LLMs) and is configured to connect to a llama.cpp backend.

Architecture

graph TB
    A[User] --> B[Nginx Ingress]
    B --> C[OpenWebUI Service]
    C --> D[OpenWebUI Pod]
    D --> E[External llama.cpp Backend]
    
    F[FluxCD] --> G[Git Repository]
    G --> H[OpenWebUI Kustomization]
    H --> I[Kubernetes Resources]
    
    J[Cert-Manager] --> K[Let's Encrypt]
    K --> L[TLS Certificate]
    L --> B

Deployment Configuration

Core Components

  • Namespace: openwebui
  • Image: ghcr.io/open-webui/open-webui:main
  • Backend: http://llama-cpp.<yourdomain.com>:11434 (external Ollama-compatible API)
  • Storage: 10Gi PersistentVolume for user data persistence
  • Access: HTTPS via nginx-ingress with Let’s Encrypt certificates

Resource Specifications

# Resource Limits & Requests
resources:
  limits:
    cpu: 500m
    memory: 1Gi
  requests:
    cpu: 100m
    memory: 256Mi

Key Features Configured

  1. Persistent Storage: User conversations and settings preserved across pod restarts
  2. TLS Encryption: Automatic HTTPS certificates via cert-manager + Let’s Encrypt
  3. External LLM Backend: Configured to use existing llama.cpp server
  4. GitOps Management: Fully managed via FluxCD from Git repository

Access Information

  • URL: https://openwebui.<yourdomain.com>
  • TLS Certificate: Auto-provisioned by cert-manager using Let’s Encrypt production issuer
  • DNS Challenge: Uses Cloudflare DNS-01 for certificate validation

FluxCD GitOps Structure

The deployment follows GitOps principles with the following structure:

apps/openwebui/base/
├── kustomization.yaml          # Resource list
├── openwebui-namespace.yaml    # Namespace definition  
├── openwebui-deployment.yaml   # Pod deployment
├── openwebui-service.yaml      # ClusterIP service
├── openwebui-ingress.yaml      # Nginx ingress with TLS
├── openwebui-certificate.yaml  # Let's Encrypt certificate
├── openwebui-pv.yaml          # PersistentVolume
└── openwebui-pvc.yaml         # PersistentVolumeClaim

Secrets Management

  • SOPS Encryption: Environment secrets encrypted with SOPS
  • External GPG Keys: SOPS GPG keys managed outside Git repository (security best practice)
  • Secret Mounting: WEBUI_SECRET_KEY injected from Kubernetes secret

Troubleshooting Guide

Common Issues

  1. Certificate Not Ready

    kubectl get certificates -n openwebui
kubectl describe certificate openwebui-tls-certificate -n openwebui

  2. SOPS Decryption Failures

    kubectl get secrets sops-gpg -n flux-system
flux get kustomizations | grep False

  3. Pod Connectivity Issues

    kubectl logs -n openwebui deployment/openwebui
kubectl get ingress -n openwebui

Health Checks

# Check all OpenWebUI resources
kubectl get all -n openwebui

# Verify TLS certificate status  
kubectl get certificates,secrets -n openwebui

# Test connectivity
curl -k -H "Host: openwebui.<yourdomain.com>" https://172.16.32.91

Security Considerations

Implemented Security Measures

  1. No GPG Keys in Git: SOPS GPG private keys never stored in repository
  2. TLS Encryption: All traffic encrypted with valid certificates
  3. Secret Encryption: All sensitive data encrypted with SOPS
  4. Network Policies: Pod-to-pod communication controlled
  5. Resource Limits: CPU and memory boundaries enforced

Security Best Practices Applied

  • Secrets managed via external kubectl commands, not GitOps
  • Git history cleaned of any accidentally committed GPG keys
  • .gitignore rules prevent future secret commits
  • Regular certificate rotation via Let’s Encrypt

Operational Notes

Backup Strategy

  • Persistent Data: /app/backend/data volume contains user conversations
  • Configuration: All infrastructure as code in Git repository
  • Secrets: GPG keys backed up separately from cluster

Monitoring

  • FluxCD Status: Monitor kustomization health in flux-system namespace
  • Certificate Expiry: cert-manager handles automatic renewal
  • Resource Usage: Monitor CPU/memory against configured limits

Lessons Learned

  1. SOPS Security: Never commit GPG private keys to Git - manage externally
  2. Namespace Strategy: Place cert-manager kustomization in flux-system namespace for SOPS access
  3. Certificate Dependencies: Ensure DNS and Cloudflare API tokens are properly configured
  4. GitOps Discipline: Use proper Git history hygiene for security-sensitive deployments

Last Updated: October 15, 2025 Deployment Status: ✅ Operational