Kubernetes GitOps SearXNG Search Engine

Deploy a self-hosted, privacy-focused SearXNG metasearch engine on your Kubernetes cluster for integration with AI tools like OpenWebUI.

Overview

SearXNG is a privacy-respecting metasearch engine that aggregates results from multiple search engines without tracking users. This deployment features proper SOPS encryption, IP whitelisting, and integration-ready JSON API.

Features

Privacy-focused: No user tracking or data collection
Multi-engine aggregation: Combines results from Google, Bing, DuckDuckGo, Brave, Wikipedia, and more
JSON API: RESTful API for programmatic access (perfect for AI integration)
Rate limiting with IP whitelisting: Protects against abuse while allowing legitimate usage
HTTPS with automatic certificates: Let’s Encrypt via cert-manager
SOPS-encrypted secrets: Secure secret management following GitOps best practices

Repository Structure

├── apps/
│   └── searxng/
│       └── base/
│           ├── kustomization.yaml
│           ├── searxng-namespace.yaml
│           ├── searxng-settings.yaml
│           ├── searxng-deployment.yaml
│           ├── searxng-service.yaml
│           ├── searxng-certificate.yaml
│           └── searxng-ingress.yaml
├── infrastructure/
│   └── security/
│       └── searxng-secrets/
│           ├── kustomization.yaml
│           └── searxng-secret.yaml  # SOPS encrypted
└── clusters/
    └── production/
        ├── apps/
        │   └── kustomization.yaml  # References searxng
        └── flux-system/
            ├── kustomization.yaml  # References searxng-secrets
            └── searxng-secrets.yaml  # Flux Kustomization

Deployment Steps

1. Create Application Structure

Create the application folder structure:

mkdir -p apps/searxng/base
mkdir -p infrastructure/security/searxng-secrets

2. Create Namespace

Create apps/searxng/base/searxng-namespace.yaml:

# File: apps/searxng/base/searxng-namespace.yaml
---
apiVersion: v1
kind: Namespace
metadata:
  name: searxng

3. Create Settings ConfigMap

Create apps/searxng/base/searxng-settings.yaml with IP whitelisting:

# File: apps/searxng/base/searxng-settings.yaml
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: searxng-settings
  namespace: searxng
data:
  settings.yml: |
    # Use default settings as base
    use_default_settings: true
    
    # Override only what we need
    server:
      port: 8080
      bind_address: "0.0.0.0"
      base_url: "https://searxng.<domain.com>"
      limiter: true  # Enable limiter with whitelist
      public_instance: false
    
    search:
      safe_search: 0
      autocomplete: ""
      default_lang: "en"
      formats:
        - html
        - json
        - rss
    
    general:
      debug: false
      instance_name: "SearXNG"
      privacypolicy_url: false
      donation_url: false
      contact_url: false
      enable_metrics: false
  limiter.toml: |
    # Bot detection configuration with IP whitelist
    [botdetection.ip_limit]
    # Enable filtering but allow private networks
    filter_link_local = false
    filter_private = false
    filter_loopback = false
    
    [botdetection.ip_lists]
    # Whitelist your specific network range
    pass_ip = [
      "127.0.0.1/32",
      "10.0.0.0/8",
      "192.168.0.0/16"
    ]
    block_ip = []
    pass_searxng_org = false
    
    [botdetection.real_ip]
    # Handle proxy headers properly  
    x_for = 1
    ipv4_prefix = 32
    ipv6_prefix = 48

4. Create Encrypted Secret

Generate a secure secret and create the encrypted secret file:

# Generate random secret
NEW_SECRET=$(openssl rand -hex 32)

# Create secret file
cat > infrastructure/security/searxng-secrets/searxng-secret.yaml << EOF
apiVersion: v1
kind: Secret
metadata:
  name: searxng-secrets
  namespace: searxng
type: Opaque
stringData:
  SEARXNG_SECRET: $NEW_SECRET
EOF

# Encrypt with SOPS
sops --encrypt --in-place infrastructure/security/searxng-secrets/searxng-secret.yaml

5. Create Deployment

Create apps/searxng/base/searxng-deployment.yaml:

# File: apps/searxng/base/searxng-deployment.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: searxng
  namespace: searxng
  labels:
    app: searxng
spec:
  replicas: 1
  selector:
    matchLabels:
      app: searxng
  template:
    metadata:
      labels:
        app: searxng
    spec:
      containers:
      - name: searxng
        image: searxng/searxng:latest
        ports:
        - containerPort: 8080
        env:
        - name: BASE_URL
          value: "https://searxng.<domain.com>/"
        - name: SEARXNG_SECRET
          valueFrom:
            secretKeyRef:
              name: searxng-secrets
              key: SEARXNG_SECRET
        volumeMounts:
        - name: settings
          mountPath: /etc/searxng
          readOnly: true
        resources:
          requests:
            memory: "256Mi"
            cpu: "100m"
          limits:
            memory: "512Mi"
            cpu: "500m"
        livenessProbe:
          httpGet:
            path: /
            port: 8080
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /
            port: 8080
          initialDelaySeconds: 5
          periodSeconds: 5
      volumes:
      - name: settings
        configMap:
          name: searxng-settings

6. Create Service and Ingress

Create apps/searxng/base/searxng-service.yaml:

# File: apps/searxng/base/searxng-service.yaml
---
apiVersion: v1
kind: Service
metadata:
  name: searxng-service
  namespace: searxng
  labels:
    app: searxng
spec:
  selector:
    app: searxng
  ports:
    - name: http
      port: 8080
      targetPort: 8080
      protocol: TCP
  type: ClusterIP

Create certificate and ingress for HTTPS access:

# File: apps/searxng/base/searxng-certificate.yaml
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: searxng-tls-certificate
  namespace: searxng
spec:
  secretName: searxng-tls
  duration: 2160h    # 90 days
  renewBefore: 720h  # 30 days before expiration
  commonName: "searxng.<domain.com>"
  dnsNames:
    - "searxng.<domain.com>"
  issuerRef:
    kind: ClusterIssuer
    name: letsencrypt-production

# File: apps/searxng/base/searxng-ingress.yaml
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: searxng-ingress
  namespace: searxng
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - searxng.<domain.com>
    secretName: searxng-tls
  rules:
  - host: searxng.<domain.com>
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: searxng-service
            port:
              number: 8080

7. Create Kustomizations

Create apps/searxng/base/kustomization.yaml:

# File: apps/searxng/base/kustomization.yaml
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - searxng-namespace.yaml
  - searxng-settings.yaml
  - searxng-deployment.yaml
  - searxng-service.yaml
  - searxng-certificate.yaml
  - searxng-ingress.yaml

Create infrastructure/security/searxng-secrets/kustomization.yaml:

# File: infrastructure/security/searxng-secrets/kustomization.yaml
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - searxng-secret.yaml

8. Create Flux Kustomization for Secrets

Create clusters/production/flux-system/searxng-secrets.yaml:

# File: clusters/production/flux-system/searxng-secrets.yaml
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: searxng-secrets
  namespace: flux-system
spec:
  interval: 10m
  path: ./infrastructure/security/searxng-secrets
  prune: true
  sourceRef:
    kind: GitRepository
    name: flux-system
  decryption:
    provider: sops
    secretRef:
      name: sops-gpg

9. Update Kustomizations

Add SearXNG to clusters/production/apps/kustomization.yaml:

resources:
  # ... existing resources ...
  - ../../../apps/searxng/base

Add secrets to clusters/production/flux-system/kustomization.yaml:

resources:
  # ... existing resources ...
  - searxng-secrets.yaml

10. Deploy and Test

Deploy the changes:

git add apps/searxng/ infrastructure/security/searxng-secrets/ clusters/production/
git commit -m "Add SearXNG search engine with SOPS encryption and IP whitelist"
git push origin main
flux reconcile kustomization flux-system --with-source

Test the deployment:

# Check pod status
kubectl get pods -n searxng

# Test web interface
curl -s https://searxng.<domain.com>/

# Test JSON API
curl -s "https://searxng.<domain.com>/search?q=test&format=json" | jq '.results[0].title'

API Usage

The search API is accessible at:

https://searxng.<domain.com>/search?q=<query>&format=json

Example response:

{
  "query": "kubernetes",
  "number_of_results": 0,
  "results": [
    {
      "url": "https://kubernetes.io/",
      "title": "Kubernetes",
      "content": "Production-Grade Container Orchestration",
      "engines": ["google", "bing", "duckduckgo"],
      "score": 9.0,
      "category": "general"
    }
  ]
}

OpenWebUI Integration

Configure OpenWebUI to use SearXNG for web search:

Go to Settings → Web Search

Set Searxng Query URL to:

https://searxng.<domain.com>/search?q=<query>&format=json

Enable Bypass Embedding and Retrieval
Enable Bypass Web Loader
Enable Trust Proxy Environment

Security Features

IP Whitelisting: Only allows access from specified IP ranges
Rate Limiting: Prevents abuse while allowing legitimate API usage
SOPS Encryption: All secrets encrypted at rest using PGP keys
HTTPS Only: All traffic encrypted with Let’s Encrypt certificates

Monitoring

Check the application status:

# Pod status
kubectl get pods -n searxng

# Certificate status  
kubectl get certificate -n searxng

# Ingress status
kubectl get ingress -n searxng

# Logs
kubectl logs -n searxng deployment/searxng

Troubleshooting

Common issues and solutions:

Pod CrashLoopBackOff

Check logs: kubectl logs -n searxng deployment/searxng
Verify secret is properly encrypted and deployed
Check settings.yml format

403 Forbidden on API calls

Verify IP whitelist includes your network range
Test from whitelisted IP range

This deployment provides a production-ready, privacy-focused search engine perfect for AI tool integration while maintaining security best practices.

Overview#

Features#

Repository Structure#

Deployment Steps#

1. Create Application Structure#

2. Create Namespace#

3. Create Settings ConfigMap#

4. Create Encrypted Secret#

5. Create Deployment#

6. Create Service and Ingress#

7. Create Kustomizations#

8. Create Flux Kustomization for Secrets#

9. Update Kustomizations#

10. Deploy and Test#

API Usage#

OpenWebUI Integration#

Security Features#

Monitoring#

Troubleshooting#

Pod CrashLoopBackOff#

403 Forbidden on API calls#